Mental health is an important issue especially during COVID-19 times. The virus affects every person in some way thus the idea of the project is to take users ‘for a walk on Mars’ and help to forget what is happening on Earth at least for a few minutes. With the help of images produced by NASA we educate users by providing a relative image’s descriptions and most importantly invite them to hear Mars in the way we imagine it. The application combines art of sounds and images to provide a relief during a difficult time as well as show the extraordinary images of the red planet.
As most of the challenge topics are scientific or technology related, we used the topic “The Art of It All” as an opportunity to show that these areas and art can be mixed together and solve important problems at the same time giving emotional satisfaction.
We decided that we want to combine technology and art and develop a product which is accessible to every person. Our approach was to select already amazing art, which was provided by NASA, and find a way to experience it in a different way. Our focus was put to help users escape current situations for a bit and to reach people in their everyday lives. Users have 3 main options: focus, meditation or escape, which allow them to adapt to their current mood. It is known that different sound tones (brown/purple/white noise, different tone pitch, intensity, etc.) affect people differently thus for each option we tried to select a unique main sound that would put the person in a certain mood. By moving through the picture in the user's inputted intensity (left to right), our application computes and adjusts each sound note accordingly and provides a unique experience.
The tools used for this challenge were based on Javascript and Python. The mobile application was developed using React Native to allow the use of applications for different platforms. The api was developed in Python, which was used due to its many libraries which helped to analyze images (scikit-image, numpy) and extract required details as well as to create a sound from those details (pyo). Python also enables usage of Tensorflow or other ML libraries for more personalised and accurate music creation in the future.
The two of us (team OutOfBounds 3.0 members) have 0 background in music, thus we can proudly say that by thinking out of the box and going out of the comfort zone, we successfully created music from Mars images with lines of code. Having to introduce ourselves with different music tones and varying it with the help of programming in a few hours, we achieved every challenge we raised for this project from a music-programming perspective. However, there are still ways to make this application better and improve it for it to move to the next level.
The further improvements we are aiming to achieve after the challenge is improved sound rendering and increased amount of image qualities which are used to generate new sounds. Moreover, due to the small amount of time we limited our music processor to generate tones of main tones mentioned. This could potentially be extended in the future to filter each sound tone even more that will give even more unique experience for the user.
https://youtu.be/z-7jOGYURwA
Mars and Earth photos from NASA web pages listed below
https://eros.usgs.gov/image-gallery/earth-as-art