Spacetastic Koalas - Space Apps Challenge

Spacetastic Koalas| The Isolation Solution

The Isolation Solution

Social distancing policies enacted the world over during the COVID-19 pandemic have left many people socially isolated. Your challenge is to develop innovative solutions to combat social isolation.

"Senses" - Experiential multi-sensory stimulation as means to overcome social deprivation

Summary

Introduction of experiential multi-sensory stimulation presents great potential to overcome the behavioral and health performance challenges induced by social distancing, isolation and confinement. The proposed framework integrates various stimuli inputs, such as, auditory, tactile, visual, olfactory and gustatory, to enhance life experiences and re-enforce interpersonal connectedness. The proposed solution demonstrates great potential to benefit people in global communities and in outer space.

How We Addressed This Challenge

COVID-19 pandemic enabled the global communities to understand astronaut’s isolation and confinement experiences on long duration space flights. It has revealed the importance of personal relations and interactions with family and friends and how lack of thereof affects behavioral health and performance (3). These findings were further supported by the scientific literature that documented negative consequences of social isolation, including sleep loss, performance decrements, increased anxiety, fatigue, illness and mortality (4,5). It was the inability to participate in important life events, share experiences and create memories with the loved ones that had the most profound effect on the way isolation was perceived and experienced on an individual-level. As such, we came to realization that life is a combination of events and experiences that shapes personalities and establishes life paths. A more in-depth analysis of our experiences revealed that fond-memories are created through multi-sensory experiences, including smell, touch, sound, taste and vision. While technological advancements have led to creation of numerous tools to support video- and audio-communication, there wasn’t a tool to provide a multi-sensory experience to create or relive some of life’s favourite moments and this has led to the development of the proposed experiential multi-sensory framework, demonstrated within the prototype application called “Senses” (6).

The experiential multi-sensory stimulation proposed within this project is demonstrated within a framework schematically represented in Figure 1. The proposed framework utilizes an integrative approach to combine the various sensory inputs, including visual, auditory, tactile, olfactory and gustatory to create a vivid experience that can be relived or shared with others. As such, the person can choose to record or receive an experience, basic functionality of which is presented within the prototype.

Figure 1. An integrated multi-sensory experience through a combination of various stimuli, including visual, auditory, tactile, olfactory and gustatory.

To record an experience, a subject of interest would access the “Senses” application, which will collect multivariate data of the experience one is trying to record, including video, any relevant tactile stimulations, flavours, smells and sounds. This information will be supplemented with the relevant environmental data, including current weather conditions, local events, news and activities.

Visual and auditory data acquisition has been done through various techniques for over a century, as such the available methods and tools are well known and won’t be discussed in great detail within the context of this proposal.

Gustatory stimulation is the sensation of flavours through activation of various receptors on the tongue. Innovative technological solutions enable emulation of tastes through thermoelectrical stimulation of gustatory receptors (7). Thermoelectrical stimulation of the tip of the tongue by the temperature range of 20 to 35C induces transient sensation of sweetness, while the cooling to temperatures below 20C introduces a sour taste. As such, our proposed framework incorporates the five main taste categories, including sweet, salty, sour, bitter and umami. The user will be required to choose one of the taste categories, the sensory stimulation of which will be encoded through machine learning algorithms, in accordance with the specific chemical and thermal properties of the food, schematically represented in Figure 2 (8,9).

Figure 2. The concept of machine learning algorithms to encode food tastes in accordance with their respective thermal and chemical properties, adapted from (8,9).

Tactile stimulation is a sensation of external objects or forces experienced through physical contact with the body. As such, we propose the use of smart textile sensors, such as the ones integrated within HaptX Gloves that provide realistic sensation of the shape, texture and motion of virtual objects (10). The proposed tools could be used to simulate a touch or a hug someone wants to share, tactile stimulation of which can be reproduced through an integration of pressure sensors, utilizing an array of small pneumatic actuators and microfluidic air channels, which employ a feedback mechanism, concept of which is represented in Figure 3.

Figure 3. Conceptual representation of tactile sensory stimulation.

Olfactory stimulation greatly enhances various sensory stimuli and user experiences. There are ten main types of odours that can be perceived by human olfactory receptors and they include fragrant, woody/resinous, fruity, chemical, minty/peppermint, sweet, popcorn, lemon, pungent and decayed odours (11). We propose the use of fabricated fragrance microcapsules that can be integrated into a user's headset to stimulate the desired controlled release of olfactory sensation. This approach demonstrates great potential to improve user experience, behaviour and health performance through various forms of aromatherapy (11,12,13).

Environmental data has been identified as an important component to supplement experiential multi-sensory stimulation, helping to contextualize the atmosphere and the environment of an event or experience. The user experience can be enriched through meteomatics data, such as the weather conditions, including temperature, precipitation, humidity, etc. This data will be incorporated into the user experience profile by using GPS position and the time of the recorded experience/event. The data can be retrieved from meteomatics API (14). Lastly, local news and events provide a valuable source of information to contextualize the environment the user is in and identify external factors that may impact user experience. As such, important themes from the reliable news outlets will be synchronized with the user experience entry.

Case Scenario is presented in the section to follow to provide the reader with the context of the proposed solution.

Case Scenario:

Emily is turning five. Her father, Andy, is an astronaut, participating in an exploratory-class mission on the ISS to support scientific advancements for the humankind. On Emily’s birthday both Andy and the little girl are emotionally deprived, as they are unable to celebrate this important life event together.

Existing solution: Andy Skypes with Emily to wish her a Happy birthday, while longing for the feelings and the atmosphere experienced during the celebration.

Proposed solution: Andy receives an experiential multi-sensory stimulation. He is watching a video of Emily blowing the birthday cake candles, supplemented with the sound of her contagious laugh, the smell of the candles she just blew, enhanced by the sweetness of the cake she is about to eat. He can experience the strong hug she is about to give him. All of which is supplemented with the local metereomatics data, that helps the user to immerse in the mindset and experiences of people celebrating Emily’s birthday.

How We Developed This Project

Motivation:

Emergence of the novel corona virus (COVID-19) has led to significant changes in all aspects of human life. The rapidly spreading corona virus outbreak was declared as a global pandemic that closed the international borders and paralyzed much economic, industrial and social activities. As the number of active COVID-19 cases continued to rise exponentially, on a daily basis, the local, provincial and global authorities were called upon to enact strict quarantine measures in an attempt to reduce the spread of disease. In fact, physical distancing was identified as one of the most effective ways to limit the spread of the novel corona virus (1). Physical distancing introduced profound changes to people’s daily routines, by minimizing social interactions, participation in social activities and human to human contact (1). Physical distancing introduced the experience of social isolation on a global-scale, raising awareness of the various physiological and psychological implications induced by limited interpersonal contacts, most aspects of which astronauts are familiarized with. Astronauts are exposed to considerable spaceflight-induced psychological effects, attributed to restricted mobility, enforced limited interpersonal contacts, reduced interaction with family and friends, all of which significantly impact behavioural and health performance.

Social deprivation has led to realization that we as humans are highly dependent on quality interpersonal contacts to maintain our physical and psychological health and well-being. Our lives and personalities are shaped by our experiences and memories, which have been significantly affected by imposed isolation. As Mike Barratt pointed out “I think that the only times I ever felt isolated up there was when I knew there were events going on on the ground that I really wanted to be a part of. Mostly family events. Other than that, not really” (2).

As such, our team was inspired by the behavioural and health performance challenges imposed by social distancing and isolation and ways, in which meaningful connections and experiences can be fostered through multi-sensory stimulations. Our team has also been motivated by the great potential to develop solutions to benefit people in global communities and in outer space to improve the quality of life, physical and cognitive performance, as well as enhance the sense of interpersonal connectedness.

Methodology:

Design thinking methodology was used to develop this project. To arrive at the proposed solution, five process steps have been followed, more specifically, empathize, define, ideate, prototype and test. Initial process step was to empathize with the population of interest, drawing on the feelings and experiences of global communities during imposed quarantine and the various aspects of social deprivation. Then, the spectrum of experiences and feelings was analyzed to understand what the proposed solution should provide to the people of interest. The ideation step was based on exploration of possible solutions, as a result of which, the concept of experiential multi-sensory stimulation has been identified as potential means to foster meaningful connections, share memories and experiences in such a way, as to improve the quality of life, behavioural and health performance, and re-enforce a sense of meaningful interpersonal connectedness. Subsequently, the proposed prototype was designed, developed and tested, highlighting existing limitations and opportunities for future implementation.

Use of Space Agency Data:

Open source space agencies data, more specifically CSA and NASA data repositories, have served as a foundation for the project’s research questions and evidence-based development of the solution. Social isolation and space section within the NASA Human research program has set the context of social isolation while establishing parallels between space travel and experiences of global communities during the COVID-19 pandemic. CSA sources on physical distancing were used to contextualize the isolation environment. Physiological and psychological effects imposed by confinement and isolation were explored within the NASA study on ‘Characterization of Psychological Risk, Overlap with Physical Health, and Associated Performance in Isolated, Confined, Extreme (ICE) Environments’. NASA’s Human Research Program archive identified important themes in behavior and health performance that will be noted in the proposed solution. Said solution highlights the interconnectedness of biological systems and the various environmental stimuli.

Tools and Coding Languages Used:

First React Native, a front-end development app, was utilized to develop a prototype of the proposed solution and demonstrate the basic functionality of the experiential multi-sensory stimulation. The first react native has a simple, but thorough development interface that will allow the team to present the proposed concept within the designed and developed prototype application named ‘Senses’. The proposed application “Senses” will, in theory, enable users to record and receive experiences through an integrated framework of multi-sensory stimulations. The proposed interface will integrate environment/home simulation components and behavior-monitoring technology.

Challenges and Successes:
We have a wonderful multidisciplinary international team, including team members from Brazil, Canada, Germany and the USA. Initially, we were a group of six, but due to some prior commitments two team members, unfortunately, had to leave. We thank them for their contribution in helping us understand the context of isolation from their personal experiences during the pandemic.

SpaceApp COVID-19 Challenge enabled us to e-meet one another, as we haven’t met prior to the Hackathon. The time zone differences posed some challenges in coordinating the activities but we have managed to successfully overcome them. Being a multidisciplinary team we had some challenges in defining a clear scope of the solution we wanted to propose. This has been a great learning opportunity, where we were able to learn from one another, share our knowledge and contribute our expertise in developing the solution for the betterment of behaviour and health performance outcomes around the globe and in outer space.