Conversation Fuel Designing a virtual reality exposure therapy tool with counsellors that supports remote learning of conversation skills for clients on the autism spectrum. Conversation Fuel: Designing a Virtual Reality Exposure therapy tool with counsellors that supports remote learning of conversation skills for clients on the autism spectrum. By Binoodha Kunnath MDes 2021 Supervisor: Jonathan Aitken A Thesis Project submitted in partial fulfillment of the requirement for the degree of MASTER OF DESIGN Emily Carr University of Art + Design 2021 © Binoodha Kunnath Acknowledgment Over the past two years, many people have contributed immensely to the fruition of this thesis project. I am sincerely grateful for their support and encouragement. First of all, I would like to express my sincere gratitude to my family for their unconditional love and support throughout the process. It was a challenging year, and their confidence in me helped me think rationally to navigate the challenges that arose during the research. I want to say a special thank you to my supervisor, Jonathan Aitken. His support, guidance and overall insights in this field have made this an inspiring experience for me. Without his wise guidance, this project would not have been the same! I would also like to thank all the counsellors who lent me their valuable time from their busy counselling schedules to provide me with professional insights that shaped the design outcome of this thesis project. Their enthusiasm to provide continued support for research in the field of autism is inspirational. Besides my participants and supervisor, I would also like to thank my internal reviewer, Maria Lantin, for being a source of inspiration in my field. Finally, I would like to thank my instructors at Emily Carr University, the MDes 2021 cohort and the REB for influencing and challenging my design practice. Glossary of Terms VR Exposure Therapy Virtual reality exposure therapy (VRET) is an altered form of behavioural therapy using virtual reality technology. Instead of simply using the mind’s eye, or simulating the actual environment used to recreate the troubling situation to be processed, VRET allows an individual to place themselves in a virtual environment that evokes the sights, sounds, smells and virtual feel of an environment which closely resembles a therapeutic situation. This allows for the safe processing of memories, feelings, and emotions in a controlled setting. High Functioning Autism High-functioning autism (HFA) is an autism classification where the individual exhibits no intellectual disability but may exhibit deficits in communication, emotion recognition and expression, and social interaction. Support tool Support tool are softwares, web analytics tools or other technology used by counsellors in their clinical practice to monitor, maintain or improve the therapy session. Biofeedback Biofeedback is the process of gaining greater awareness of many physiological functions of one's own body, by using electronic or other instruments as sensors. Teletherapy Teletherapy refers to therapy that is provided by a therapist via a secure video connection. Gameplay Gameplay is the pattern of play defined through a game’s rules, the connection between the player and the game, challenges and overcoming them, plot and player's connection with it. AI avatar AI avatar, also known as a digital avatar, are human-like bots created by AI-powered technology to increase human interaction. While digital avatars don't only have a humanoid appearance, they can also communicate with people with Natural Language Processing (NLP) algorithms. Natural language processing is a subfield of linguistics, computer science, and artificial intelligence concerned with the interactions between computers and human language. Contents 1 Abstract and Keywords 01 2 Introduction 02 3 2.1 Thesis statement 2.2 Project objectives 2.3 Context and framing 02 2.4 Precedents 06 06 2.4.2 immersive digital experiences as a tool—Case studies 07 Research Approach 3.2.1 Generative Research approach 09 09 09 09 3.2.2 Alternative source analysis 10 3.2.3 Interviews 12 3.2.4 Co-design workshop 13 3.2.5 Analysis of data 16 3.3 Design Outcome 19 3.3.1 Concept development 19 3.3.2 Game strategy 23 3.3.3 Wireframing 30 3.3.4 Preliminary user tests and feedback 35 Reflections 4.1 Future design directions 4.2 Design tensions and limitations 5 04 2.4.1 Review of existing counselling support tools and their use 3.1 Ethics and informed consent 3.2 Research methods 4 03 Appendix 5.1 Works cited 5.2 List of Figures 5.3 Primary Research materials 38 38 38 40 40 42 43 1. Abstract and Keywords Keywords: High-functioning Autism, Virtual Reality, Role-playing, Generative Research, Conversation Challenges Many young adults with high-functioning autism spectrum disorder do not receive adequate professional support to develop communication skills during puberty. As a result, these individuals face poor transition outcomes in critical areas like developing social connectedness through conversational skills. Previous research investigating these outcomes' psychological impact has shown that these individuals often expressed their desire to build friendships despite their social skill difficulties. These unique challenges have intensified during the pandemic owing to a further lack of in-person interaction opportunities. This project details the everyday challenges these individuals face based in Canada through the lens of their counsellors. The counsellors were able to provide in-depth information about what they expect in a tech-supported tool that can help their clients overcome challenges in communication and practice self-regulation through role-playing. It was important to study this subject with the counsellors' help to incorporate the right self-regulatory exercises into the tool. I used generative design research to understand the current support tools used by counsellors in their practice to develop their clients' conversational skills who self-identify themselves as high functioning ASD. Many autism interventions during in-person counselling sessions depend on playful interactions and games such as flashcards and board games, grounded in findings on the positive influence of play on learning. Previous studies have investigated the potential of interactive technology for autism education, showing that it enhances the learning experience and reduces the need for continuous assistance. However, these strategies are rarely used as direct input for designing tech-supported tools that counsellors could use in their virtual counselling practice and acknowledge the variability in the tool functionality based on different individuals with high functioning autism. The tool is focussed on embracing those differences and allowing the counsellors to tailor the exposure therapy based on individual needs. The research so far has indicated the possibilities of incorporating virtual reality platforms in tandem with biofeedback as a virtual reality exposure therapy. I applied the research results as a guide to building a remote virtual environment to support exposure therapy. The virtual reality platform could be an adequate interface to improve the virtual counselling sessions by providing in-tool options to customize the scenarios, build avatars, observe body language and incorporate assessment goals using biofeedback into the experience. These scenarios and exercises are based on the input by the counsellors during the co-design workshop. Abstract and Keywords / 01 2. Introduction 2.1 Thesis Statement Designing a virtual reality exposure therapy tool with counsellors will support remote learning of conversation skills using role-playing techniques for clients on the autism spectrum (High-functioning). Introduction / Thesis Statement / 02 2.2 Project Objective This project's overall objective was to develop a tool that could support Clinical Counsellors in helping their clients on the high functioning end of the Autism Spectrum learn conversation skills to support social relationships. The first objective was to understand the challenges young adults on the autism spectrum face while trying to develop social relationships. This information would significantly support scaling into a specific challenge to be addressed by the designed tool. To ground this work from a counsellor’s perspective, the second objective was to learn more about the tools counsellors currently incorporate in their therapy to help develop conversation skills for their clients. This also included understanding the limitations of these support tools. Further, it was necessary to understand the assessment goals used by the counsellors to support the continual growth of skill acquisition of their clients. These assessment goals influenced the VR tool's framework and further helped improve the functionality by incorporating the right biofeedback model. Introduction / Project Objective / 03 2.3 Context and Framing This project necessitated first understanding the communication challenges faced by adults living with high functioning autism spectrum disorder and their impact on building social relationships. High-functioning autism isn’t an official medical term or diagnosis. However, it is an informal term commonly used to talk about people with an autism spectrum disorder (previously known as Asperger syndrome) who can speak, read, write, and handle necessary life skills. The term high functioning, used in this thesis project, in no way wishes to undermine other individuals on the autism spectrum, and I also fully acknowledge that all individuals on the spectrum can have a varying degree of therapy requirement during their clinical sessions. Extensive evidence demonstrates that social skills acquisition and generalisation of skills often form the most critical challenges for children and adolescents living with high-functioning autism spectrum disorders (Faridi 2017; Khosrowabadi 2017). A more limited body of literature demonstrates that social skills deficits remain prevalent for young adults (ages 18–23) with ASD (Barnhill 2007; Howlin 2000). here is currently an evident need for more research to support these transitioning adults in improving their social skills. Social deficits in adults with ASD may exacerbate or lead to problems with friendships and romantic relationships, daily living, and vocational success (Barnhill 2007; Howlin 2000). The challenging aspects of ASD appear most significant for adolescents and young adulthood, possibly due to the greater salience and complexity of peer relationships (Tantam 2003). This thesis project uses a generative research approach. It proved challenging to directly contact individuals on the spectrum at a very early stage of this research. While I intended my design outcome to be heavily influenced by these individuals, I had to acknowledge that I didn’t have the skills needed to interact with the community while dealing with conversations that can induce stress and trauma. For this reason, it was beneficial to have access to seven Clinical Counsellors working with ASD clients in Canada. The counsellors were able to provide a window into the lives of different clients on the autism spectrum and share their extensive knowledge on the subject. The analysis of these interviews indicated a gap in the existing tools used in their practice to develop their clients' communication skills. I began by conducting in-depth interviews with the counsellors in early May 2020. At this point, COVID-19 had impacted the way counsellors were engaging with their clients—especially in the way they had to shift their clinical practice to be able to work remotely. Teletherapy became increasingly popular, but counsellors found it challenging to observe specific assessment measures through video conferencing effectively. Some of their concerns included not being able to gauge their clients’ body language during virtual sessions. Bricker and Bell (2012) indicated that social and communication skills learned by individuals with ASD in a virtual environment like Massive Multiplayer Online Role-playing Game (MMORPG) might generalise to another environment, including face-to-face situations. Bricker and Bell's study led me to consider the possibilities of using virtual reality as a training tool for individuals on the spectrum to blend behavioural science with gaming and role-playing into a new kind of counselling experience. Such simulations could target social situations for individuals on the spectrum. Additionally, counsellors could observe this virtual interaction as well as collect non-visual cues through neuro- or biofeedback. Introduction / Context and framing / 04 There has been substantial research conducted to test the benefits of VR exposure therapy for individuals on the spectrum by allowing them to practice difficult or individually challenging social interactions in a less-anxiety producing platform (Kandalaft et al., 2013, Maskey et al., 2014, Parsons and Mitchell, 2002, Wainer and Ingersoll, 2011). These VR experiences used in therapy are integrated with immersive environments, real-time graphics, body tracking devices and other sensory inputs to allow the clients to engage with VR in a safe space. My interviews with counsellors established roleplaying as a standard method used in developing communication skills. This prompted me to explore VR exposure therapy possibilities using role-playing as a game function within the same challenging context of a real-life environment. While role-playing has been well-established as an effective methodology to build social skills (Corsini, 1966), there is less research into how this might transition into a virtual space. As much as technology and artificial intelligence are human-made, there are gaps in understanding its responsiveness that we are yet to perfect. For this reason, the involvement of a counsellor in the counselling process during VR exposure therapy will be beneficial. The counsellors will be able to step in and guide the interaction if they are not satisfied with the AI avatar's responsiveness and coherence. Using Artificial Intelligence in chatbots to have conversations in ‘human language’ can be challenging and require machine learning algorithms to improve functionality. The technology learns about the user’s interests and habits over time, even adopting one's linguistic syntax and conversational quirks in much the same way that a close friend might speak in real-life scenarios. However, it is important to look at approaches like ‘Prompt Engineering’ that allows users to select temperature settings for their AI’s responses. A low-temperature setting means the AI will put together words that it has often seen together before, taking few risks and causing few surprises (Strickland, 2021). I had to also look at precedent work to study the impact of learning in the virtual space and the ability for individuals to generalise it into actual practice in their daily lives. After an initial interview, the counsellors got further involved in a series of co-designing workshops where they shared their knowledge about various tools they currently use to measure their clients’ progress during therapy. We looked at multiple scenarios where communication is presently challenging for their clients and explored various self-regulatory exercises that can benefit them. Introduction / Context and framing / 05 2.4 Precedents 2.4.1 Review of existing counselling support tools and their use The counsellors predominantly use role-playing as a method to improve conversational skills in the clients. These role-playing sessions are supported by some card games and board games repurposed to fit the counselling needs. The counsellors also incorporate a lot of video examples to train their clients to identify appropriate conversational skills and body language. However, these resources are not available at a single source; hence, the counsellors have to spend a lot of time finding these resources that support their practice. Over the last decade, there have been many support tools that have been incorporated into counselling practices that support individuals with high functioning autism. While some of these tools were specifically designed to support autism therapy, the others have been repurposed by the counsellors to suit their practice. Neurofeedback is one of the most popular technologies used in clinical counseling to help change unhealthy or undesirable brainwave activity into normal, healthy, organised activity. Research has suggested that neurofeedback provides an effective intervention to address the core symptoms and problems associated with autistic spectrum disorder (Coben et al., 2010). While Neurofeedback therapy is gaining popularity, the devices' cost is still high and increases the counselling session's cost; Everyday Speech is another popular social-emotional learning platform that provides counsellors with a video-based Social-Emotional Learning (SEL) curriculum. The platform also provides activities, visuals and games relevant to the curriculum. The platform is designed to cater to individuals in elementary to high school; Cogmed is another clinical tool designed by cognitive neuroscientists and psychologists. It was designed for counsellors and psychologists working with individuals with attention and learning difficulties. It provides effective intervention based on the principles of neuroplasticity. The training is individualized by an algorithm that presents the tasks in a rotating schedule and adapts to the user's capacity level in real-time. Introduction / Precedents / 06 2.4.2 immersive digital experiences as a tool—Case studies The following case studies inspired the trajectory of this research project. They include three interactive digital experiences: Centervention (2015) - An online game to improve behavior and social and emotional skills for students in elementary and middle school designed by 3C Institute; Stressjam (2019) - A virtual reality game by Jamzone that uses biofeedback to change stress mindset; Business Life (2016) - A virtual reality game by Samsung to help the player overcome their fear of public speaking; These three case studies explore various nuances that shape virtual reality exposure therapy and portray various creative processes that inform the immersive experiences. Through these case studies, the projects all explore the possibilities of learning through gameplay while generalizing the lessons from virtual space to real-life situations. Centervention (2015) - An online game to improve behavior and social and emotional skills for students in elementary and middle school designed by 3C Institute Centervention (2015) has an online, interactive program, Stories in Motion , where elementary school students with ASD can create personalized social narratives around common school scenarios. In this program, the students can create a character to represent themselves and then choose from twelve different topics to create a personalized narrative using dialogue and thought options. Some of the topics discussed include anxiety, bullying, cooperating, coping, friendship, joining in, listening, personal space, perspective taking, sportsmanship, stop and think, and transitions. According to DeRosier, a clinical psychologist and CEO of 3C Institute, “A game provides a safe place to practice; you can try things out and see what happens, without any negative social consequences in the real world.” Stressjam (2019) - A virtual reality game by Jamzone that uses biofeedback to change stress mindset; StressJam (2019) is a Virtual Reality (VR) game that uses biofeedback to train stress-mindset and help the player improve their stress regulation. The game allows the player to play in real-time guided by their own stress system. The VR headset transports the player to an exotic island where they carry out different challenges. The game is personalized by real-time sensors that measure the player’s heart rate variability and teaches the player to use their stress system to advance in the game. The game provides the player with a game coach as well as the option to play by themselves. Introduction / Precedents / 07 Business Life (2016) - A virtual reality game by Samsung to help players overcome their fear of public speaking Business Life is a VR app developed by Samsung to help players overcome public speaking fear.. In this game, the player learns to speak comfortably in the five most relevant scenarios to work life. These scenarios include job interviews, business lunches, team meetings, management presentations, and a job fair. The game uses the player’s voice volume, speaking pace, eye contact, and heart rate to gauge their progress. According to Samsung, the training program helped more than 87.5% of participants reduce their anxiety level by 23.6%. Moreover, the participants who had a fear of public speaking saw an average reduction of 18.7% in anxiety. Although nestled in different use case scenarios, the three projects consider how VR could be used to impart skill development effectively. All projects looked beyond conventional VR games and incorporated biofeedback to improve game personalization. These case studies helped to understand how biofeedback can be incorporated in exposure therapy to improve conversation challenges through gameplay, making the experience unique for every high functioning individual working with their counsellor. However, this thesis explores further the areas that talk directly to the experience of individuals with high functioning autism and conversational challenges. The counsellors interviewed for this thesis project displayed a wide range of adaptation to technology. Some used paper-based tools, while others were more technologically driven. However, the counsellors who wished to include more technology in their practice found it difficult to discover tools specifically designed for autism therapy. While many of the above projects can be beneficial, the VR exposure therapy tool to help individuals on the spectrum improve their conversation skills needs more personalized experiences that support counselling needs. Based on the theoretical knowledge gained through secondary research, the thesis project further probed a more contextual inquiry through the generative research process. Introduction / Precedents / 08 3. Research approach 3.1 Ethics and Informed consent This thesis project was approved by the Emily Carr University Ethics Committee (see Appendix 5.3). Six counsellors based in Canada were approached to participate in this thesis project based on their prior experience working with clients on the spectrum. The counsellors also already incorporated various support tools in their clinical practice. 3.2 Research Methods 3.2.1 Generative research approach The reason for adopting a generative research approach for this project was to develop a deeper understanding of the challenges faced by individuals on the spectrum to find opportunities for design innovation. The name ‘generative tools’ refers to the creation of a shared design language that designers/researchers and other stakeholders use to communicate visually and directly with each other. The design language is generative in the sense that with it, people can express an infinite number of ideas through a limited set of stimulus items. Thus, the generative tools approach is a way to explore ideas, dreams, and insights of the people who will be served through design. (Sanders and Stappers, 2012) Theories of generative design discussed in the book Convivial Toolbox (Sanders and Stappers, 2012) provide an insight into various tools that can be used to generate ideas during co-creation workshops with counsellors and other experts in the field of autism. According to Watson (2019), another key feature of generative research is inclusiveness, including adapting the research environment, methodology and dissemination routes to permit the widest and most accessible engagement or engagement from specific groups. While this project initially intended to conduct the research activities in-person, the current pandemic scenario required the project to look at alternatives. The collaborative research tools had to be designed to be simple enough to use by non-designers while keeping in mind the target audience's design sensitivities. Research approach / Ethics and informed consent / 09 3.2.2 Alternative source analysis Having no direct access to the individuals on the spectrum posed an important challenge at this project's onset. Fortunately, there are many resources available on public platforms such as blogs that provided a window into their daily conversation challenges. Many individuals on the spectrum use online communication tools like blogs, podcasts and forums as mediums to communicate their daily experiences to support other individuals like themselves in the community. These forums fostered an active community that allowed the project to follow their trail of influence in adopting new ideas and dialogues in ways simply not available through the counsellors' lens. These blogs enable freedom of expression in their writers, which may not be readily available in other media (Hull 2007). Hence, they serve as an interesting alternative method for ethnographic observations. These blogs and videos were analysed to observe themes around conversation challenges in the community by noting down keywords, motivations, references across time, attitudes and feelings in the content. Further, some excerpts from the blog content were used to generate journey maps based on the author’s blog entries. Figure 1: Blog analysis table synthesized based on data from various blogs run by people on the spectrum. Carefully chosen outfit. Cannot function without a plan. Deviating from morning routine causes stress ROUTINE Alarm sounds cause pain No meal till lunch because of medications. A lot of medication for someone that age. Wears headphones to cancel out sounds. Likes listening to songs but compulsively changes them without completing a track Love my headphone, It’s like my shield. Hate noises like chewing, bells and rustling papers Faces rude remarks at school, like someone saying “thats autistic.” Follows instructions but gets a headache following math and cannot comprehend sarcasm and jokes. INSTRUCTIONS BULLYING Spends time with family and then goes back to bed when my mother says ‘bedtime’. SOUND SENSITIVITY Figure 2: Journey mapping of a teenager living with ASD created based on content from www.themighty.com/2018/05/day-life-autistic-teen/. Research approach / Research methods / 10 Has the same breakfast everyday but needs to be prompted to begin eating. Receives IEP (Individualized education program) Loves his metal beads and has many of them PREDICTABILITY My kid thrives on predictability Overload of emotions caused due to puberty. My kid is confused, moody or frustrated. Indulges in stimming.He loves his metal beads that he keeps playing with. He wants his meal on time. Throws a tantrum if the meals are not on time. Relaxing time to lower his energy before bedtime STIMMING Puberty is a difficult time for a transitioning adult on the spectrum SOCIAL INTERACTION I read his face when he returns from school to know his mood for the rest of the day Receives home therapy where they teach him how to interact socially PUBERTY Figure 3: Journey mapping of a parent of a teenager living with ASD created based on content from www.parents.com/health/autism/parenting/day-to-day-raising-autistic-child/. Research approach / Research methods / 11 3.2.3 Interviews Figure 4: Affinity map analyzing the results of the interview session with the counsellors. Six counsellors practising clinical counselling in Canada and one psychology researcher based in the Netherlands volunteered to be part of this study. All the counsellors had prior experience working with clients on the autism spectrum. Upon receiving the ethics approval, the experts were contacted to participate in an in-depth semi-structured interview to discuss the conversation challenges individuals on the autism spectrum face. They were probed further to talk about the challenges faced commonly by their clients on the spectrum and discuss the current tools and methods used in their practice to help them navigate those challenges. During the course of the interview, the counsellors revealed ‘social anxiety’ to be one of the biggest concerns of their clients. According to Marie Watler (Personal communication, July 8, 2020), a Registered Clinical Counsellor based in Canada, her clients find conversations challenging because they don’t necessarily understand literal language. Another prominent discussion during the interviews was around the challenges faced during virtual counselling sessions. While the counsellors found it challenging to adapt to virtual sessions during COVID-19, they also found creative ways to incorporate online game platforms like ‘Centervention’ into their sessions to make them more effective. Unfortunately, the video conferencing tools had limitations in allowing the counsellors to observe their client’s body language. The results of these interviews were analyzed using affinity maps to reveal common themes like intervention goals, current counselling methods, assessment methods, tool expectations, current challenges in practice, current tech tools used by the counsellors and ASD associations the counsellors worked with. Research approach / Research methods / 12 The interviews were invaluable in understanding the counsellors' expectations around tools that might help them in their practice. These expectations were rightly expressed in the best interest of their clients. Some of the expectations included: providing clear instructions for the clients during all stages of the immersive experience; allowing room for modularity in the tool; having assessment methods incorporated into the tool; and finally, incorporating role-playing in a ‘safe’ immersive environment as a method to provide this training. One of the experts, Fallon Elkerbout (Personal communication, July 17, 2020), co-designed a ‘VR Autism Experience’ with the Creative Director of 360Creators, Ronald van der Weijden. This Virtual reality experience allowed the user to step into the world of an individual with autism. She believed that taking the perspective of others in VR produced empathetic behaviours in people immediately after going through the experience. The counsellors mentioned that VR-based techniques are ideal for exposure therapy, as the sense of presence experienced in VR allows the patient to immerse the patient in the feared environment tailored to match specific aspects of their fear structures to activate and modify these structures. VR exposure therapy has also been evaluated in many research pieces studying its effectiveness as a clinical counselling tool (Boeldt et al., 2019; Powers et al., 2008; Glanz et al., 2003; Wald et al., 2000). Based on the interviews with the experts and secondary research up until now, It was evident that a VR exposure therapy would provide the best immersive ‘safe’ environment for individuals on the spectrum to practice and improve their conversational skill remotely with the help of their counsellors. Further, a co-design activity with the counsellors was used to holistically investigate the VR exposure therapy's gameplay and support the design. 3.2.4 Co-design workshop Co-design is a methodology that enhances non-designers participation in the different activities that compose a design project (Sander 2006, Brandt 2005), especially in the preliminary phase to avoid generating ideas that may be ill-suited to the user. This project facilitated a co-design workshop with four experts who volunteered to be part of this research. This workshop was conducted online using an online collaborative platform called Miro, which allowed the experts to partake in a scenario building exercise. The Miro board contained a recap of the results from the previous interview sessions with the counsellors. Since the counsellors took part in this research activity separately, the collaborative board helped the participants view other counsellors' experiences. These results were generalized to maintain anonymity. The counsellors were provided with a private link to the board before the day of the activity. This board included the research question and activity details to clarify the discussion topic. After analyzing the interview results, which inspired the development of a VR exposure therapy as the design outcome, this activity was designed to gain more in-depth knowledge about different scenarios where conversations are challenging for individuals with high functioning autism. The activity also probed to understand the exercises employed by the counsellors to navigate these conversation challenges, the technology the counsellors currently used or wish to use in the future and the biofeedback measure that help counsellors assess the progress of their clients. The counsellors were also asked if they preferred to be actively present in the VR experience as an avatar or oversee the experience while guiding the client. Research approach / Research methods / 13 The counsellors were asked to select from seven pre-conceived challenging conversation scenarios based on the interview analysis. They were asked to generate four ideas using the pre-designed scenarios, the choice of presence in VR, and their expert knowledge about the best exercises to navigate those challenging scenarios, technology best suited for the scenario and biofeedback measures to assess the progress. They were also encouraged to elaborate on the exercises they currently provide in their practice and how those could be translated into the VR experience. Figure 5: The Miro board for the co-design activity with the counsellors. Research approach / Research methods / 14 Ultimately, this activity helped me understand the most challenging scenario for individuals on the spectrum to have conversations with other individuals based on the scenarios chosen by the counsellors to discuss further. The activity was able to gauge more personal experiences and creative counselling methods used by these counsellors in their practice while developing the use case scenarios for the VR experience. Figure 6: A closer look at the instructions for the activity to build the game strategy for the VR exposure therapy. Research approach / Research methods / 15 3.2.5 Analysis of data The alternative source analysis (section 3.2.2) had broadly indicated the real-life scenarios that were challenging for individuals on the spectrum. The resources helped identify and establish the problem space that the design outcome hoped to support. However, these blogs and videos had limited insight into the individual’s counselling experiences, specifically during COVID-19. It was also challenging to find information regarding exercises or counselling activities to develop conversation skills for individuals on the spectrum. Nevertheless, the online resources proved invaluable in having a broader understanding of the area this thesis project wanted to focus on. The resources also supported the framework for the co-design activity and interview questionnaire. The interviews provided in-depth knowledge about the various concerns for clients on the autism spectrum by clinical counsellors based in Canada. Most clients approached counsellors with concerns regarding their social interactions. Lack of social skills restricted many areas of their lives, and they approached the counsellors after having tried and failed alternate resources. This was essential in understanding the significant role of the counsellors in this VR exposure therapy. Their expert knowledge in the field was irreplaceable and had to be incorporated into the tool to make it more effective. The interviews also helped deconstruct some psychological terminologies and strategies specific to high functioning autism that would affect this tool's design and function. During the interviews, the experts also provided a plethora of resources available to them to benefit the VR exposure tool development. Some of these resources included a study on micro-expressions by Paul Eckman (2004), clinical strategies for developing conversation skills, books and flashcards used to develop conversation skills during counselling sessions, and psychological assessment scales to measure progress and information about how technologies like neurofeedback are currently incorporated in their practice. The experts also acted as a gateway to a larger resource pool, i.e. organizations and associations working for the autism community. These organizations, like Pacific Autism Family Network (PAFN), provide a support network for individuals with Autism Spectrum Disorder and their families across the province (British Columbia). The counsellors also utilized organizations like PESI Inc (1979) as an information hub where they discover new technology tools they can incorporate into their practice. Since the interviews were conducted during June, when the counsellors were transitioning their practice into the virtual space, these interviews helped me understand some challenges the counsellors faced in providing therapy virtually. All the experts responded differently to the transition, and their personal journey was critical to the design of VR exposure therapy. While the final tool would benefit individuals on the spectrum, this thesis project also had to acknowledge the counsellors' user experience who would be using this tool with their clients. The co-design activity delved deeper into the gaps of knowledge regarding clinical counselling for individuals on the autism spectrum. The activity was designed to involve the counsellors in design plus discussion format while using an online collaborative tool. Through a card sorting exercise, the experts recognized Small talk, Conversation starters, Active listening and Romantic conversations as the key area of conversation that their clients found challenging. Based on these scenarios, they discussed in detail the various exercises they provide to help self regulate their clients' social anxiety during these scenarios and strategies they could practice to improve their conversation skills. While they all acknowledged that virtual reality's immersive environment might prove beneficial, they also looked at how biofeedback could be incorporated into these scenarios to provide better counselling results. Some of their preferred biofeedback tools included Heart Rate Variability (HRV) sensors. Research approach / Research methods / 16 Eye-tracking devices and Voice AI (Artificial intelligence). Finally, this exercise also helped in situating the autonomy of the counsellors in the VR experience. They preferred to be the observer of the interaction between their client and an AI-supported avatar. The ability to be an observer, in their opinion, reduces the complexity of their client having to be distracted with more than one guiding presence in the gameplay, as well as provide the counsellors with an opportunity to provide better feedback. Research approach / Research methods / 17 Figure 7: Analyzing the results from the co-design workshop and secondary research Research approach / Research methods / 18 3.3 Design Outcome 3.3.1 Concept development The results from the co-design workshop indicated the importance of four key components of a conversation. This included; Small talk, Active listening, Conversation starters and Romantic conversations. While all four categories overlapped each other in certain instances, for the most part, they all had their own distinctive challenges and self-regulatory exercises. The stages of a conversation stood out as an excellent reference for analysing progression in VR therapy. For this thesis project, the design development will focus on ‘Small Talk’ as the VR experience's entry point. The Counsellors were able to identify various paradigms within building conversation skills to improve small talk for their clients. Everyday Speech is one of the main tools currently used by counsellors in their practice. Everyday Speech provides social-emotional learning materials for speech therapists, special educators, and diverse learning professionals. However, in products like Everyday Speech and Centervention, the counsellors indicated that their clients responded less positively towards avatars that were not realistic. The counsellors also used remodelling exercises by showing many video examples of appropriate and inappropriate conversations to encourage their clients to understand the subject matter better. The clients were also lauded for following those examples in real-life scenarios. Overall, there was consensus around including positive reinforcement in the gameplay to allow room for a better learning experience. The co-design workshop also revealed Galvanic Skin Response (GSR) as a better biofeedback tool to measure clients' stress levels since clients with post-traumatic stress could have lower heart rate variability (HRV) (Dennis et al., 2014). This co-design workshop inspired the game strategy for ‘conversation Fuel’, a VR exposure therapy game for clients with high functioning autism to improve their conversation skills. Figure 8: The selected stage of conversation to inform the gameplay. Research approach / Design outcome / 19 The co-design workshop results were put through another rigorous design sprint with a few interaction design students. This design sprint focused on understanding the gameplay's interaction mechanics and other elements that supported the content of the VR experience. The designers were asked to use a Morphological Matrix for rapid idea generation. Morphological analysis is used within the conceptual design as the basis for exploration of the design space, where the functions represented in the functional decomposition are listed against the means to achieve each of those functions in a two-dimensional matrix (George, 2012). This allowed us to generate different combinations of VR game ideas using additional support technology and self-regulatory exercises identified by the counsellors (See Figure 13). Scenario Focus Improve Conversation Technology Focus HRV Technology Increase awareness of different triggers and how to communicate them based on HRV feedback. 3 options on ‘how to react’ they can choose from.Show them each options’s consequences. Allow to player to practice talking and mirror tonality of voice. AI Audio Eye Tracking The avatar’s expression can be modulated based on the player’s tone of voice. AI character give vision points to guide the player on where they can focus during the conversation. Recognize Emotions Mystery game like Mystery rooms using HRV to navigate them through different rooms of emotions and using HRV to unlock the rooms. Conversation scenarios, where you recognize the avatar’s emotions and work towards changing them using tone of voice. Recognize who is approachable to have a conversation based on their emotions by gazing around a crowded room and see who makes eye-contact. ‘Read the room’ Figure 9: A tabulated representation of the morphological matrix. Further, four-game concepts were explored to fit these individuals' counselling needs based on the design sprint and co-design workshop with the counsellors. These game concepts focused on ‘Small Talk’ are the first in the VR exposure therapy game. Once the client successfully progressed in the gameplay, they would play Active listening level 2, Conversation starters as level 3 and Romantic conversations as level 4. Research approach / Design outcome / 20 Figure 10: Brainstorming concept flow for Conversation Fuel Finally, one of the four concepts was selected based on its gameplay effectiveness. This was judged based on the counsellor's level of involvement in the process, learning objectives of the gameplay, and self-regulatory strategies incorporated in the game. This concept is further explained in section 3.3.1.2 Game strategy. Figure 11: An image from the final concept user flow brainstorming After this exercise, personas were developed to inform the design strategy better. These personas were crafted based on all the insights gathered so far through discussions with the counsellors and analyzing personal blogs of individuals on the spectrum. These personas (See Figure 14) helped to encapsulate the desired functions in the final design. They act as a canvas visualizing the intent, people (users), and emotions—all of which deliver a more informed, empathetic, and exciting design. For the design strategy, the personas were synthesized for the primary stakeholders, i.e. the individuals on the spectrum. Research approach / Design outcome / 21 Cheerful Artistic Hardworking “I have spent my life trying to empathize with the neurotypical population of the world” Bio: Wayne is a web developer, and enjoys meeting new people. He is part of various board game groups and writing groups. He often finds himself trying to read everyone in the room. He has difficulties in understanding body language and the subtleties of social interaction. Age: 32 Work: Web developer Family: Single Location: Vancouver, British Columbia Character: Gamers Motivations Guilt Meeting deadlines Social Goals Frustrations - Wants to engage with others - Travel the world - Improve online persona - Attention to details - Sensitivity to visuals - Narrow range of interests Preferred communication channels Blogs Podcasts Youtube Twitter Reddit Figure 12 Two personas created to visualize the users of ‘Conversation Fuel’. Research approach / Design outcome / 22 3.3.2 Game strategy System map A system map was created, representing all the key stakeholders involved in the VR exposure therapy tool. This map was critical in understanding how different service components were connected and took a closer look at the user flow. The mapping process was broken down into 3 stages - onboarding to Conversation Fuel (VR exposure therapy tool), facilitation of the counselling service and the in-headset experience of the therapy by the client. Service flow Pre-assessment form filled by the client and reviewed by the counsellor Information flow Information from pre-assessment use as game inputs by the counsellor VR settings adjusted to suit the client Back end Checklist for GSR wearable and VR controls that support the gameplay Take notes during the session Pause the experience Counsellor controls Restart the experience from a bookmarked point Counsellors enters the game inputs Record short videos Set progress bar UI in the experience Learns about Conversation Fuel from their counsellor Decides to become a member Fills in application form for rentals and therapy Reads the terms and conditions and agrees Receives the headsets and GSR band in the mail Client books an appointment with the counsellor Counsellors customizes the avatars Client is placed in virtual waiting room Client enters VR environment for practise Client receives a demo/instructions on using the system Practice sessions to acclimatize with the tool Counsellors choose the environment Counsellors can input their deep breathing strategy Scenario 1 Client is not stressed Uninterrupted gameplay session Completes VR session Detected by GSR Read notes made by the counsellor Scenario 2 Client is moderately stressed Watch playback videos of the session Game is paused Stress level back to normal Completes VR session Complete the session and set goals based on progress Practice sessions without the counsellor Practise deep breathing strategies Detected by GSR GSR activated pause to activate deep breathing exercises Counsellor controls pause for review and deep breathing strategy Progress bar is automated by the gameplay but can be edited by the counsellor based on their review Review the experience so far Scenario 3 Client is highly stressed Game is paused Detected by GSR Stress level back to normal Completes VR session Stress level still fairly high Review the experience so far GSR activated pause to activate deep breathing exercises Counsellor controls pause for review and deep breathing strategy Complete the session and set goals based on progress Progress bar is automated by the gameplay but can be edited by the counsellor based on their review Figure 13: System map depicting the service components of ‘Conversation Fuel’. Research approach / Design outcome / 23 Conversation Fuel - Design description Conversation Fuel is designed to encourage the development of conversational skills for adults with high functioning autism in a safe and immersive environment. These individuals may have greater difficulty developing social skills during this phase because they commonly experience social anxiety. While real-life situations can be unforgiving, VR offers a safe space for them to practice and develop their conversation skills. When combined with biofeedback signals in therapy, VR can help improve self-regulation and develop skills that can be generalized to real-life scenarios. In this VR exposure therapy, the adults on the autism spectrum can practice challenging conversation with an AI avatar. These practices will be undertaken under the guidance of their counsellors, who will have autonomy over various features that enhance the VR experience. These features include pausing the experience, providing prompts, choosing conversations between the avatar and the player, and providing personalized breathing strategies as self-regulatory exercises. Throughout the experience, the counsellors can monitor their client's stress level with the help of the GSR sensor worn by the client. The clients can benefit from the 360, interactive environment that uses a role-playing technique to support conversational skill development. Conversational Fuel is meant to be an inclusive, accessible counselling tool so that individuals on the spectrum can develop social skills and build meaningful relationships. Choices regarding the gameplay, aesthetic, and interactions were made regarding creating an experience that appeals to individuals with high functioning autism. In this experience, users learn these complex topics while practising deep breathing exercises that can improve self-regulation in real life. Essence statement The user interacts with an AI avatar in the VR environment influenced by Galvanic skin response (GSR). The user can select different stages of a conversation they want to dive deeper into and practise by talking to the AI avatar. The experience will be gamified to include positive reinforcements and self-regulatory strategies they can generalize into real-life situations. This will be facilitated by the use of GSR sensors and deep breathing exercises in the gameplay. Key product details VR Headset GSR sensors Target audience: Individuals with high functioning autism and individuals with conversation challenges. Visualisation: 3d art (stylised) (TAFI avatars) Platform: Web based Customisable features for the counsellors Research approach / Design outcome / 24 How is it played? Once in the environment, the counsellors have the ability to select a level of the conversation they want to focus on. Level 1: Small talk, Level 2: Active listening, Level 3: Conversation starters and Level 4: Romantic conversations. Once they pick a stage, they can customize the environment, avatar and topics of conversation. Each level will have different key learnings. The counsellors are also provided with the option to display the progress bar graphics during the game or after the session. Once the player begins the experience, they can start talking to the AI avatar on the topics pre-selected by their counsellors. The progress bar is designed to resemble a real-life relation trajectory where you meet someone as an acquaintance and develop a friendship over time. The players progress from acquaintance to friends based on the length and reciprocity in their conversation with the AI avatar. Some avatars will not reciprocate conversations; these avatars need to be skipped by the player by studying their body language. The counsellors can pause and point out such instances during the game. If the player cannot have a conversation longer than a minute and a high-stress level is detected by the GSR sensors, the game automatically pauses to allow the player to practice deep breathing strategies to reduce their stress level. The counsellors will also be able to pause the game based on their judgement. Once they can regulate their stress levels, they can proceed with the experience or end their session depending on the counsellor’s decision. Feature details Core loop Self-regulation exercise videos Enter customized virtual environment Topic selection by the counsellor Practice conversation skills with AI avatar and GSR sensors Assessment goals/stats Achieve conversation goals Bookmark practice scenes Build in deep breathing strategies Research approach / Design outcome / 25 Style of project Conversation Fuel will be designed, taking into consideration the sensory sensitivity of the individuals on the spectrum. The design will positively contribute to the intuitiveness of interactions in the experience. The virtual world, including all objects inside, will be built in a realistic art style closely simulating real- life. Before entering the virtual world, a realism audio style will increase the contrasts between the real and virtual world. The sound effects and atmosphere built into the environment will gradually be introduced to the user to orient the user to the virtual environment. There will be no fierce or loud sound effects. During the pause moments where the user practices a deep breathing strategy, relaxing and gentle music will be played. Art guide/moodboard These images inspire the art style of the VR environment. The realistic style model will allow the player to connect with the experience on a deeper level and be more appealing to the age group. An emphasis will be laid on using colours that adhere to the colour sensitivities of individuals on the autism spectrum. The background will be minimal and not have too many textures that will distract the player during the gameplay. The UI design for the experience will also follow a minimalistic style to provide a better screen real estate, and the interactive UI will be actionable only when focussed upon. Figure 14: Moodboard curating the art style for the environment in Conversation Fuel Research approach / Design outcome / 26 Affinity diagramming Figure 15: Affinity diagramming of the game mechanics for Conversation Fuel Research approach / Design outcome / 27 Storyboard Figure 16: Storyboarding for Conversation Fuel An initial storyboard was created narrating the experience from both the counsellor’s and client’s perspective. This storyboarding exercise was conducted to understand the elements in the frame. In the future, the storyboard has to evolve to offer both an interplay between God’s Eye View and the First Person POVs for a complete picture of the entire experience. This would include looking at the important content zone and curiosity zone in a VR experience. Research approach / Design outcome / 28 VR Presence map Move through the VR timeline portal Supporting actions Small Talk (Meeting a colleague at a bar) Audio The music slowly fades in. Any sudden sound will be avoided. Ambient sounds of a bar. Distant chattering sounds of other people present at the bar. Light music playing in the background. Story Objectives A screen indicating the takeaways from the experience. To get to know the colleague Have a conversation regarding various topics Understand body language Practice deep breathing to manage stress Scene How to have better conversations with your colleagues and get to know an acquaintance better? Emotional Objectives Primary players The counsellor can customize the AI avatar to resemble their acquaintance. The client on the spectrum and the AI avatar Experience time frame A translucent screen with deep breathing exercise to practice till the player is ready to jump back into the experience. If the individual’s stress level are high as indicated by the GSR signals, the experience will be paused automatically. The counsellors will also be able to pause it at any time. The progress bar will be non intruisive and interactions can be recorded by the counsellors Talk to AI avatars and be able to talk on all the topics selected by the counsellors. Players can skip non-responding avatars based on their body language. Longer the conversation and more topics discussed can increase the game progression and move them from acquaintance to friend. Actions Figure 17: Presence map outlining the in game experience for Conversation Fuel Research approach / Design outcome / 29 3.3.3 Wireframing Screens for onboarding and client profile Screens for customization (Counsellor’s view) Research approach / Design outcome / 30 Waiting room for the client Research approach / Design outcome / 31 VR interaction (client view) Research approach / Design outcome / 32 VR interaction (counsellor view) 8 7.5 7 Sn 6.5 6 0 20 40 60 80 Research approach / Design outcome / 33 After therapy session screens (counsellor’s view) 8 7.5 7 Sn 6.5 6 0 20 40 60 80 Research approach / Design outcome / 34 3.3.4 Preliminary User Tests and feedback The user test for Conversation fuel focussed on testing the user flow based on the initial wireframes. Peers from the MDes cohort were recruited to participate in a usability test for the Conversation fuel prototype. Since the users recruited didn’t identify as being on the autism spectrum, the test focussed on technical and UX aspects of the experience. The users had some experience playing VR games and were fairly comfortable with the VR headsets. With this exercise, we wanted to test the hypotheses: (i) The onboarding process will be easy and simple. (ii) The experience's UI functions are satisfactory. (iii) Users would anticipate some interactions in the experience (Prompts from the counsellor and pause functions). The test results revealed some challenges the users faced during the onboarding process. Some of the coherent themes among all the users were noted and analyzed using affinity diagramming. Task details Figure 18: A user testing the wireframe prototype using web VR The users were briefed about the application before wearing the headset. The initial wireframes were prototyped using the Adobe XD software along with the Draft XR plugin. This allowed the users to experience the wireframe in the 3D environment using web VR. From this point, they were asked to complete certain tasks. The users were also encouraged to think out loud during the course of the test. Each user completed the test by doing the tasks from the standpoint of a user and a counsellor. The test administrator observed their intuitive behaviours during the experience. All the devices were sanitized before and after each test. Research approach / Design outcome / 35 Results The user interviews revealed the need for an ‘easy to use application with a seamless user flow. A table with the list of tasks as rows and users’ observations as columns was created to document observations. Tasks Onboarding process Users’ Observations Would like to see the new client's profile once it has been added. It will be helpful to have a tutorial that is easily accessible for both the counsellors and clients. Would like to know what the user sees while the counsellor is taking care of the customizations. What happens in the 'waiting room' experience? UI design Add new client option to be above the next button. Having it djacent to each other is confusing. Would like to toggle or adjust the progress bar for the clients. The topic prompts looks like an UI call to action button. It would be nice to have the GSR signals and other features at the same time (live). An option to hide it when they don't want it. Interaction expectations Would like to see preview for the customization sections. No. of maximum prompts that can be uploaded. Option to hace the therapy without prompt topics. Would like more customization options. Can the client ask for a prompt from the counsellors? Would like a waiting room experience before the deeo breathing exercise to make the transitions smooth. Would like to see how the clients can view the counsellor's notes. Research approach / Design outcome / 36 After completing this test, it was evident that we need to conduct further tests with more users to detect more major/minor problems, ideas, general observations and positive findings that could improve the experience further. These tests could also include remote user interviews, usability (including accessibility) and UX tests. The following are the questions or systems to be checked in the future tests for the experience by segmenting the project into parts and tasks and test them one by one. (i) Test transition systems (ii) Test loading systems (iii) Test navigation systems (Especially since this experience is going not to involve physical mobility) (iv)Test dialogue systems (when and how you hear the dialogues during the experience as well as audio controls) (v) Test or ideate on further interactions to be incorporated into the experience Research approach / Design outcome / 37 4. Reflections 4.1 Future design directions While this thesis project contributes to understanding the tech-supported tool needs of counsellors working with clients on the autism spectrum, further research needs to be done to refine the scenario and user experience for VR exposure therapy. The design outcome for this thesis project was based on the knowledge gained by talking to seven counsellors. Further development of the project would include an interdisciplinary team of designers and counsellors to build a VR environment that can support all individuals with high functioning autism. This thesis project leaves a framework for using generative research methods to co-design the VR exposure therapy with counsellors. However, more user tests need to be carried out with individuals on the spectrum to understand how they perceive this tool. How do we create a VR exposure therapy that satisfies the needs of both the counsellors and their clients on the autism spectrum? It would also be beneficial to have individuals on the autism spectrum contributing to the knowledge of challenging scenarios for conversation in their daily lives. In acknowledging that every client on the spectrum would have different conversation challenges, there need to be further explorations on how these scenarios can be customized by the counsellors in real-time, making the user experience seamless. In particular, there needs to be focussed testing conducted with individuals from all genders to understand how sex differences in autism spectrum disorders can affect Conversation Fuel (Sparkes, 2021). Unprecedented and unpredictable, the COVID-19 crisis has had devastating effects on counselling therapy facilities for individuals on the autism spectrum. The impact of this crisis has catalysed healthcare innovations to adapt to the new norms and provide counselling support remotely. However, this fast-tracking cannot be supported by the technological progress in AI at present due to its unpredictability. Lack of proper turing tests might have dire consequences like Microsoft’s conversational AI, Tay, that tweeted racist comments (Perez, 2016). Overcoming these barriers in the technology will require more extensive research. In the meantime, Conversation Fuel can develop an interim framework that focuses on counsellors taking the avatar's role in the virtual environment. This can negate any AI-generated decision-making process in VR exposure therapy and gives complete control to handle complex clinical scenarios during remote role-playing to the counsellor. The counsellors can also then access the recording of the session to review and tailor the therapy. Finally, Conversation Fuel’s customisable features can be developed further to be adapted to counselling needs of a wide range of neurodivergent individuals. Reflections / Future design directions / 38 4.2 Design tensions and limitations Due to time limitations, this thesis project's scope focused on conducting a co-design workshop with only four counsellors. More counsellors could be involved in the study to gain a deeper insight into the subject matter. A more in-depth examination of individuals' experiences on the spectrum also needs to be considered to improve the tool's functionality. There were additional challenges in researching during the COVID-19 pandemic. Many initial research methods had to be altered to minimal viable generative exercises to comply with social distancing measures. While most counsellors were comfortable using online collaborative platforms to participate in the co-design workshop, adequate preparation was needed to include experts who weren’t well-versed in the tool (Miro) and design process. They had to be provided ample time to familiarize themselves with the platform to contribute effectively. The participants in this research were also going through a phase of transitioning their practice to a virtual space during this thesis project and working with clients on the spectrum who were experiencing aggravated anxiety during the lockdown. While these experiences were pivotal to the research, this project also acknowledges the participants' emotional challenges contributing to this thesis project. There has been a phenomenal shift to remote practices and technology to support remote working in the past year. This rapid and dramatic shift to a digital space has certain implications. For one thing, the number of technology adopters has increased. Among the expanded audience: children, the elderly, individuals with physical disabilities, and those sceptical about replacing in-person meetings with digital interactions. However, the technology used in clinical counselling remains expensive. Over the years, there have been more innovations in VR, neurofeedback and biofeedback to make it more accessible and less expensive. These additional support tools currently increase the counselling session rates with additional training required for adopting these new technologies. Furthermore, VR tools require fibre optics for high-quality videos to avoid motion sickness for users wearing the headset. The field of VR provides many promising opportunities (Riva, 2009) for clinical counselling. There are already location-based experiences and exposure therapies that are being developed to be used for therapy. But this thesis project wishes to explore ways in which this VR exposure therapy can function as effectively remotely without the client having to visit the counsellors physically. This tech-supported tool also hopes to provide a seamless user journey in any scenario, including disruptions caused due to a pandemic. Reflections / Design tensions and limitations / 39 5. Appendix 5.1 Work Cited Boeldt, D. et al. (2019). Using Virtual Reality Exposure Therapy to Enhance Treatment of Anxiety Disorders: Identifying Areas of Clinical Adoption and Potential Obstacles. 773. doi:10.3389/fpsyt.2019.00773 Coben, R. Linden, M & Myers, T. (2009). Neurofeedback for Autistic Spectrum Disorder: A Review of the Literature. Applied psychophysiology and biofeedback. 35. 83-105. 10.1007/s10484-009-9117-y. Corsini, Raymond J. (1966). Roleplaying in psychotherapy. Chicago: Aldine, in series: Modern applications of psychology.Transaction publishers. Dennis, P. A. et al. (2014). Posttraumatic stress, heart rate variability, and the mediating role of behavioral health risks. Psychosomatic medicine, 76(8), 629–637. https://doi.org/10.1097/PSY.0000000000000110 Ekman, Paul, and Gavin Yamey. "Emotions revealed: recognising facial expressions: in the first of two articles on how recognising faces and feelings can help you communicate, Paul Ekman discusses how recognising emotions can benefit you in your professional life." Student BMJ, vol. 12, 2004, p. 140+. Gale Academic OneFile, . Accessed 6 Jan. 2021. Faridi F., Khosrowabadi R. Behavioral, cognitive and neural markers of Asperger syndrome. Basic Clin Neurosci. 2017;8(5):349–359. doi: 10.18869/nirp.bcn.8.5.349. George D. J., (2012). Concept generation using morphological and options matrices. All Theses. 1558. https://tigerprints.clemson.edu/all_theses/1558 Glanz, K., Rizzo, A. (S.), & Graap, K. (2003). Virtual reality for psychotherapy: Current reality and future possibilities. Psychotherapy: Theory, Research, Practice, Training, 40(1-2), 55–67. https://doi.org/10.1037/0033-3204.40.1-2.55 Hull, D. (2007). Blogging Between the Lines. American Journalism Review. Dec 06/Jan 07: 62-67. Kandalaft, M. R., Didehbani, N., Krawczyk, D. C., Allen, T. T., & Chapman, S. B. (2012). Virtual Reality Social Cognition Training for Young Adults with High-Functioning Autism. Journal of Autism and Developmental Disorders, 43(1), 34-44. doi:10.1007/s10803-012-1544-6 Appendix / Work cited / 40 Maskey, M., Lowry, J., Rodgers, J., Mcconachie, H., & Parr, J. R. (2014). Reducing Specific Phobia/Fear in Young People with Autism Spectrum Disorders (ASDs) through a Virtual Reality Environment Intervention. PLoS ONE, 9(7). doi:10.1371/journal.pone.0100374 Parsons, S., & Mitchell, P. (2002). The potential of virtual reality in social skills training for people with autistic spectrum disorders. Journal of Intellectual Disability Research, 46(5), 430-443. doi:10.1046/j.1365-2788.2002.00425.x Perez, S. (2016). Microsoft Silences its new AI bot after twitter users teach it racism.https://techcrunch.com/2016/03/24/microsoft-silences-its-new-a-i-bot-tay-after-twitter-user s-teach-it-racism/ Powers, M., Emmelkamp, P. (2008). Virtual reality exposure therapy for anxiety disorders: A meta-analysis, Journal of Anxiety Disorders, 22(3), 561-569. https://doi.org/10.1016/j.janxdis.2007.04.006. Riva, G. (2009). Virtual reality: an experiential tool for clinical psychology, British Journal of Guidance & Counselling, 37:3,337-345, DOI: 10.1080/03069880902957056 Sparkes, M., (2021). Men who are bad at public speaking can get help from a virtual clone. https://www.newscientist.com/article/2267298-men-who-are-bad-at-public-speaking-can-get-help-f rom-a-virtual-clone/ Strickland, E., (2021). OpenAI's GPT-3 Speaks! (Kindly Disregard Toxic Language). https://spectrum.ieee.org/tech-talk/artificial-intelligence/machine-learning/open-ais-powerful-text-ge nerating-tool-is-ready-for-business Wainer, A. L., & Ingersoll, B. R. (2011). The use of innovative computer technology for teaching social communication to individuals with autism spectrum disorders. Research in Autism Spectrum Disorders, 5(1), 96-107. doi:10.1016/j.rasd.2010.08.002 Wald, J., Taylor, S. (2000). Efficacy of virtual reality exposure therapy to treat driving phobia: a case report, Journal of Behavior Therapy and Experimental Psychiatry, 31, 3–4. https://doi.org/10.1016/S0005-7916(01)00009-X. Watler, M. (2020, July 8). Personal interview [Personal interview]. Appendix / Work cited / 41 5.2 List of Figures Figure 1: Blog analysis table synthesized based on data from various blogs run by people on the spectrum Figure 2: Journey mapping of a teenager living with ASD. Figure 3: Journey mapping of a parent of a teenager living with ASD. Figure 4: Affinity map analyzing the results of the interview session with the counsellors. Figure 5: The Miro board for the co-design activity with the counsellors. Figure 6: A closer look at the instructions for the activity to build the game strategy for the VR exposure therapy. Figure 7: Analyzing the results from the co-design workshop and secondary research Figure 8: The selected stage of conversation to inform the gameplay. Figure 9: A tabulated representation of the morphological matrix Figure 10: Brainstorming concept flow for Conversation Fuel Figure 11: An image from the final concept user flow brainstorming Figure 12: Two personas created to visualize the users of ‘Conversation Fuel’ Figure 13: System map depicting the service components of ‘Conversation Fuel’ Figure 14: Moodboard curating the art style for the environment in Conversation Fuel Figure 15: Affinity diagramming of the game mechanics for Conversation Fuel Figure 16: Storyboarding for Conversation Fuel Figure 17: Presence map outlining the in game experience for Conversation Fuel Figure 18: A user testing the wireframe prototype using web VR . Appendix / List of figures / 42 5.3 Primary research materials Research ethics course PANEL ON RESEARCH ETHICS TCPS 2: CORE Navigating the ethics of human research Certificate of Completion This document certifies that Binoodha Sasidharan Kunnath has completed the Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans Course on Research Ethics (TCPS 2: CORE) Date of Issue: 29 January, 2020 Research ethics certificate Emily Carr University Research Ethics Board (ECU-REB) Research + Industry Office 520 East 1st Avenue Vancouver, BC V5T0H2 +1 604 844 3800 ext 2848 ethics@ecuad.ca CERTIFICATE OF RESEARCH ETHICS APPROVAL The Emily Carr University Research Ethics Board approves the following project: File # 100380 Title Developing support tools for counsellors who are working with high functioning ASD clients to improve the clients' social relationships. Principal Investigator: Jonathan Aitken Other Investigators Binoodah Kunnath The current approval dates are: Approval Date April 29, 2020 Expiration Date April 29, 2021 The nature of the approval is as follows: Type of Event New Approval Process Type of Review Delegated Review Approved Documents Survey questionnaire/Interview consent and form/Media Release/ It is the researchers’ responsibility to meet all research ethics requirements in the jurisdictions in which the research takes place. The procedures and protocols described in this certification must be followed closely. Note the following conditions associated with this approval: Delegated review completed and the committee has given approval once conditions regarding Covid-19 issues have been resolved and we are able to resume research activities as normal. For multi-site or partnered research, researchers are required to comply with all research ethics requirements that apply. Researchers are expected to share notice of this approval with partners, sites of research, or other research ethics review boards, as applicable. If changes to the approved application and documents are required by new partners, sites of research or other research ethics boards, researchers are required to inform the ECU-REB of these changes. Researchers are required to report anticipated changes, adverse incidents, and project completion for further research ethics review. All reporting is managed through the research portal on the Research Management System Process Pathways Romeo https://ecuad.researchservicesoffice.com/. Login and complete “event” reports for changes, adverse conditions, renewals, and the completion of this research ethics file. This research ethics approval is in compliance with Tri-Council guidelines (TCSP2 2018) and Emily Carr University policies and procedures. Dr. Cameron Cartiere Chair, Emily Carr University Research Ethics Board Emily Carr University of Art + Design Cc. Research Finance Office, Emily Carr University Appendix / Primary research materials / 43 Thesis Participant invite Appendix / Primary research materials / 44 Thesis Website Appendix / Primary research materials / 45 Literature review Everyday life Inspiration Empathic design approaches can initiate Make sense of people e thinking processes within individuals in which r sh r ve Fr A new approach to design that focussed on experinces, meaningful everyday practices,emotions and to connect them to innovative solutions. O bse they try to relate their own experiences in order to understand other people. Skills required (Koskinen et al. 2003) Collaborative skills Open mindedness Observational skills Curiosity Design specific methods Storytelling Games Experience Posters Role-play Visual collages Personas Video 3D models Annotation Making Reflecting Be mindful about Process phases Using jargons/language everyone in the team can understand. Communicating preliminary results and end results. Field data into design materials. Using triggers in the process. Motivation to explore Immerse without judging Connections through reflections Detach from personal experiences Observing Empathy trap Retelling exhibitions If designers are not vigilant, the attempt to be empathic might articulate popular reflections instead of innovative, more radical futures. Sensitivity in 4 layers Sensitivity towards humans Sensitivity towards designs Sensitivity towards techniques Sensitivity towards collaboration Reality twists Is this speculative design? Imaginative research where you ask important questions to spark imagination of your participants. Designers as facilitators Helping people with no design training must contribute to design activities, be users or stakeholder. Discover ! What is a blog? Look at broader contexts, understand other aspects of the user’s life that may or may not be alligned with your research question. The process can influence changes in the research questions, topics and research methods. Immerse Detach Blogging can be described as the offspring of personal Webpages and user-generated content (Haferkamp & Kramer, 2008; Turkle, 1995). Make connections Learn Participatory research methods connect researchers with relevant communities to Evolution of design’s interrogation into healthcare design Focussed on relationship between the doctor and patient. achieve shared goals. Design shifts focus to healthcare systems which are the modes through which medicine is integrated in social structure. Fe e d b ack Through the processes of feedback the group or organism remains clearly in balance with its specific environment and the dynamic demands of the moment. The need for an interdisciplinary team Some minds more than others can readily recognise particular areas of needs. Preliminary Phase Confrontation Phase of participation Blogs and micro-blogs offer low cost (or no cost) access to vast pools of data that are publically and readily available for downloading and subsequent analysis. Such data can be gathered with tools already at the researcher’s disposal: computers with Internet access. No expensive lab equipment is necessary, nor will obvious equipment such as tape- or video- recorders alter the participants’ behavior. No further costly or time-consuming documentation of the data such as transcription is necessary. Further, online access means immediate access to the words of people with whom the researcher might never speak in face-to-face mode--people who are socially, geographically, and culturally distant from the researcher. How to analyse blog data? Understand the motivatio to blog Look for keywords Observe comments+threads+themes Reference across time Quality of research method ‘Slow science’ movement (Alleva, 2006), Emphasises investment of time and resources in the thoughtful consideration and selection of ideas before data collection. D ialo gu Accelerated development process (Matchett, 1971) Arnstein’s ladder Another key feature of participatory research is inclusiveness including adapting the research environment, methodology and dissemination routes to permit the widest and most accessible engagement, or engagement from specific groups (e.g. non-speaking autistic people and people with additional intellectual disabilities. e e m ent ” Relate the needs of the patient to his environment and then interpreting these in a form which can be fully appreciated by the various professional groups who are concerned with maintaining such patients in balance with their surroundings. (Williams, 1971) ” By participatory research, we mean incorporating the views of autistic people and their allies about what research gets done, how it is done and how it is implemented. (Cornwall and Jewkes, 1995) ag Design looks at the entire ecosystem of healthcare technologists, administrators, scientist and government. Personal blogs for user study and eng Notice attitude and feelings Sample current thinking Contextualise findings within real-world settings Check for repetition in content+themes Experiential expertise Forcefulness in textual data Important considerations in ASD research Supportive environments Methodological challenges Balance individual and collective opinion Case studies Observe video tags Analyze as matrices, flow charts, network diagram and contingency display. Re-orientation Phase Creative Phase Work cited in the literature review Matchett, E. and Williams, K.G. “Strategic design for International Healthcare.” Proceedings of the Design Research Society's Conference Manchester, edited by Nigel Cross, 1971, p.99. Mattelmaki, T. et al. “What Happened to Empathic Design?” Article in Design Issues, 2014, Volume 30, Number 1. DOI: 10.1162/DESI_a_00249 Mattelmaki, T. et al. “On designing open-ended interpretations for collaborative design exploration.” Article in CoDesign, 2011, pp. 79-93. DOI:http://dx.doi.org/10.1080/15710882.2011.609891 Watson, S.F. et al. “Making the future together: Shaping autism research through meaningful participation.” Autism, Sage Publications, 2019, Volume 23, pp. 943-953. DOI: 10.1177/1362361318786721 Webb, L. and Wang, Y. “Techniques for analyzing blogs and micro-blogs.” Advancing Research Methods with New Technologies, 2013, p. 206. DOI: 10.4018/978-1-4666-3918-8.ch012 Appendix / Primary research materials / 46 Emily Carr University of Arts + Design, 2021 © Binoodha Sasidharan Kunnath