The Projection of Mango Mawenzhu Shi BFA, University of British Columbia, 2020 A THESIS SUPPORT PAPER SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF FINE ARTS EMILY CARR UNIVERSITY OF ART + DESIGN 2024 ©Mawenzhu Shi, 2024 Shi 1 Abstract This project and research explore how perceptions of self are produced, deconstructed and reshaped through alternative narratives and interactive installation. It urges people to reconsider how individuality emerges through engagement with new media technologies. Combining digital video and interactive technologies with philosophical concepts such as I-Ching and Agential Realism, this project explores the process of becoming through a video essay that charts the lifecycle of a fictional character, named Mango. Mango is a complex entity that is both an imagined being and an abstract idea, taking on many forms throughout the work. Her changes reflect an ongoing instability and shift in her identity. Using an interactive video installation, I conflate the viewer with Mango's image and narrative, mixing subject and object identities. These experiences demonstrate how we have adapted to an environment so heavily reliant on hyper-visibility and cybernetics. Our understanding of ourselves is integrated with the screen image. The body of work – Mango – combines different media techniques including video collage, 3D modelling, facial recognition and interactive design to implicate the audience in a cycle of formation. Through these approaches, Mango offers a new perspective on how we come to look and perceive the world in the projections of others, both animate and inanimate entities. It considers technology as an extension of human desires that hold invisible biases and values. Shi 2 Table of Contents ABSTRACT .............................................................................................................................................................. 1 ACKNOWLEDGEMENT ............................................................................................................................................ 3 INTRODUCTION ..................................................................................................................................................... 4 POSITIONALITY ...................................................................................................................................................... 9 ARTISTIC AND RESEARCH METHODOLOGY ........................................................................................................... 11 DIGITAL EVER-FOLDING ORIGAMI – THINKING OF INTERCONNECTEDNESS .......................................................... 14 FROM FRUIT TO FORM: THE METAMORPHOSIS OF MANGO AND THE FLUID IDENTITY ................................................................16 REFLECTIVE ORIGAMI: AN INTERPLAY OF SELF AND OTHERS .................................................................................................19 SHAPE AND MOVEMENTS: THE DYNAMICS OF CONNECTION ................................................................................................22 THE NARRATIVE – A TEST ON THE POWER OF TECHNOLOGY ................................................................................ 25 THE INTERACTIVE SYSTEM – INTO A HYBRID MODE ............................................................................................. 28 AN ONGOING FUSION AND DISPLACEMENT................................................................................................................28 A PROCESS OF BECOMING .............................................................................................................................................31 MONSTER – BECOMING UNPROJECTABLE ............................................................................................................ 37 TRANSFORMING INTO THE STRANGE OBJECT ...............................................................................................................40 THROUGH THE DEFINITION OF MONSTER – GUAI WU .........................................................................................................45 UNFOLDING THE MECHANISM OF MANGO ..................................................................................................................47 CONCLUSION ....................................................................................................................................................... 50 LIST OF FIGURES ................................................................................................................................................... 51 WORK CITED ........................................................................................................................................................ 52 WORK CONSULTED............................................................................................................................................... 54 POST-PRESENTATION REFLECTION ........................................................................................................................ 55 Shi 3 Acknowledgement I would like to express my deepest gratitude to my supervisors for their invaluable support, expert guidance, and dedicated mentorship throughout this journey. I want to thank Peter Bussigel for generously spending your time sharing knowledge on Max and helping me with improving the details in my patches and my installation. I want to thank Gwenessa Lam for your critical insights into both the project and the paper, and for helping me to refine my language and crystallize my ideas. My appreciation extends to Matt Stephanson for guiding me on the complicated recording and sound processing techniques, Sara Osenton for thoroughly examining my ideas and asking me “why”, and Yang Hong and Kevin Romaniuk for addressing my installation queries and offering practical solutions. I am deeply grateful to all my course instructors for their engaging discussions and genuine feedback. Equally, my gratitude goes out to my peers in the 2024 cohort for being such a talented and supportive group. Sharing this journey with you is a cherished memory. Furthermore, I must extend my thanks to the member of my thesis committee for their feedback that inspires and motivates my future exploration. Special thanks to all the people doing tutorials online for sharing their expertise and aiding in my learning process. Finally, I want to thank my friends and families for every “wow” that you said when seeing my work and for all the emotional support provided. This thesis would not have been possible without the contributions of each and every one mentioned above. Shi 4 Introduction This project is a 10-minute single-channel video installation designed for a single viewer (Fig 1). This installation includes an interactive component where a camera, positioned above the screen, captures the viewer's face. These images are subsequently integrated into the video resulting in a personalized, distorted reflection of the viewer. The video develops around the life story of Mango, a complex rhizomatic being central to the narrative. As the story unfolds, the events and the choreography of her movements illustrate the concepts of I-Ching and Agential Realism that contribute to the process of her transformation. As we are living in a time when information and images are easily generated, manipulated, and distributed, access to information and images is a crucial factor that contributes to the reality that we perceive. How we interact with information affects both our self-realization and our recognition of the world. Therefore, this project Mango asks how individuals manifest through various forms of interaction with others1. It investigates how interactivity and educational media contribute to authenticating and constructing beliefs that may or may not be true. Yet the format and methods of media present them as fact, making it easy for viewers to mold their own values and identities Others being inclusive of all individuals, events, and entities not previously acknowledged as part of one's own identity. 1 Shi 5 alongside what is presented. The viewer's interaction with Mango is a way to explore how digital technologies can shape the viewer's sense of self. Figure 1. Shi, Mawenzhu. Thesis Exhibition installation. 2024 Shi 6 Projection is the term that I use to address the act of interpreting the gap between what is known and unknown. This research takes the concept of projection as a means to explore the interconnected relationships between viewers and mediated images; they function as a type of feedback loop where viewers and mediated images influence and shape one another. A person’s ability to contextualize or understand an unknown person or object is based on their past experiences as well as their subjective (mis)understandings. In this piece, the process of coming to know Mango is similar to using a projector to project onto a canvas that already has an image on it2. The viewer’s interpretation acts as this external light that casts a new image onto the existing picture. The result is a layered image that recontextualizes both the original subject and the environment through which the viewer forms their interpretation. Therefore, the projection forms a mixed image generated from this interaction between the viewer and Mango. Mango resolves in different ways depending on who is viewing her. Through my research on the formation of individuality3 and the fragmentary narrative, I apply these two concepts to my video to create a looped work that not only portrays the life cycle of Mango but also embodies the power of projection within the 2 The original image refers to the conventional understanding or portrayal of the fruit mango. I am using “individuality” within Karen Barad’s framework of Agential Realism. Within this framework, individuals are considered not as separate, pre-existing entities. Therefore, individuality is not an inherent quality that one possesses, rather, it emerges through intra-action. This will be elaborated on page 30. 3 Shi 7 story. I utilize video art and interactive design to introduce a complex folded being Mango, whose rotation and enfolding abstractly represent her capacity to connect multiple planes of existence. Depending on the perspective from which the viewer witnesses her unfolding, Mango shifts identities and performs different relationships. Due to the lack of closure with respect to Mango’s identity within the story, she becomes a vessel that accommodates different individuals. She is a shuttling ghost, who lives inside everyone and resolves within the narrative frameworks to which the viewers are most attuned. She also serves as a subject and a storytelling medium, engaging the viewers in an experience where an imagined individual is formed through projection and enfolding into seemingly disparate narratives. The lifecycle of Mango consists of different stages (Fig 2): Figure 2. Shi, Mawenzhu. The loop of Mango. 2024 Shi 8 To unpack the thought-process behind this lifecycle, the following sections of the paper offer an analysis of the creative process and narrative structure of Mango. First, I begin with a positionality statement reflecting on how personal experiences influence my project and research. Next, I discuss the artistic and research methodologies employed. Last, I trace the lineage of the chapters4 of the video essay, explaining the process for each of the chapters and discuss how individuality comes into being in the interactive part of the video. By bridging the concepts of cyborg, tourist gaze, Agential Realism, I-Ching and Buddhism, I progressively unveil technology’s capacity to blur the boundaries between subject and object positions and foreground the shift of perspective between Mango and the viewer. 4 Chapters in this paper will always refer to the chapters of the video essay, not different sections of the paper. Shi 9 Positionality As a member of the Dai minority group, raised in the urban environment of Shenzhen, China, my connection to the cultural traditions of my heritage, specifically those of Yuanjian, Yunnan province, has always been nuanced by physical and emotional distance. The use of the mango derives from my affection for this fruit through a journey of reconnecting with my cultural roots. In my undergraduate project “Harvest”, I rewrote a song from my father that is dedicated to the Golden Mango Festival. Rewriting the song involves researching the festival, which is a way of experiencing my culture through technologies such as screens and media portrayals. It adds a layer to my identity formation, shaping my connection with my roots through the representations found online. In this process, I reinterpreted my own identity and found myself projecting a foreign gaze onto my own narrative. This gaze is similar to what is described as the “tourist gaze” by Urry John – a specific kind of gaze that manipulates the object’s self-identification as the result of catering to the aesthetics of the subject, the tourist, that is looking5. In this case, I am both the observer and the observed, navigating the dual roles of the tourist exploring unfamiliar terrain and the native deeply connected to the cultural narrative under scrutiny. This duality highlights a form of selfalienation that reshapes my understanding of the self from both external and internal 5 Urry, John, and Jonas Larsen. The Tourist Gaze 3.0. 3rd ed, SAGE, 2011. Shi 10 positions. I see this as a form of projection, in which the self is constantly being reshaped through relations and interactions. It forced me to constantly question my identity as a Dai descendant living away from my cultural homeland, and consider how my personal narrative and cultural identity are continually reconstructed in response to both external perceptions and my internal monologues. Therefore, the impetus for this research lies in my contemplation of being an observer, a canvas, a bridge and an intervention in the process of knowing. Shi 11 Artistic and Research Methodology This thesis project is a combination of a video essay and interactive installation with a narrative that unfolds to reveal the different stages in the life cycle of Mango. The story is inspired by the Golden Mango Festival held every year in my hometown Yuanjiang, Yunan, China. Researching the festival is a way to learn about my own culture and form a better understanding of how human activities and mangoes interact with each other and affect the status of each other’s being. This approach includes gathering and generating images and videos that are then incorporated into the construction of the story. In creating the video, I worked with the visual elements after I finished the narrative script. This process involved collecting found images online, filming videos, building 3D models and creating animations. Using the videos and images I collected and generated, I experimented with various distortions and collages to help support the storytelling and highlight some of the technical and theoretical ideas. Simultaneously, I gathered sound samples from recordings of the physical origami object moving. I rearranged the sound samples to compose a new soundtrack that serves as the language of Mango. These approaches create space for contingency, exploring how sensorial experiences are influenced by technology. The results of these manipulations are not fully predictable. The directions of where these manipulations lead to are influenced by the capabilities Shi 12 and constraints of the software. I conceive of this process as a collaboration between machine and human, foregrounding a type of cybernetic authorship. The foundational question of my research is an inquiry into how new digital technologies such as cameras and computers contribute to reshaping the subjectivity of viewers through both passive and active interactions. This exploration involves an examination of the dynamics between the viewer and the viewed within the context of the interactive video essay. My artistic approach in this project involves experimenting with, and designing interactive technology to create a more engaging experience for the viewer. By utilizing face tracking and image distortion in Max/MSP, the interaction points to another layer of understanding the relationship between humans and machines. Along with evaluating the possibilities and limitations of the outcomes, this is a reflexive process that continues to evolve as I explore how individuality is generated within relations. Through this approach, I attempt to explore how the identity of the viewer and the viewed could potentially be changed through the interaction between the human viewer and technology. Alongside these experiments and tests, my research methodology continues to be shaped by theorists such as Karen Barad, Donna Haraway and Dorothée King, as well as works by other filmmakers and new media artists such as Apichatpong Weerasethakul and Hito Steyerl whose artistic practices have influenced my thinking and have provided Shi 13 frameworks for my analysis of media art and interactive video installation. In thinking with these theorists and artists, I construct an understanding of the process of displacement between the subject (the viewer) and the object (the viewed) while reflecting critically on the concept of projection and its relationship to systems of power. Shi 14 Digital Ever-Folding Origami – Thinking of Interconnectedness The first two chapters of the video offer a definition of Mango and explain how Mango is formed. Mango adopts an origami shape that emphasizes the connection of her surfaces and how folds give shape to her structure. She serves as a form for me to meditate on the concept that everything is interconnected. By acknowledging that Mango is not merely a tropical fruit, I explode her definition and properties beyond the conventional understanding of mango and unveil her rich connections to culture and identity formation. This exploration is marked by a deliberate playfulness. As I am projecting onto Mango, she reflects and projects herself back to the viewers, transcending the role of exotic food and transforming into an entity with consciousness that dreams of self-actualization. Shi 15 Figure 3. Shi, Mawenzhu. Digital reflective origami. 2022 Shi 16 From Fruit to Form: The Metamorphosis of Mango and the Fluid Identity At the beginning of the video essay, Mango undergoes a transition from a conventional-looking mango to an ever-folding origami with a reflective surface (Fig 3). This transition presents Mango as an entity holding multiple intersecting identities and narratives. By assigning the name Mango to this reflective object, the disembodied articulation and act of naming initiates the unfolding, breathing life and identity into an otherwise inanimate form. This transformative act is likened to the power of projection – imposing a subjective definition onto something. Naming is a gesture for me, as the artist, to impose my subject definition onto the origami. The name is an aggregation of labels. Assigning the name to origami and personalizing it are both ways of showing the power of projection. The labels that are being put on the origami contradict what people usually think of what origami is. How this origami performs within the video also differs from the conventional understanding of the fruit mango. This cacophony showcases my ability, as the artist, to change the material that I am using. Also, it is a way of thinking about how a material’s agency emerges through manipulation. One essential difference between mango and origami is their differences in affordance. Affordance, according to Kirchhoff in Material Agency, refers to “the ways in which the environment lends or offers itself for perception and action – that is, how the environment creates and shapes opportunities for action in relation to an organism” Shi 17 (Kirchhoff 209). The suggested action between a human and the fruit mango involves eating, whereas the origami involves folding and unfolding. These distinct forms of engagement highlight the affordances of these objects. My conflation of two completely different objects and their affordances creates an absurd and playful scenario that demonstrates how we, as viewers, can suspend our beliefs to accept Mango’s transformation from fruit to folding origami. This change of material and affordance shifts our behaviour and perception of Mango. As viewers, we are moulded and defined by the perceptions presented to us, and in turn, how we see ourselves influences how we perceive and relate to others. This irrational equation drawn between mango and origami denies the dichotomy between animate and inanimate objects. This approach superimposes distinctly different definitions on Mango and suggests that Mango doesn’t have a fixed identity. It brings disruption to how viewers perceive things using labels and names and sets the stage for further displacement between different identities in the following chapters. Also, the traditional concept of mango is extended into the digital world using 3D animation. This action prolongs the lifetime of Mango and expands her morphing abilities. Shi 18 Figure 4. Shi, Mawenzhu. Physical reflective origami. 2022 Shi 19 Reflective Origami: An Interplay of Self and Others The shape of Mango is derived from a physical origami that I made using reflective film (Fig 4). The reflective surface of the ever-folding origami mirrors whatever is being projected on her back to the environment. As observers interact with the origami, their images and environment are integrated into its surface, becoming a part of what is viewed. Therefore, the surface of the origami is always mirroring its environment. How it looks is influenced by its surroundings. In the manifesto Of Other Spaces: Utopias and Heterotopias, Michel Foucault claims that the mirror is an example of heterotopia – a juxtaposition of time and space within one place: “The mirror functions as a heterotopia in this respect: it makes this place that I occupy at the moment when I look at myself in the glass at once absolutely real, connected with all the space that surrounds it, and absolutely unreal, since in order to be perceived it has to pass through this virtual point which is over there” (Foucault 4). As Foucault explains, we can only perceive the self-image and assure our position in a virtual world that does not exist in the real world. Therefore, the mirror has the power of both emphasizing the presence of the body and also undermining the acknowledgement of where the body is positioned. Looking into a mirror is a way of finding the self, depending on a virtual that does not exist. This structure of locating Shi 20 implies that the self-realization is based on a fiction of the mind and an illusion formed by the light. During the interaction with reflective origami, the folds create different facets that reflect the surroundings from different angles. All the surfaces simultaneously create interconnected virtual reflections that are tangled with the real world. With the movement of folding and unfolding, different virtual worlds expand and collide onto each other. This is a system of generating and destroying at the same time. It is designed to diminish the significance of the viewer by cancelling the singularity which supports the one true subject position. It makes viewers become a part of the object that is being viewed so that viewers occupy both the subject and object position. This potential shift in position is formed via the interaction with the reflective origami and is navigated by the projection from the viewers themselves. Shi 21 Figure 5. Shi, Mawenzhu. Folding unit. 2024 Figure 6. Shi, Mawenzhu. Folding along z axes. 2024 Figure 7. Shi, Mawenzhu. Folding along y axes. 2024 Shi 22 Shape and Movements: The Dynamics of Connection The origami's folding pattern visually embodies Mango's role and identity in the story. To make the origami into a digital image, I followed the steps of how I made it physically. I first shaped out the single unit of the fold in Blender (Fig 5) and then duplicated it 7 times. Depending on which axes they are duplicated, the outcome differs (Fig 6) (Fig 7). This final shape of the origami is a result of multiplicity that is guided by the continuity of the folding pattern. In origami, it is not only the folding lines that construct the shape, it is also how the folding lines connect to the different surfaces and how they relate to one another. The resulting structure varies depending on how I fold the origami. By creating and folding this shape in the digital environment, I meditate on the possibility of Mango being a conceptual fold. In Gilles Deleuze’s framework, the fold is “an ontology of becoming, multiplicity and differentiation while maintaining a continuity” (Krissel). It is a metaphor for the constantly changing process of becoming. In this narrative, Mango goes through different states of change by folding and unfolding – different edges and planes come into contact and connect but subsequently separate. Each configuration represents a different potential state of being. Through the connections and relationships between each fold and plane, Mango develops a new identity. This constant flux destabilizes any fixed identity and enables Mango to hold several subject positions. The ever-folding origami form becomes a visual motif within Shi 23 this story – presenting Mango as a fluid identity that is constantly changing and being shaped by different connections. Mango keeps folding and unfolding to create a loop that has the past, the present and the future all juxtaposed within the trajectory. The movement of the origami is a vehicle for my contemplation of the idea that everything is interconnected. The shape of the origami is built with 8 units of folding. The way they connect to each other allows certain movements to happen. These movements include moving upwards, moving downwards, moving outwards, moving inwards, and staying still. These five movements align with the Five Phases(wuxing/五行) in Taoism which are employed to describe interactions and relationships between phenomena. The Five Phases are also used to explain the interconnectivity of all things. Paralleling these movements with the Five Phases is a way of thinking about the formation of this origami through its movements and how it is defined by its folded structure. By animating these movements in the video, I enact this philosophy of interconnectedness. I examine how Mango finds her way of becoming through the process of folding and unfolding, and demonstrate an ever-changing state guided by these movements. The concepts of the fold and Taoism, which is about how things are connected, form the basic logic of this newly constructed world which breeds the rebirth of Mango. They envelope different ways of knowing about this loop of Mango’s lifecycle with no Shi 24 beginning and ending. The interconnectedness of the form performs as an interlude between Mango’s evolving identity and the increasing potency of her transformative power developed through the video’s chapters. Naming the ever-folding origami Mango serves as a pivotal moment, paving the way for a deeper exploration of Mango’s agency and the complex transitioning dynamics that execute the displacement between the Mango and the viewer. Shi 25 The Narrative – A Test on the Power of Technology The interactive video essay follows a storyline that consists of Mango’s lifecycle – the birth, the growth, entering the market / being celebrated, being consumed, becoming a monster and rebirth. This story includes different stages of Mango’s becoming and illustrates how she morphs through various interactions. By adopting the tone of an educational documentary, the style adds an authoritative dimension to the elaborated content. Utilizing this form of narrative examines the power of media in delivering information. By educating and authenticating the unsubstantiated information through a video essay, it’s an approach to reveal video playback technology’s influences on viewers’ subjectivity, as it alters the way viewers perceive the world and the world that they are perceiving. This educational format enhances the transformative power of media and its projection onto its viewers. This approach to composing a narrative is influenced by Hito Steyerl’s video essay How Not to Be Seen: A Fucking Didactic Educational .MOV File. In this work, she structures the video in an educational form with five different lessons offering strategies on how not to be seen in this era of over-visibility. She addresses the significance of the rules that are buried within the digital exchange methods and how tightly our society and culture are bonded with these selections filtered by technology. Within this world so heavily reliant on technology, the widespread distribution and easy-to-manipulate nature of digital images make them the ultimate condensation of social forces, akin to Shi 26 the way propaganda operates by shaping perceptions and influencing public opinion on a mass scale. Whoever acquires the knowledge of and has control over the handling of digital images possesses an unseen hegemonic power. In this way, digital images not only mirror the technological dimensions of our society but also encapsulate the social and political systems in action, along with their inherent biases. This frame of understanding the digital image influences my approach to handling images within my work. By using the video essay to tackle issues within this digital era, I am also interested in employing an ironic and humorous tone to unfold these lessons. Steyerl’s video not only talks about the visual disappearance within digital images through technology but also points out the wipe-out and vanishing of certain populations under the political power of our social structures. It explores how technology exhibits bias, leading to the marginalization of certain groups in social and political realms. This insightful interpretation of technology aligns with my research on the inherent power of technology to influence our behaviour through what is made visible. Similar to Steyerl, I used a machine-generated voice to narrate the video, Specifically, I used the voice from Google Translate which is often heard when people try to translate a language into another. Translation is not just about converting words from one language to another, but also about navigating and interpreting cultural contexts and nuances. During this process, some of the original meaning may become distorted or lost, highlighting the subjective nature of translation, especially when trying to Shi 27 communicate something more complex. The use of the Google Translate AI voice serves to reflect how the programming inherent within technology can be subjective. These biases or assumptions built into the technology shape our perception and understanding of reality. Similar to Steyerl’s approach to the video structure, I am also using chapters to address the transformative power within Mango’s iterations. The utilization of an educational format enhances the text's authority. However, it also highlights the complexities and dualities inherent in conveying information. Political propaganda frequently uses this educational structure to communicate a biased agenda or interpretation. Therefore, by combining the educational structure with a fictional story, this video raises questions about the authoritative voice associated with this format. It engages the viewer in a critical dialogue about the reliability and manipulation of information. In this way, the educational format serves not only to inform but also to expose the inherent subjectivity and power structures that shape how knowledge is constructed and conveyed. Shi 28 The Interactive System – Into a Hybrid Mode The interactive aspect of the video piece further explores the human-machine relationship and how these relations bridged through technology shape the viewer’s identification with the projected image. When the viewer looks into the monitor, the camera captures and represents their image back to them; the machine now becomes a spectator and the viewer enters into the frame. “While technology never simply determines, it cannot but affect the context in which ideas are formed.” (Mulvey 9) This interactive system shifts the viewer from the subject position to the object position, creating a new understanding of the viewer’s own identity through their relationship with the machine. This shift in position alludes to the bi-directionality of humanmachine interfacing – as much as we, human beings, claim to be the creators of technology, technology is also changing us. An Ongoing Fusion and Displacement Reflecting on the impact of technology on us, I integrated an interactive component into the video, crafting an alternative approach to multimedia storytelling. By introducing a webcam that captures the face of the viewer which then merges into the video, I create a field where displacement between the viewer and Mango could happen. In order to achieve this, I wrote a program using Max/MSP that tracks faces from the webcam feed and synchronizes the movement of faces within the frame with Shi 29 Figure 8. Shi, Mawenzhu. Max Patch. 2023 Figure 9. Shi, Mawenzhu. Image generated by Max. 2023 Shi 30 the spectral content of the soundtrack (Fig 8). The sound is sampled from the crinkling of the physical reflective origami in the attempt to construct an abstract language for Mango. When the viewer moves, the sound is filtered according to the movement. This is an aural hint for the viewer to notice that the distorted image (Fig 9) that they are looking at is responding to their movement. Additionally, I incorporate another layer of interaction with the sound in the case where faces are not detected. When the viewer doesn’t offer their face to the interactive system, the aural feedback becomes a consistent crackling sound of the physical origami, forming a fierce glitch-like sound effect that responds dynamically to the viewer’s movements. In this system, the webcam reflects and operates upon the human desire to look, projecting the machine’s gaze. The dynamic aspect of the technological gaze is discussed in Interface Cultures: Artistic Aspects of Interaction. Dorothée King notes that in the exploration of new territories, humans sometimes produce and develop new technology based on the logic of human behaviour.6 In the article, she connects the concept of the “gaze”7 to digital interactive environments. The gaze is a force of control that has the power to regulate an individual’s behaviour under a potential observation8. 6 Sommerer, Christa, Laurent Mignonneau, and Dorothée King. "Interface cultures." Artistic aspects of interaction. Bielefeld: transcript (2008). 7 How Dorothée King uses “gaze” is similar to the concept of projection that I am using in this paper. Gaze not only indicates the direction of the looking but also implies the power dynamic between the subject of looking and the object being looked at. 8 In Michell Foucault’s’ work Discipline and Punish: The Birth of the Prison, he suggests that when the viewers know that they are potentially being looked at, they begin to regulate their own behaviours to Shi 31 Therefore, the one who is being gazed at is subjected to this relationship. By drawing a parallel between the power of gaze and machines that replicate human action, she argues that interactive artworks that “focus on the exploration and replication of the human body” (King 175) have a shared ability to construct a hierarchical structure in relationships. By introducing the webcam within this interactive system, I am replicating and reversing the typical human interaction within machines. The webcam is an extension of the human desire to look and thus it carries a pre-coded gaze. The technology, therefore, informs the viewer that they are also being watched within this relationship. This approach leads the viewer into the object position of looking. A Process of Becoming The pattern of distortion in the video is designed to eventually reveal the face of the viewer. The image will morph into a portrait of an overlayed viewer’s face – a result of the viewer’s own identity and the machine’s projection. This uncanny juxtaposition emulates the process that happens in daily life – an individual comes into being in every moment reconstructed by relationships and interactions. With the nuanced movement and track volume synchronized to the viewer’s face location, the viewer comes to recognize their own face in the image created by the conform to social expectations. In this context, the gaze acts as a mechanism of control, where individuals self-regulate their behaviour in response to being under observation. Shi 32 machine. The attempt to control the technology system would further merge them with the machine to a point where the machine becomes part of themselves as an unattached prosthetic that controls the sound. This fusion creates a hybrid identity – cyborg – that is defined by both the machine and viewers. According to Donna Haraway, “A cyborg is a cybernetic organism, a hybrid of machine and organism, a creature of social reality as well as a creature of fiction” (Haraway 150). She argues that the distinction between human and machine, nature and culture, mind and body are no longer valid. This blurring of boundaries leads to a penetration of agency that is distributed across these multiple elements. In this sense, agency is not exclusive to the viewer alone, it also emerges from the Mango. As a resemblance to human actions, the interactive system of Mango offers feedback that steers the behavioural and emotional reaction of the viewer. It affects the viewer’s agency and reconstructs the viewer’s identity. This cybernetic system functions as a continuing fusion between technology and humans, it forms a co-constitutive relationship between the two where they constantly shape each other9. Philomène Longpré, in her interactive robotic art installation work Formica, adopts a similar approach to discuss the relationship between machine and viewer. In Formica, Longpré developed a system that uses infrared and ultrasonic detectors to 9 Haraway, Donna. Simians, Cyborgs, and Women: The Reinvention of Nature. Routledge, 2013. Shi 33 track movements and locate viewers. The projector located at the centre of the room projects a video of a fictional figure trapped in a world of red tape in reaction to the viewer’s movements and behaviour. Simultaneously, as viewers realize the reactive nature of the figure, they instinctively adhere to the computer's logic and test the limits of its capabilities. The feedback from Formica forms a pedagogy on establishing the rules about how to communicate with the machine. Viewers will pick up the ways in which they found most effective in controlling the image. Fostered by this self-learning process, the viewer and the fictional figure construct a mutual observation – the viewer is observed and examined in the same manner in which they observe and examine the fictional figure. This dynamic prompts a perennial question: are we controlling the technology, or is technology controlling us? In this exploration, Formica is also observing the viewer. Viewers become a part of the outcome of the work, falling into the ambiguity between subject and object. Similar to Formica, my work also examines the ways in which human and machine affect each other’s existence within the relationship. The interaction between machines and viewers is an inquiry into how individual identity is formed under the intervention of others. In the theoretical framework “Agential Realism”10, Karen Barad describes intraaction as a relationship that subverts the conventional notion of causality that pre-exists 10 Kleinman, Adam. "Intra-actions." Mousse Magazine 34.13 (2012): 76-81. Shi 34 before the determination of individuality. They claim that “’individuals’ do not preexist as such but rather materialize in intra-action” (Kleinman 77). Simply put, individuals come into being through the intra-action with others. The separation and differentiation between different individuals are not a fixed inheritance; they also emerge through intra-action. The way people intra-act can create distinctions, giving them unique abilities to take actions and shape their own identities. This concept of intra-action is not merely a way of learning how everything comes into being, it is also an ongoing inquiry about how differences are made. It is a concept that eliminates the notion that separations are inherent but emerge within the relations. This agency that materialized within relations is a way of denying fixity of any kind. Guided by Barad’s approach which rejects dichotomy and foregrounds the process of becoming, I designed the interactive system as a denial of the dichotomy between the viewer and the viewed. According to Karen Barad, the separation of individuals is formed by the intra-action between different entities or discourses amongst different topics. Therefore, this identity that emerges in watching Mango is not a fixed attribute that is assigned to the viewer. The viewer is coming into being and “mattering”11 through the intra-action with the work. As the intra-action proceeds, the generated image (Fig 10) of the viewer contributes to the narrative of Mango. The Karen Barad uses the term” mattering” to describe the emerging process of meaningfulness and significance through intra-action. 11 Shi 35 viewer becomes the object that is being consumed within this relationship. According to the allegory12, Mango is consumed in the Golden Mango Festival. Therefore, this interactive system suggests that the viewer becomes Mango in the story. This shift in identity reflects the idea that individuality emerges through intra-action and is defined within relationships. It implies that Mango is a medium, a vessel, a container that allows others to read into the ever-looping story that happens here – at the moment of viewing, and there – in the narrative of the video. This also leads to a final, what is Mango becoming? 12 Refers to Mango’s life time story composed within the video essay. Shi 36 Figure 10. Shi, Mawenzhu. Distorted image of the viewer. 2023 Shi 37 Monster – Becoming Unprojectable In the 5th chapter, Mango enters the process of being redefined as a monster. The video offers an explanation of the concept of “monster” via a Chinese cultural lens (Fig 11). In the video, I specifically use the Chinese translation “Guai Wu” (怪物) instead of the “monster” to address the result of Mango’s transformation for it is a more neutral and comprehensive term. I explain the term by adopting the traditional Chinese way of reading – “Chai Wen Jie Zi” (拆文解字 / unpick words and decode characters). Even though the term Guai Wu is interchangeable with “monster”, the Chinese term is composed of two characters that, when read word by word, are literally translated as “strange object”. Guai Wu is used in situations when something unfamiliar occurs which is undefinable and uncategorizable at the moment. It acts as a temporary substitute until a more permanent decision is established for this undefinable phenomenon. At the beginning state of Guai Wu’s emergence, the notion of Guai Wu is vague in the way that it does not immediately fit in a particular context. As time passes, Guai Wu will eventually settle down to a stable definition of category where it loses its novelty and becomes an ordinary concept (Fig 12). This is a transformation from undefined to defined, transparent to opaque, being incomprehensible to being understood, being unseen to being seen. Therefore, the real Guai Wu is an ephemeral phase before the strange object is crowned by the term permanently. Shi 38 Figure 11. Shi, Mawenzhu. Explain words through a Chinese lens. 2024 Shi 39 Figure 12. Shi, Mawenzhu. Stills from the video. 2024 Shi 40 Transforming into the Strange Object As the video progresses towards the end, a sequential displacement happens to Mango from the conventional image of fruit to digital reflective origami, and then to the concept of the monster. Mango oscillates between solid states of definable and fluid states of undefinable. This makes Mango shift between projectable and unprojectable throughout this journey. The visibility and invisibility of Mango’s imagery resonates with my exploration of handling the image. The process of making the video consists of researching, gathering, collecting, fabricating, and manipulating. The first stage is to find images that are of interest. This procedure introduces a certain level of bias from me – as the creator of images, and technology – representing the algorithm and machines’ capability. This creates a narrative of truth through subjective choices made in the collaboration between human effort and technological functionality. To illustrate the “monster becoming” phase through image distortion, I incorporate different image handling techniques in the video. The collage and overlay effects invite the viewers to engage the story in a more subjective way. These two effects are often employed to amalgamate diverse elements into a single image. The choice of images and the manner of their combination can lead to various interpretations of the narrative. They introduce a non-linear narrative structure to the viewer, inviting them to bridge the narrative gaps and construct their own story by finding connections among the visual elements presented. These Shi 41 connections are made based on the viewer’s memories, history, culture, and life experiences. By actively piecing together fragments, the viewer is completing the story through their interpretation, transforming from a passive observer to an active participant. The kaleidoscopic effect and interactive system serve to embed the shape of Mango into a flat surface narrative. The kaleidoscope duplicates a section of the video and fans it out through reflections and rotations to create a repeating pattern that evokes the idea of a loop. This loop echoes the structure of the ever-folding origami. Even with completely unrelated footage, this looping pattern will remind the viewer that the story develops out of the origami-formed creature – Mango. It reinforces the notion that Mango’s growth and form are continually shaped by varying connections. The interactive system functions somewhat similarly to the kaleidoscopic effect, as it duplicates and reflects the image. However, it adds another layer of time by overlapping the past images with the present image. Therefore, the final result becomes a continuous distortion that forms a dynamic blend of the memory and the now. It is also a way to illustrate how the present is formed through projection of the past. In addition to the image handling techniques that I described above, I also add noise to the images to test the limits of recognition. I am interested in discovering how much an image can be altered until it becomes indecipherable to machines, yet remains within the grasp of human perception, or vice versa. Introducing noise serves as a direct Shi 42 message to the viewer that the image before them has been deliberately altered and transformed. It underscores the idea that what we see is not an authentic depiction but an alternate interpretation of reality. Despite these visual or audible disruptions, viewers have the capacity to to resolve and intimately connect with the abstractions and textures. The noise is not merely a disturbance but a narrative device, layering the visual experience with a subjective resonance that varies from one viewer to the next. It's an invitation to look beyond the surface, to engage with the imagery on a personal level, interpreting the unfolding events in the video through subjective understanding. The post-production for the collected footage is a process of alchemy that embraces chance and contingency (Fig 13). Images with different levels of distortion are layered together and exported and inputted over and over again to reach the effect that visualizes the Guai Wu phase before becoming projectable. The final result is a chaotic and noisy delusional re-enactment of the perceivable truth with irrational interpretation (Fig 14). Distorted images hover between the recognizable and unrecognizable, resonating with the Guai Wu phase that embodies the attempt to break free from the projections of others. Shi 43 Figure 13. Shi, Mawenzhu. Image being processed multiple times. 2024 Shi 44 Figure 14. Shi, Mawenzhu. Noisy delusional effects. 2024 Shi 45 Through the Definition of Monster – Guai Wu Asian films and philosophy are a great influence on the concept of Guai Wu as defined within my own work. As I researched the concept of Guai Wu, I also gathered information from movies that are related to the monster. In Hirokazu Koreeda’s most recent film Monster, 2023, he unveils a social phenomenon in which prejudice and oppression intertwine like an invisible hand that twists and smashes the good things into hideous tragedies. The monster that Koreeda proposed is a force that changes people and nests within the mind of every individual. Apichatpong offers another definition of the “Monster” in his movie Tropical Malady, 2004. The original title of this film is “S̄ạtw̒ prah̄lād” (สัตวป์ ระหลาด) which means monster or strange animal in Thai. He posits that love is a kind of disease. In the fervent search for loved ones, you will be devoured and swallowed by the desire. Within this game of chasing, one will eventually retrieve to the stage of a beast and become subject to the relationship. It implies a transformative power that turns humans into savage that are only loyal to their primitive lusts. These narratives on the monster are different from the Western portrayals. Akin to my approach, these two films both interpret the monster as a force of alteration that is formless and habitats within relation. In this case, there is not a physical proxy that is Shi 46 assigned with the title monster. The monster is only embodied through the interaction as the intra-action mentioned above13. The idea that the monster is formless is informed by Wu Xiang (无相), a term that is used both in Daoism and Buddhism. Wu Xiang, translated as “formlessness14” or “nonform”, is the idea that nothing is inherent with a definition from external interpretation15. It suggests an absence of fixity in all the rules and encourages a return to the intrinsic nature of phenomena by letting go of what you already know. It also suggests that rules, seemingly independent, are all interconnected to weave the truth of the world16. Therefore, the formlessness in Wu Xiang is the entry point to explore the essential rules of Guai Wu. In my video, Mango’s porousness and transformative capacities function to get rid of the original form. Every figure with form could fit in the vessel of Mango while Mango’s spirit is also shuttling between all the entities within this world. This process is guided by the idea of Wu Xiang by removing the form of Mango, and embodying the transformative power of projection, entangling with the ever-looping life cycle of Mango to narrate the metaphysics of Guai Wu. 13 Page 30 The formlessness that I am referring to is a concept in Daoism and Buddhism, not the concept of l'informe introduced by Georges Bataille. 14 15 Liu, Yu Zhen. Book Review: “Ru dao an xi yao fang bian fa men” from Dao Xin Chan Shi 道信禪師《 入道安心要 方便法門》 研究. (2019): 100 16 Ibid Shi 47 Unfolding the mechanism of Mango The whole of the fifth chapter is dedicated to explaining the Guai Wu phase. This phase, along with the interactive component employed within the chapter, is an embodiment of the mechanism propelling Mango through her transformative journey. Unlike the former chapters, this chapter functions differently. It unfolds the abstract mechanism of Mango’s becoming in the journey while the other chapters serve as the text for viewers to delve into. The difference makes the video into a two-part structure. This two-part structure is also influenced by Apichatpong Weerasethakul approach to the narrative in his film Tropical Malady. In this film, he constructs the narrative with two seemingly disconnected sections. The first part is a love story set in the Thai countryside between a soldier and a young worker. The second part takes an abrupt turn toward the surreal and abstract, commencing with a folklore about the mythological shapeshifting tiger spirit that lives in the forest, then follows up with a long chasing game between the two men who appeared in the first part. In this second part, the soldier becomes the hunter, and the young worker becomes the prey that is chased by the hunter. The identities of these two men undergo continual shifts during this chasing game. At one point, the hunter starts to hide and becomes the prey, and the young man becomes the predator that searches for the trace left by the hunter. Their encounter at the last part of this chase is accomplished by a mutual gaze into each other’s eyes. This mutual gaze mirrors the interactive installation that I built where the machine and human participants Shi 48 develop a reciprocal consumption of the opponent’s imagery through this act of looking. Similar to what happened within my interactive installation, through the encounter of the hunter and the young man, their identities fuse and become one. Tropical Malady is significant to my artistic research for several reasons. The first is its structure. The two-part narrative structure seems to depict disparate stories, ultimately converging into a singular narrative. Much like browsing a website and inspecting the source code to understand its functionality, the first part of the movie serves as textual information of the narrative for the viewer to dive into; the second part embodies the momentum that drives the two men with desire and attraction, unfolding the mechanism hidden within the relationship. Another distinctive characteristic of Apichatpong’s work is his approach to narration. He claims that he wants to reproduce the unlimited desire of fantasy of dreams in his film. Through long-take single shots and static framing, his film is a feast of installations within a certain duration of time. It is a compilation of cinematic pauses that only serve to construct the atmospheric pastiche. Any attempt to articulate the plot of his film inevitably descends into somniloquence (sleep talking) or nonsensical rambling. What this film leaves you with is just a sense of feeling that is translated by the sensorial memory. My approach in Mango also adopts a non-linear way of narrative. The story is weaved by multiple processes of giving definitions. It only serves as the Shi 49 descriptive function in constricting an atmospheric framework. What truly happens is only determined by the experience of the viewers when engaging with the work. The fifth chapter serves as a pivotal moment, summarizing the internal changes Mango undergoes through various stages of her existence. By unpacking the term “Guai Wu” through traditional Chinese linguistic analysis, this chapter functions as both an extension of Mango’s transition and an explanation of the mechanisms behind her process of becoming. This narrative strategy adds a way of understanding Mango’s fluid identity and its formation in relation to technology and viewer interaction, emphasizing the reciprocal influence between humans and machines. Shi 50 Conclusion In conclusion, this thesis embarks on a multifaceted exploration, utilizing Mango as a shuttling ghost to delve into the intricate dynamics of identity, agency, and the transformative power inherent in the act of projecting. By intertwining disconnected concepts, the work disturbs the conventional dichotomy between object and subject. With a journey that unfolds through chapters, Mango goes through different stages of metamorphosis. She oscillates between being seen and unseen, adapting to an identity that is at times clearly defined and at other times left undefined. By utilizing interactive installations and video essays, the work amplifies a discourse on the act of looking, and the intricate relationship between humans, technology, and identity. By bridging different philosophical ideas and theories, Mango evolves and loops back to her origin. In this iteration, the narrative forms an allegorical structure that embodies how individuality is constructed, manipulated, and interpreted. This full cycle, encapsulated within one of Mango's millions of iterations, serves as a poignant reflection on the perpetual nature of this exploration. As the loop continues, it becomes evident that Mango's journey is but one iteration among many, hinting at the infinite possibilities and continued evolution yet to unfold. Shi 51 List of Figures Figure 1. Shi, Mawenzhu. Thesis Exhibition installation. 2024 ....................................................... 5 Figure 2. Shi, Mawenzhu. The loop of Mango. 2024 ...................................................................... 7 Figure 3. Shi, Mawenzhu. Digital reflective origami. 2022 ........................................................... 15 Figure 4. Shi, Mawenzhu. Physical reflective origami. 2022 ......................................................... 18 Figure 5. Shi, Mawenzhu. Folding unit. 2024 ................................................................................ 21 Figure 6. Shi, Mawenzhu. Folding along z axes. 2024 ................................................................... 21 Figure 7. Shi, Mawenzhu. Folding along y axes. 2024 ................................................................... 21 Figure 8. Shi, Mawenzhu. Max Patch. 2023 .................................................................................. 29 Figure 9. Shi, Mawenzhu. Image generated by Max. 2023 ........................................................... 29 Figure 10. Shi, Mawenzhu. Distorted image of the viewer. 2023 ................................................. 36 Figure 11. Shi, Mawenzhu. Explain words through a Chinese lens. 2024..................................... 38 Figure 12. Shi, Mawenzhu. Stills from the video. 2024 ................................................................ 39 Figure 13. Shi, Mawenzhu. Image being processed multiple times. 2024 ................................... 43 Figure 14. Shi, Mawenzhu. Noisy delusional effects. 2024 ........................................................... 44 Shi 52 Work Cited Deleuze, Gilles. The Fold: Leibniz and the Baroque. Rev. ed., Continuum, 2006. Foucault, Michel. "Discipline and punish: The birth of the prison, trans. Alan Sheridan." (1977). ---. “Of Other Spaces.” Diacritics, vol. 16, no. 1, 1986, pp. 22–27, https://doi.org/10.2307/464648. Haraway, Donna. Simians, Cyborgs, and Women: The Reinvention of Nature. Routledge, 2013. Kirchhoff, Michael David. “Material Agency: A Theoretical Framework for Ascribing Agency to Material Culture.” Techné: Research in Philosophy and Technology, vol. 13, no. 3, Nov. 2009, pp. 206–20. www.pdcnet.org, https://doi.org/10.5840/techne200913323. Kleinman, Adam, and Barad Karen. “‘Intra-Actions’ (Interview of Karen Barad by Adam Kleinmann).” Mousse Magazine, vol. 34, no. 13, 2012, pp. 76–81. Krissel, Matthew. The Architecture of Space and the Fold. Liu, Yu Zhen. Book Review: “Ru dao an xi yao fang bian fa men” from Dao Xin Chan Shi 道信禪師《 入道安心要方便法門》 研究. (2019): 100 Longpré, Philomène. Formica V.2. 2007, Philomène Longpré's website, https://www.philox.net/formica_system/. Accessed 13 Oct. 2023. Mulvey, Laura. Death 24x a second: Stillness and the moving image. Reaktion Books, 2006. Shi 53 Sommerer Christa et al. Interface Cultures : Artistic Aspects of Interaction. Transcript 2008. Steyerl, Hito. "How Not to Be Seen: A Fucking Didactic Educational .MOV File." Year of creation. Urry, John, and Jonas Larsen. The Tourist Gaze 3.0. 3rd ed, SAGE, 2011. Weerasethakul, Apichatpong, director. Tropical Malady. 2004. Shi 54 Work Consulted Cartwright, Lisa, and Marita Sturken. "Practices of looking." An Introduction to Visual Culture 2 (2001). Deleuze, Gilles. "Postscript on the Societies of Control." Surveillance, crime and social control. Routledge, 2017. 35-39. Jones, Rachel. "On the value of not knowing: Wonder, beginning again and letting be." On not knowing: How artists think (2013): 16-31. Lakatos, Mihály. "Sights and sounds of big data: Ryoji Ikeda’s immersive installations." Acta Universitatis Sapientiae, Film and Media Studies 18 (2020): 109-129. Sartre, Jean-Paul. The Transcendence of the Ego: An Existentialist Theory of Consciousness. Hill and Wang, 2003. Thompson, Krista A. An Eye for the Tropics: Tourism, Photography, and Framing the Caribbean Picturesque. Duke University Press, 2006. Shi 55 Post-Presentation Reflection In crafting this thesis, I started a journey that was as enlightening as it was challenging. Through the process of developing the Thesis Supporting Document, I engaged deeply with my own creative concepts, refining and redefining them to better align with the thematic core of my work. This process not only functioned in clarifying and solidifying what I did but also unpacked the work through the entanglement with broader philosophical and technological discussions. Through this process, I learned the importance of considering the viewer’s experience by guiding them through the text to understand the key concepts that I am trying to convey. I recognize that this contextualization is an area that I need to develop further. The Thesis Exhibition and Presentation were pivotal moments for reflection and revelation. It brought to light aspects of my work that remained unclear or underdeveloped. The dialogue and feedback received during the exhibition and thesis presentation were instrumental in pinpointing areas that needed refinement within my paper, allowing me to crystallize some of the complex concepts that I had struggled with. These experiences highlighted the importance of framing my work through a more personal lens. I learned that by sharing my ideas through a personal narrative, rather than solely through abstract, broad discussions, I could foster a stronger connection and empathy with the audience, making the concepts more accessible and resonant. Shi 56 Furthermore, the questions and feedback received during the exhibition and presentation are leading to new ideas for further exploration in my work. They prompt me to think about the future possibility of human-machine interaction within Mango’s looping story. As I explored the various visual representations in the video showcasing Mango's transformations, it became clear that humans and machines interpret images in markedly different ways. In this project, both Mango and the audience engage in a mutual examination through image-based information. Therefore, further explorations could involve various forms of storytelling– not only residing in the digital realm but also manifesting in the real world. The insights gained from crafting Mango’s digital story provide a foundational experience that now informs my efforts to translate this narrative into physical form, using alternative methods of narration that involve objects and sensory interactions. As I mentioned in the conclusion, this thesis project represents one iteration within the infinite lifecycle of Mango. Being a constantly shifting form, Mango’s transformations pave the way for the transition from digital to physical embodiments, enhancing the audience's engagement with the narrative. This entire presentation process has underscored Mango’s potential to inspire a broader exploration into how narratives can thrive across various mediums, continuously evolving and redefining their connections with the audience.