DOI: https://doi.org/10.33008/IJCMR.2021.42 | Issue 7 | Oct 2021
Kerryn Wise (De Montfort University)
Abstract
Dis_place is a mixed reality performance that takes audiences on a journey using a range of virtual reality (VR) technologies, immersive sound, and live dance performance. Through close analysis of my practice as research project, this article presents reflections on the developing creative strategies and approaches to making VR-based mixed reality performance. It traces the creative process in the making of the work, combining links to the VR artwork, video footage of the live performance, and images from the project. This is combined with my observations and analysis of audience feedback. Through this analysis, the writing assesses the affordances of using VR technologies within immersive performance practices, addressing some of the technological, practical, choreographic, and conceptual concerns. Concluding that these technologies have huge potential for offering audiences new embodied encounters that can shift perspectives and produce transformational, intimate, emotive, and unsettling experiences. Dis_place VR should be viewed on a head-mounted display (HMD). It can be accessed through itch.io here and Viveport here.
Video documentation of Dis_place VR
Edited documentation of Dis-place Live
Introduction
Dis_place is a mixed reality performance [1] that takes audiences on a journey using a range of virtual reality (VR) technologies, immersive sound, and live performance. Through close analysis of my practice as research project, this article presents reflections on the evolving creative strategies and approaches to making VR-based mixed reality performance. As a choreographer interested in making intimate works, using new technologies, the practice aimed to create a performance encounter, offering new perspectives and experiences to the spectator through multi-sensory bodily engagement. Dis_place draws on established practices from immersive and one-to-one performance, dance, film, and digital artworks. This writing traces the creative process in the making of the work, combining links to video footage and images from the project, with my observations and analysis of audience feedback. Through this analysis, the article assesses the affordances of using these technologies within immersive performance practices. Furthermore, using VR technologies within live performance is a new area of investigation within the field and this examination aims to identify the techniques used and to unravel some of the technological, practical, choreographic, and conceptual concerns. Due to limitations of space, I have summarised the artistic concepts underlying the work.
Within this article, I use several terms that need explanation. I use the term ‘360-degree video’ (3DV) to refer to a spherical video file that is generated by a single omnidirectional camera or multiple cameras capturing a view in every direction recorded at the same time. When these images are stitched together using a software programme, they create a panoramic video that can be viewed from any direction in 360 degrees from the perspective of which it was filmed. This captured footage can then be viewed via a head-mounted display (HMD) or on devices through platforms such as Facebook or YouTube (Simcoe, 2018: 120). I use the term ‘Virtual Reality’ (VR) to refer to the immersive volumetric video work which allows the viewer to move around a room-scale VR experience [2], to see the work from multiple angles, which differs from seated or standing only VR experiences. I use the terms ‘participant’ and ‘spectator’ interchangeably rather than ‘viewer’, as these roles depend on the context, and highlight the more active role that the audience member plays in each element of the project.
Context
Dis_place is a one-to-one site-based immersive performance and VR experience which I created with Creative Technologist Ben Neal. Funded by Arts Council England, The Questlab Network with a range of local industry partners [3]. The first live performance took place in October 2019 at the People’s Hall in Nottingham [4], a decaying Georgian building with a rich history. Audiences were invited to experience a performance journey that moved across virtual and physical space through a combination of 3DV, immersive audio, and live dance performance. Dis_place highlights the memories and histories of the inhabitants and former uses of the building, revealed in the beautiful, decaying architecture, creating a unique, multi-sensory immersive experience.
The accompanying VR piece captured the People’s Hall space and dancers using the Microsoft Kinect sensor [5] with Depthkit software, using Unity to recreate a virtual representation of the space. This was presented at the Broadway Gallery opposite the People’s Hall building. This practice repurposes the Microsoft Kinect to capture dancers volumetrically [6] and places them in a virtually captured real historic site. The work guides the viewer through internal features of the space which are explored and highlighted by the digitised dancing performers. The interactive nature of VR allows the viewer to be part of a physical and performative response to the building expressed through movement and becomes an integral part of this new reality. Furthermore, the digital techniques enable new choreography, that would not be possible otherwise.
The key aims of Dis_place were to assess whether it was possible to integrate the 3DV into the overall immersive journey, using the virtual space to offer a different perspective and type of intimacy between the audience participant and the virtual and live performers. The practice aimed to explore how participants negotiate the layering of these intimate interactions across digital and visceral encounters, seeking to discover if a tactile closeness can be sensed through and across digital and physical immersive space. The project also acted as a prototype to test out new strategies and techniques when making this type of mixed reality performance work that uses new VR technologies.
The main development of this project was a two-week research and development phase that took place at the disused People’s Hall building in Nottingham in October 2019. The outcomes were prototypes for a one-to-one mixed reality performance and a VR dance work, both responding to the site. My collaborative team, which comprised a choreographer, four dancers, a creative technologist, lighting, sound, and costume designer, responded to the architecture and history of the Peoples Hall, to generate a site-based performance experience. Audience participants moved through the building experiencing 3DV technologies, immersive and live sound, with real performers who engaged with audiences through the mixed reality experience. This created a multi-sensory immersive experience which, accompanied by the VR piece, tested my central research questions through professional practice.
Furthermore, throughout this research, approximately forty-five participants experienced the one-to-one performance and thirty experienced the VR, resulting in twenty-five audience responses to the one to one performance and four in response to the VR piece. I gathered audience data via questionnaires, although some chose to submit audio recordings and freeform written responses. Some questions were adapted from Measuring Presence in Virtual Environments: A Presence Questionnaire (Witmer and Singer, 1998). The questionnaires were extensive and related to the overall experience, artistic content, engagement of the senses, connection to the dancers, as well as posing questions about the integration of the 3DV into the overall performance. This qualitative data has been examined using thematic analysis to understand audience responses to this type of mixed-reality performance environment and the associated questions it raises regarding choreographic approaches, director and spectator agency, and intimacy.
Research Process
I began the research process in March 2019, with several site visits, spending time with the unique atmosphere of the site, identifying distinctive physical features, and researching the history of the building. The creative team explored and responded to the building through a series of agile test and explorations to allow the site to inform the content of the performance. Alongside this, we had to negotiate the extensive technical elements of creating this type of mixed reality performance. Considering factors such as methods to enable the mixture of live and pre-recorded sound through a wireless Bluetooth network. Contemplating whether it would it be a guided or free-roaming experience, with multiple or single audiences, the placement of the 3DV and its integration within the overall immersive experience. This was in addition to all the usual production elements. The technical considerations for this work were immense and, in such a short development period, this had to be balanced with the artistic development.
Figure 1: The Peoples Hall building frontage and Broadway Gallery opposite.
Image credit: Kerryn Wise and Julian Hughes.
Initially, I had planned to combine the volumetric captured VR elements within the live performance, however, the limitations of the People’s Hall space, which was without mains power, hindered this. Therefore, the live performance used 3DV [7], and the VR became an interrelated element which was shown in the Broadway gallery opposite the People’s Hall. On reflection, this separation allowed me to compare the potentials of both technologies, focusing on the intentions within my practice, leading to some useful insights. In Figure 2, I have identified several key features of both 3DV and Volumetric VR, important within my work, and compared them [8]. The horizontal axis identifies a feature and the vertical axis suggests how much scope there is to achieve in each technology, the higher the number the more potential the technology has [9].
Figure 2: Comparison chart, 2020.
Dis_place: Volumetric room-scale VR
The first stage of the project involved exploring the potentials of volumetric video, a technology that expands the boundaries of 3DV further and enables some of the affordances of computer-generated VR. Volumetric capture extends audience agency in 3DV virtual space, by allowing spectators the ability to move freely within a room-scale virtual environment (VE) and view the digitised dancers from multiple perspectives. Our explorations involved ‘hacking’ the normative use of the Microsoft Kinect in conjunction with Depthkit to volumetrically capture the dancers, cost-effectively, and to place them within a virtual space created using the Unity software programme.
Figure 3: stills showing how the Kinect camera captures depth using infrared, 2019.
Our explorations initially tested how the camera captures the moving body. At the time of developing this work, Depthkit only supported the use of one Microsoft Kinect [10], which meant that the capture zone was quite small, and the dancers’ hands and feet kept getting clipped outside the capture area. Through this experimentation with space, we noted that if you were to isolate and capture a specific body part, it had to be dismembered at the correct point on the body to avoid looking macabre when placed in the virtual space. For example, a whole torso or full arm seems natural; however, if the body part is cut at the elbow or mid-thigh, it seems unnatural. Although the effect we eventually worked with had a fractured aesthetic, moving away from photorealism [11], we did not want to move towards the horror genre, or for the spectators to experience the repulsion associated with the uncanny valley [12].
Returning to the process, we experimented with different types of movement, including combining single and multiple dancers in the capture zone and experimenting with full-body movement and isolating specific body parts. We found that the smaller gestural sequences, involving intricate actions with the hands and arms whilst standing, worked best to capture the whole body. Furthermore, we experimented with how the dancers entered and exited the capture space as this would impact how they entered the virtual space, which influences how the spectator may interpret and engage with these virtual dancers. It was difficult to capture two dancers moving in contact together, or layered spatially, as the ‘shadowing’ that one person throws against another (caused by blocking the infrared sensor) causes a hole to appear in them as the sensor can no longer reach them. We realised that any layering to give the effect of the dancers moving together in contact would have to be created in the virtual space.
Paul and Levy’s term ‘glitch’ refers to ‘images and objects that have been tampered with…and these images can be created by adjusting or manipulating the normal physical or virtual composition of the machine or software itself, or by using machines or digital tools in methods different from their normative modalities’ (2015: 31). I began to make aesthetic decisions guided partly by how the technology was capturing and rendering the dancers’ bodies, alongside how these visual choices could influence the viewer’s potential engagement with the virtual dancers. Several visual style choices can be used in Depthkit, from very pixelated graphics to high fidelity photorealistic resolution. I decided to capitalise on the fragmented, glitchy quality, which balanced semi-realism with fractured edges. I felt that this style choice complemented the visual qualities of the decaying building we would be working in for the final capture, enhancing the eerie qualities within the work.
Figure 4: still of Dis_place VR, showing fractured edges of performers and space, 2019.
Furthermore, I intended to balance the slightly eerie quality this style creates, whilst avoiding the dancers becoming too pixelated and therefore alien to the viewer. My aim was for the spectator to be able to find some connection to the dancers and their narratives told through the movement, and to cultivate a sense of intimacy between dancer and spectator within the virtual realm. I hypothesised that this intimacy could be enhanced by the spectator’s ability to move, towards, through, and close to the dancers spatially; however, I was equally aware that these digital representations could also appear inhuman and thus, provoke limited engaged connection.
For the final VR work, the choreography was developed in response to the site’s heritage and suggests to the viewer the unknown histories of the space. The viewer is guided through internal features of the captured space, which are highlighted by the digitised performers. The interactive nature of this type of room-scale VR allows the viewer to be part of a physical and performative response to the building expressed through movement and becomes an integral part of this new reality. To begin, we separately captured each wall of a room in the People’s Hall building and the choreographic phrases using the Kinect sensor with Depthkit, utilising the learning developed from stage one. Using Unity, the creative technologist recreated a virtual representation of the room and we then began the process of choreographing the captured performers within the VE. Choreographing in virtual space is an innovative area of research, where the composition potentials are significantly expanded.
Figure 5: still from Unity work, 2020.
As we began this process, we were increasingly aware of how many choices were available using this method; we originally had over a hundred clips which we reduced to thirty-five. These were of three dancers which could be placed anywhere within the virtual space, clips could also be layered and duplicated. In a process like traditional video editing, we were working on a timeline to a five-minute composed soundtrack that had been developed in response to the building. We began by giving the spectator time to acclimatise to this new virtual space, offering a chance to explore their surroundings. After this acclimatisation, one dancer appears; the rationale being that I did not want the spectator to be shocked by the sudden appearance of multiple dancers all around them. I decided to create the effect of the dancers appearing from inside the walls, which adds to the eerie atmosphere created. This aimed to enhance the sense that the dancers belonged to or were part of the building and its history.
Figure 6: still from HMD VR view, Dis_place VR, 2019.
In designing the choreography within this three-dimensional virtual space, initially, I placed the dancers on the periphery, gradually introducing the different dancers successively. At times, the dancers’ presences overlapped in the space and looked like they were dancing in contact, or the dancers were placed in different parts of the room, making the spectator choose which to watch. As the piece progressed, I gradually brought the dancers closer to the centre of the room, which is the location that the spectator initially finds themselves in. We wanted the dancers to progressively get nearer to the spectator, building a more intimate relationship and allowing them a closer look. However, this relies on the spectator staying relatively still and central, which is contrary to the affordance of this technology; the spectator’s ability to move around the room-scale environment. Interestingly, it became apparent that the spectator's agency to move reduces my agency as a director to control the developing choreographic vision [13]. This artistic control had appealed to me in my previous work with 3DV and live performance, Exposure (2017) [14], where I played with the viewer's lack of agency. In 3DV, the viewer can look around; however, they cannot move in the virtual space and thus can be caught in a particular viewpoint or relationship with a performer and this can become a powerful device for a director.
In the VR work, whilst audiences reflected on the benefit of being able to move freely and ‘look at the movement from different angles’, there were differing responses to how connected audiences felt to the dancers. One respondent suggested that ‘the “glitchiness” and disappearing into walls made it clear that I was looking at digital representation so did not directly feel connected to them’. However, other responses suggested that ‘the feeling of connection grew throughout – probably as I got closer to the dancers’. Another highlighted that ‘the three-dimensionality of the dancers made them “real” but still removed from my immediate senses’. Likewise, these responses offer a range of perspectives on the reading and interpretation of the digital bodies within this work. Highlighting several factors including the effect of aesthetic choice in the capturing, and the positioning and closeness of the dancer to the spectator in the choreographic spatial design, elements I am exploring further as the practice develops. Overall, there was a distinctly lower level of connection experienced between the audience and digital performers in the VR work than in the live performance which combined digital technologies with live action.
Dis_place: one to one live performance
Figure 7: still from Dis_place live. Image credit: Julian Hughes, 2019.
The live performance took one spectator at a time on a journey that moved through a physical building where they encountered live performers who they followed, and as part of this journey, they were invited to experience 3DV via a HMD. This was used to offer the participant new perspectives on the choreography and as a shifting point in their relationship to the performers in the work. As in much immersive theatre work, there is a performer guide role, which acts as an anchor to the participant, managing their journey and easing the negotiation of the practicalities the technology. In this work participants also wear a pair of bone-conducting headphones [15] which allows them to hear the pre-recorded sound as well as the live performers and the amplified parts of the building, furthering the elements of mixed reality. They are given these at the start as part of the onboarding [16], to make sure they are comfortable and working. One of the issues with using HMD based VR in live performance is how the participant puts on and takes off the headset without it disrupting the overall sense of immersion [17]. The best method I have found is to have the guide place it on and off for the participant as part of the performance. In response to the question ‘How distracting did you find putting on / off the headset? participants noted that ‘I thought it was fairly seamless’ and ‘it worked well with the guide’. Having a performer guide to negotiate this whilst keeping the chosen atmosphere and extending the narrative can work effectively.
Returning to my initial research question which aimed to understand how 3DV technologies can be integrated within live works, there was a question posed within the feedback which asked, ‘How did the 360 VR experience fit within the overall live performance?’. Spectators responded that ‘it enabled more things to happen that would not have been able to happen in reality’, another indicated that it ‘added to the overall impact significantly at being transported through time and space’. Another said that ‘it transformed the space, but the textures of the film and the real smells continued with the movement and sound made me feel like I was inside the walls/fabric of the building’. We decided that the headphones would not be removed for the 3DV element, which aided the continuity for the spectator, as the 3DV visuals were timed with the soundtrack that the participant was already listening to throughout the performance. Another respondent noted that ‘it felt like a natural extension – continuing the images, connections and expanding the perspectives possible’, which is what we had set out to achieve, as well as offering the participant new viewpoints and a unique engagement with the live and virtual performers. Overall, the feedback was positive concerning this, however, two respondents noted that ‘the introduction to it seemed disconnected’, and that ‘the stopping to put the headset on broke the experience a little, however, the gentle guidance kept the performance continuity’. Largely, I think that this was well-managed, although it is always going to be a transitional point and needs to be considered carefully [18].
Figure 8: still from Dis_place live. Image credit: Julian Hughes, 2019.
In the development of this work, I was interested in framing certain views and perspectives of both the building and the dancers interacting with it. I often use framing devices from traditional film and photography to capture images that I display to the audience. This work was no different, and although the spectators can look anywhere, I was carefully curating both the journey they take through the building and the ‘framed’ images they see on that journey. I concluded quite early that the route needed to be set and would be for one audience member at a time, rather than a free-roam experience, which resulted in limited spectator agency within this structure. However, this meant I could control the timing, pace, imagery, negotiation of the technology, and order of scenes. Drawing on techniques used in immersive theatre practices, subtle lighting cues, sound, and the performer's actions were used to guide the participant's attention both in the physical and 3DV space, and the guide was on hand, should participants get disoriented.
Engaging the senses and finding intimacy across physical and virtual space
In recent years, several performance works have been produced which combine live performance with VR technologies including; ZU-UK’s Goodnight, Sleep Tight (2017), Curious Directive’s Frogman (2017), and Draw Me Close by Jordan Tannahill (2019). In his recent article, Harry Wilson considers the effects of bringing VR technologies to live performance, stating that:
…their specific modes of engagement and the ways of seeing, feeling and being that they produce are the unique result of the meeting point between virtual reality technologies and live performance practices. Furthermore, the specific forms of embodied spectatorship afforded at the intersection of VR and live performance can facilitate the movement between the actual and the virtual, intimacy and distance, immersion and making strange: producing very real and potent effects (2020: 115)
Wilsons’ reflections mirror my experiences of making works that combine intimate and immersive performance with VR technologies. The practice developed for Dis_place and its analysis support Wilson’s claim that this work can produce new, embodied experiences for audiences.
Concepts of embodiment were investigated in the questionnaire to understand how the participant's senses were engaged within Dis_place. The notion being that when spectators have their senses heightened, this can lead to more embodied experiences. Our findings noted, as expected, that participants were engaged by the sound and visual qualities, with many also mentioning the smells encountered in the building. One noted that their senses were ‘sharper’, with another noting that ‘I had a heightened awareness of space, noise, touch, movement’. Josephine Machon’s extensive writing about immersive theatre practices and her concept of (Syn)aesthetics, evidence that this type of multi-sensory engagement is common in immersive theatrical practices (2009; 2013). However, several respondents mentioned how their sense of their body and physicality was significantly intensified by the 3DV experience. The shifts in perspectives afforded by the unique 3DV perspectives, and the immersive quality of this type of virtual space, added to a fuller range of sensations for the spectator. A participant commented that ‘all the senses were combined with digital as well as real-life experiences which were disorienting at times but brought so much more to the experience of the building’.
The one to one immersive performance had the overarching idea to take participants on a multi-sensory journey that gently shifted perceptions of reality and questioned notions of real and virtual, past and present. Offering spectators new ways to experience the building and their embodied presence within it. Furthermore, the work aimed to establish a connection between the participant, the performers, and the spaces they shared across the journey, which encompassed both physical and digital environments. Questioning in a post-digital era, how audiences connect with both digital and live performers. I intended to test whether it was possible to use the virtual space to offer a different perspective and act as a transformational space that could shift the relationship between the spectator and the performer, which could be both comforting and unnerving. Scholar Sarah Whatley states that ‘virtual environments do not imitate live performance but visualisations can awaken the senses through an awareness of orientation, dislocation or displacement’ (Whatley, 2012: 277). Whatley also asserts that ‘immersive viewing environments, provide the viewers of virtual bodies an intense and transformative kinaesthetic experience, quite different from what is produced in a “live” encounter with real dancing bodies’ (2012: 266). I propose that if we combine both virtual and live performers within an immersive performance experience, new transformative embodied experiences can be created.
Dis-place was an ambitious project, and this analysis acts as a starting point for addressing some of these complex questions. In terms of experiencing connection, participants were asked to score from one to four how connected they felt with the dancers and to comment on their experiences. As expected, there was a slightly stronger connection between the spectator and the live performers, with most respondents scoring a four. However, there was also a significant number of responses with a score of four for feelings of connection with the virtual performers, with most scoring three or above. Interestingly, many had given the same score for the connection with both virtual and live performers suggesting that the outdated concept of real and virtual as binary opposites is no longer useful [19]. One participant noted that it was ‘Interesting that no particular difference’ when comparing the scores. My interpretation is that this is due to the timing and placement of the 3DV within the experience, the ongoing soundtrack, and primarily the fact that the virtual performers were the same as the live dancers, which created a sense of authenticity and continuity that aided these feelings of connection.
The Shifting Gaze of the Performer and Role of the Audience Participant
The role of the audience participant can be generally separated into pre and post 3DV within this work. Before the 3DV experience, the participant is an unseen witness, disregarded by the live performers who are preoccupied with their actions. Participants follow these performers who lead them through space, highlighting key features, yet the performers rarely directly face or acknowledge the participant’s presence. During the 3DV experience which takes place around halfway through the journey, the role of the participant shifts as the performers gradually begin to acknowledge their presence. The use of the digital performers’ gaze changes as they begin to look directly at the viewer through the camera’s lens. This use of the performers’ gaze is built up gradually through the rooms visited virtually in the 3DV space [20]. The dancers begin in front of you as you look out from a coal filled fireplace, this first perspective offers you a more traditional framed viewpoint, at the end of this scene one of the dancers notices you for the first time, their gaze is curious, yet distracted. One participant stated that:
In the final sequence one of the women seems to notice me (not ‘me’, of course; the camera) and moves closer, peers closer in a way that would again be uncomfortable in the real space but is okay again here because the screen, this hyper real VR screen is different from a screen; but also the woman is becoming enormous as she peers closer in a very weird unsettling way…
In the ensuing scene, the dancers are on either side of you in a narrow corridor with paint peeling off the walls, which looks almost as if it has been submerged underwater. This proximity forces you to choose who to focus on, as the performers begin to look directly at you, subtly encouraging you to watch them, participants noted the seductive quality of this shifting gaze.
The final 3DV scene begins with a view through a doorway into a cement washroom, aged with dirt. You transport forward virtually entering this room and find yourself very close to the performer, she directly gazes at you as she dances, and as she leaves beckons you to follow her. She is leading you back into the physical space, and when you remove the headset, she is waiting for you to continue the journey together as companions. She hands you a developing polaroid that has been taken of you whilst you were away in the virtual space, this captures and marks your presence in the building using old technology to witness new technology, yet also provokes a sense of shifting power relations. A participant noted, ‘What picture did they take while I was “away”, lost in the VR space, exposed and vulnerable to the other people in the room?’.
Figure 9: Polaroid image taken during the performance. Image credit: Julian Hughes, 2019.
This is a pivotal moment in the performance and the participant/performer relationship has shifted as the participant’s role changes from passive witness to active participant. The participant is now openly acknowledged through the performers’ gaze and use of touch, their presence has been captured by the polaroid image that marks the point of this shifting dynamic between participant and performer. Following this, the participant is taken by the hand and physically led towards the next part of the journey by the performer and encouraged to feel the texture of the building’s Georgian cornicing as you both descend the large, grand staircase. One participant commented:
She takes my hand and guides it to the wall…She guides me to feel the texture of the plasterwork; to experience what she was just experiencing. She guides me down the stairs. I’m still holding the polaroid. It still hasn’t developed. This feels as though we’re coming towards an ending.
Conclusion
This article has traced elements of the technological, practical, choreographic, and conceptual journey of Dis_place – as a live performance, VR work, and research project which broadly aimed to understand the potentials of using 3DV and VR technologies within live performance. I have offered creative strategies, reflective commentary, and audience feedback to highlight some of the affordances and issues of integrating these new technologies into live work. My analysis has led to the creation of a chart to compare the different affordances of 3DV and Volumetric VR important within my practice, focusing on features including audience connection, choreographic potential, and agency; suggesting that 3DV offers more control to the director, which is less possible in free-roam VR experiences.
This highlights that different strategies need to be employed to guide the audience in VR, methods which can be fruitfully drawn from immersive theatre practices. It also reveals the extended choreographic potential of volumetric VR. Furthermore, I have explored the relationship experienced between the spectator and the performers in the works. Finding that in the live performance, similar levels of connection were felt between spectators with both the live and digital performers, noting the lack of separation between real and virtual representations. I also noted that the participant's sense of their body could be significantly intensified by adding 3DV scenes to the wider immersive experience. Overall, it highlights the huge potential that these technologies have for offering audiences new embodied encounters that can shift perspectives and produce exciting, intimate, emotive, and unsettling experiences.
References
Auslander, Philip. 2008. Liveness: Performance in a Mediatized Culture. 2nd ed. Abingdon: Routledge.
Benford, Steve, and Gabriella Giannachi. 2011. Performing Mixed Reality. London: MIT Press.
Broadhurst, Susan, and Josephine Machon. 2011. Performance and Technology: Practices of Virtual Embodiment and Interactivity. Basingstoke: Palgrave Macmillan.
Curious Directive. 2017. Frogman.
Dixon, Steve. 2007. Digital Performance: A History of New Media in Theater, Dance, Performance Art, and Installation. London: MIT Press.
Giannachi, Gabriella. 2004. Virtual Theatres: An Introduction. London: Routledge.
Giannachi, Gabriella, and Nick Kaye. 2011. Performing Presence: Between the Live and the Simulated. Manchester: Manchester University Press.
Machon, Josephine. 2009. (Syn)Aesthetics: Redefining Visceral Performance. Basingstoke: Palgrave Macmillan.
Machon, Josephine. 2013. Immersive Theatres: Intimacy and Immediacy in Contemporary Performance. Basingstoke: Palgrave Macmillan.
Maravala, Persis Jadé, and Jorge Lopes Ramos. 2017. Good Night Sleep Tight.
Paul, Christiane, and Malcolm Levy. 2015. “Genealogies of the New Aesthetic.” In Postdigital Aesthetics: Art, Computation and Design, edited by David M. Berry and Michael Dieter, 27–43. Basingstoke: Palgrave Macmillan.
Rouse, Margaret. 2016. “What Is Room-Scale VR (Room-Scale Virtual Reality)? - Definition from WhatIs.Com.” WhatIs.Com. 2016. https://whatis.techtarget.com/definition/room-scale-VR-room-scale-virtual-reality.
Simcoe, Peter. 2018. 360 Video Handbook. [s.n.].
Slater, Mel, and Sylvia Wilbur. 1997. “A Framework for Immersive Virtual Environments (FIVE): Speculations on the Role of Presence in Virtual Environments.” Presence: Teleoperators and Virtual Environments 6 (6): 603–16.
Sparks, Matt. 2019. “Metafocus: Avoiding the Uncanny Valley in VR & Serious Games.” Learning Solutions Magazine. December 26, 2019. https://learningsolutionsmag.com/articles/metafocus-avoiding-the-uncanny-valley-in-vr-serious-games.
Tannahill, Jordan. 2019. Draw Me Close.
Whatley, Sarah. 2012. “The Poetics of Motion Capture and Visualisation Techniques: The Differences between Watching Real and Virtual Dancing Bodies.” Kinesthetic Empathy in Creative and Cultural Practices. https://pureportal.coventry.ac.uk/en/publications/the-poetics-of-motion-capture-and-visualisation-techniques-the-di-2.
Wilson, Harry Robert. 2020. “New Ways of Seeing, Feeling, Being: Intimate Encounters in Virtual Reality Performance.” International Journal of Performance Arts and Digital Media 16 (2): 114–33.
Wise, Kerryn. 2017. Exposure. Dance and 360 Video.
Witmer, Bob G., and Michael J. Singer. 1998. “Measuring Presence in Virtual Environments: A Presence Questionnaire.” Presence: Teleoperators and Virtual Environments 7 (3): 225–40.
Notes
[1] Benford and Giannachi outline the term as ‘intended to express both their mixing of the real and virtual as well as their combination of live performance and interactivity’ (Benford and Giannachi 2011, 1). Benford and Gianacchi provide a detailed exploration of definitions of mixed reality performance in their book, Performing Mixed Reality (2011).
[2] Margaret Rouse provides a clear summary of room-scale VR, stating that ‘Room-scale VR…is the use of a clear space to allow movement for someone using a VR application such as virtual reality gaming. Being able to physically move within the space helps to replicate real-world movement for the user and make the virtual environment seem more real. The term room-scale distinguishes that type of setup from the self-contained environment of a VR room and from seated or standing VR, in which the user remains stationary’ (2016).
[3] Industry partners included Dance4, the National Dance Agency based in Nottingham, NearNow; Broadway Media Centre’s studio for arts and technology, and Nottingham City Council’s Heritage Team.
[4] The People’s Hall is a dilapidated Georgian building with a rich heritage. It was built in 1750 and through the Heritage Lottery Fund is due for renovation in 2020.
[5] The Microsoft Kinect was first designed as a motion sensor camera used in conjunction with the Xbox 360 computer console. Depthkit is a creative software tool developed to provide XR developers with access to low-cost volumetric capture using the Microsoft Kinect depth camera. Unity is a cross-platform game engine, used to develop video games, it is one of the primary engines for developing VR content, alongside Unreal Engine.
[6] Volumetric video capture is a process that involves multiple cameras capturing the volume of an object or performer from different angles, which are then combined to create a realistic 3D digital video asset that can be placed within a range of VR and Augmented reality (AR) virtual environments. See Dimensions Studio for professional volumetric capturing https://www.dimensionstudio.co/solutions/volumetric-video
[7] 3DV can be easily shown in a wireless VR device, such as the Oculus Go. Thus, making this a more usable device for site and outdoor work without access to mains power.
[8] Each feature will be discussed further in the subsequent sections.
[9] This chart is focused specifically on the intentions within my own work and is not a general comparison of the potentials of 3DV and VR.
[10] In summer 2019, Depthkit released software to support the Microsoft Azure, with plans to support multiple camera capturing from Autumn 2020.
[11] Photorealism in this context refers to the rendering of computer graphics to create highly realistic imagery.
[12] The Uncanny Valley is a concept first developed by Masahiro Mori in relation to robots, which has since been developed to describe animated versions of humans used in games and film. As the visuals become increasingly realistic, any slight defect could result in feelings of repulsion in the viewer and a lack of emotional connection with the animation (Sparks 2019).
[13] There is much work being done in the field to develop narrative storytelling methods in virtual space. Using sound, action and lighting cues to guide audience attention. For this research phase we did not have enough time to explore these techniques fully within the VR work, although we are now working on using triggers which are activated by participants actions. However, we did use these techniques for guiding audience attention within the live performance.
[14] Information about Exposure can be found at kerrynwise.co.uk/exposure
[15] Bone-conducting headphones sit on the outside of the ear and sound is transmitted through vibrations on the head and jaw bones, creating the effect of hearing the recorded sound and any live sound from external sources combined.
[16] Onboarding is a term which is increasingly being used to describe how participants are prepared to enter virtual spaces.
[17] I am using the term immersion here in the theatrical sense, to encompass the overall performance experience, rather than as a term to describe the 3DV virtual space. See Mel Slater and Sylvia Wilbur article A framework for immersive virtual environments (FIVE): Speculations on the role of presence in virtual environments, for specific discussion of presence and immersion in purely virtual environments (1997, 606).
[18] It also raises an important question about what the 3DV offers to the overall experience. There is not the space to discuss this fully in this article, however, it is crucial to understand why the chosen technology is used. We are still in the infancy of this mediums development and thus drawing from Mark Cogniglio’s discussion in his article Materials vs Content in Digitally Mediated Performance, he states that early experimentation with new technologies often relates to the technological potentials before the work can become content driven (Cogniglio, M. in Broadhurst and Machon 2011, 78–84). As we move towards more affordable and accessible technologies, artistic content driven work will begin to emerge more.
[19] There has been much scholarly discussion of the terms ‘real’ and ‘virtual’, with current thinking moving away from considering these as binary opposites and towards a more integrated use of the terms. For a detailed discussion see (Dixon 2007; Giannachi 2004; Giannachi and Kaye 2011). Also see Philip Auslander’s text Liveness: Performance in a mediatized culture for further discussion of live and mediated performance (2008).
[20] The rooms visited in the 3DV virtual environment are also spaces within the People’s Hall building not used within the live experience.
Comments