Updated: May 24, 2021
DOI: https://doi.org/10.33008/IJCMR.2020.32 | Issue 6 | May 2021
Chris Caines and Bettina Frankham (University of Technology Sydney, Australia)
Abstract
In line with the original intent of the grounded theory method, the creative practice presented within this multi-piece portfolio seeks to reach an experiential truth by combining the empirical context of the artwork being experienced with both the sense memory of place and the mediated consideration of embodied context evoked by the work. In doing so, we bring this element of approach into a hybrid conceptual model of reception. It is a model that relies, not only on mediated and augmented relations to real and virtual place, but also on the extremely hybridised media forms of virtual reality and locative media installation. These hybridised media forms blend gaming, cinema, sound art, visual arts and extended forms of narrative thereby creating multimodal experiences. The two works discuss, In a Minor Key and Libration Song, both of which were realised through different, mediated forms, provide contexts for audiences to connect a sense of place to emotions of loss and mourning as well as to an enhanced awareness of what it means to contemplate the grounded environmental reality of the sites we inhabit.
Introduction
The ecological conception [of place] insists that an important aspect of the concept of place is its groundedness in topography (Dirlik 2011: 55).
Groundedness as a method of research enquiry, developed by Glaser, Strauss and Corbin (Glauser, Strauss 1967 - Strauss, Corbin 1990), has transformed into an elastic and malleable collection of ideas applied in quite different ways across a wide range of fields. The concept has been focused by Dirlik (2011) on notions of place and topography as key elements of understanding and conceptualising the impact of groundedness as an intellectual tool. Our intention in presenting the two projects below is to make use of some of these ideas as starting points from which to consider approaches to using place, as virtual, mediated and embodied manifestations, with media. In both projects we use these creative investigations as ways to explore visceral embodied and imagined responses to landscape and environmental context as methods for evoking grounded experiences in an audience. In line with the original intent of the grounded theory method, the creative practice seeks to reach an experiential truth by combining the empirical context of the artwork being experienced with both the sense memory of place and the mediated consideration of embodied context evoked by the work.
In doing so, we bring this element of approach into a hybrid conceptual model of reception. It is a model that relies, not only on mediated and augmented relations to real and virtual place, but also on the extremely hybridised media forms of virtual reality and locative media installation. These hybridised media forms blend gaming, cinema, sound art, visual arts and extended forms of narrative thereby creating multimodal experiences. The works we discuss, realised through different, mediated forms, provide contexts for audiences to connect a sense of place to emotions of loss and mourning as well as to an enhanced awareness of what it means to contemplate the grounded environmental reality of the sites we inhabit. These two works, In a Minor Key and Libration Song, provide the means by which we relate groundedness, as a broad and evocative field of concepts, with metaphors of place and awareness of a site specific, environmental context.
In a Minor Key
In a Minor Key is a spatialised essay delivered via a virtual reality platform. The work is part of an ongoing exploration by Frankham of the relationship between meaning-making and aesthetic experience within the context of expanded documentary practice. The project adopts a place based approach to consider affective and embodied contemplations of grief via the affordances of interactive virtual reality. The work manifests as an explorable environment where the interactant gathers essay elements as they progress through the 4 levels of the experience. The essay form is applied to an interactive, game-like structure, so that audio recordings of thematically related musings, anecdotes, questions and observations are embedded alongside memory objects and the sounds of local native birds, within a south eastern Australian bushland setting. What results is an affective landscape where grief is explored metaphorically as an embodied, time-based and spatialised experience.
On the surface, a fundamental element in the appeal of VR is its ability to create immersive user experiences. The technology can produce an immersive effect by constructing an illusion of depth through stereoscopic imaging (which produces the appearance of three dimensions), surrounding the user in a 360° panoramic and spherical field of vision, layering in a convincing and cohesive soundscape with the user isolated from the actual environment by a head mounted display unit and headphones. However, as Ryan (2015) points out, immersion is not just a technical specification. The experience of immersion also relates to the way a maker can create the conditions for imaginative, emotional and intellectual absorption in the mind of the viewer, as is evidenced by the fact that it is just as possible to become immersed in a book as it is in a high end, technologically dependent, VR experience. The key to immersion then is not just the fact of the technological capabilities but a consequence of skilled choices around content, creation of a coherent experience, aesthetic approach and thoughtful application of relevant technologies.
Sitting within a tradition of expanded documentary practice, In a Minor Key builds on the creative trajectories of the essay film, installation art, serious games and interactive documentary experiences. Precursors to and influences on the realisation of In a Minor Key range from the moving image essays of Chris Marker (1983) and Agnes Varda (2000 and 2008) through to the work of Char Davies in her project Osmose (Davies 1998) and the VR experience created to accompany the feature documentary Notes on Blindness (La Burthe et al., 2016). The application of game logics within the work owes a debt to first person games utilising open world gameplay mechanics, such as The Legend of Zelda: Breath of the Wild (Nintendo 2017a), to collectathon-style platformer games such as Super Mario Odyssey (Nintendo 2017b) and to serious games such as Escape from Woomera (Oliver et al., 2003).
In the maker’s personal affective map, the location represented in the VR work is an approximation of a place that lay between the hospital where she visited her terminally ill sister and her own home. This terrain, of bushland situated on an escarpment, became an important emotional transition space, for moving between the intensity of holding the space of preparing for death, and returning to the demands of everyday living. Contemporaneous narratives of climate crisis, coupled with evidence of environmental degradation due to human impact and an awareness of looming species extinction brought about by contracting habitats, meant that this became a landscape freighted and overlain with grief in many forms.
In order to reference certain features of the actual location, the 3D environment of In a Minor Key was constructed through photogrammetry. This process involved scanning the bushland location, which encompasses a small waterfall, using drone photography. The data collected through the drone scanning process was then reconstructed as a virtual 3D environment. The intention in choosing this method for creating the virtual environment was to transpose the real location into the virtual experience as faithfully as possible.
However, an important finding from the creative research process was that bushland poses a particular problem for photogrammetry. Photogrammetry is excellent at capturing discrete objects, architectural structures and solid forms. It is easy to find online examples of buildings and archaeological sites that have been drone scanned and reproduced for remote exploration. However, the natural world, filled with trees, moving leaves and grasses, is not so easily rendered. The less dynamic base - rocks, boulders, the ground and cliff faces – posed fewer challenges. However, the life forms that grow on top, operating at a, comparatively, speedier timescale, were not so easily tamed. While key, underlying structural features of the chosen location are evident in the virtual environment of In a Minor Key, the actual flora of the location had to be removed due to errors in their reproduction as 3D objects. The actual trees, shrubs and grasses were then replaced with facsimiles from library sources.
At one level, this was a significant compromise with which to work. Connection to that specific place and the particularities of the bushland in that region, were significant in the network of ideas, feelings and sensory experiences being explored in the essay. However, at this stage in the technological development of VR technology, and within the budget allowed for producing the work, the issue of the ‘uncanny valley’ (Mori et al, 2012), with representations that are almost-but-not-quite-realistic and as a result are slightly unsettling, was always going to be a factor in the eventual aesthetic experience that could be created.
Compromise in the faithful visual reproduction of the location led to a strategy of leaning a little more heavily on sound to create the desired immersive experience and sense of being in place. Sound design has the potential to add dimension, a sense of emplacement as well as a level of polish and the impression of higher production values where budgets may be limited. To this end, In a Minor Key drew on ambisonic recordings of the original environment, combined with high quality recordings of native birdlife (their bird song and movement), breezes, winds, the rustling of leaf litter under foot and small creatures in the undergrowth, to evoke the auditory experience of being in the bush.
The sense of acoustic space within the environment changes as the interactant moves between the four levels of the experience. Level 1 starts with a muffled and deadened sound quality as the user is surrounded by fog and unable to see much beyond a couple of metres from where they stand. This acoustic space shifts as they progress through the levels, moving towards greater clarity and a sense of greater auditory depth as the fog lifts and they rise up for a bird’s eye view. While this experience is still not exactly like being there in the real, material, grounded location, the affective connections that can be triggered through sounds and shifts in sound quality, particularly those that might have personal meaning or currency as iconic representations, means that the interactant is enlisted as co-creator of the virtual space, drawing on their own memories, associations and imaginings to add depth to the virtually created environment.
The scanned bushland site was chosen for the multilayered significance it held for Frankham. In a geographical sense, it is a transitional space. The location, incorporating a waterfall, represented the change in levels between the highlands and coastal foothills. This topographical structure enables the work to be built on a metaphor for the experience of grief, moving between the depths of emotion and being inward looking (corresponding with dense fog and being low down at the foot of the waterfall), and on to some form of emergence where horizons begin to open up and stretch out (aligned with the interactant moving to the top of the waterfall and able to look out past the tree line).
In a Minor Key – video of opening fly through (Frankham 2021)
Part of the creative challenge was to communicate this affective significance realising that it may not align with the interactant’s own response to the environment. It therefore required a strategy of layering emotion into the space and this is where the essay elements come into play. This is achieved by adopting a formulation that identifies the essay elements of In a Minor Key as the things that layer in emotion and a sense of narrative meaning. This formulation has the effect that all aesthetic choices and items of content operate as elements in the essay. From voiceover anecdotes and musings, through to interactive memory objects, birdsong, the weather, the quality of light, and the positioning of the interactant’s point of view; all these are ways of thinking through the embodied experience of grief, unpacking its many forms and having opportunities to know something of it in multiple ways.
These essay elements are then linked through actions that the user can undertake, which in turn amplify the potential for tapping into embodied and sensory knowledges beyond the auditory and the visual. In moving through the space, the interactant experiences a proprioceptive awareness of their position in relation to the virtual forms around them - cliff face, waterfall, valley floor and sky above. With their feet on the ground, they are grounded and can access a range of viewpoints and perspectives, their range of movement limited through visibility conditions. The duration of attention to specific objects or areas of the environment becomes a mechanic for determining for how long a sound might play. The notes of the minor key chord progression that play out over the four levels of the work, are deeply seeded within the environment so that the cultural and affective associations of music in a minor key dawn gradually on the interactant. Finally the work culminates in the interactant unlocking the ability to rise up to the top of the waterfall, thereby attaining the birds-eye view that links them to the birds that have, so far, only been heard, as well as to the broader environmental context.
The essayistic becomes experiential as the interactant establishes embodied connections between a sense of ‘virtual’ place, direct contemplations of grief (through voiceover text) and indirect evocations of grief (through quality of light, tonality, sounds, and interactive objects). This network of connections is the method for activating the recreated place as an emotional landscape, a metaphor for the shape and phenomenology of the feeling. The intention is for individual truths to emerge from the experience of In a Minor Key that are products of the empirical context created in the artwork in combination with the internalised connections an interactant makes to the contemplative fragments and sensory experiences that are offered by the piece. Future research will dig further into the possibilities of the aesthetic experience that include, but also extend beyond, content and into site specific installation, invigilation practices and process for onboarding and returning to the outside world.
Libration Song
We’re trying to connect right away to the remembered experiences that your body knows […] The walks make you hyper aware of your environment around you. I thought it would take away from that because you put a headphone on and walk around with a Discman, but all of a sudden, your senses are alert. They say media kills your senses, but it is not true because it can actually enliven them (Cardiff 2005).
In the western Sydney harbour suburb of Rhodes, where the tidal reaches of that large body of water begin to transition into the Parramatta River, Libration Song was installed as a site specific environmentally synchronised video installation in early 2020. Commissioned as public art by the local council of Canada Bay the twin screen twelve-hour piece was installed in a community library, makerspace and gallery adjacent to the tidal edge of the waterline of the harbour foreshore.
The work comprises two videos (both six hours twelve minutes long) exactly synchronised to the lunar tide cycles present at the site. Over the course of the month it was on display the tides and the motion of the work travelled through an entire slow-moving cycle arriving at the same high tide interval at the same time on the last day of exhibition. The imagery and sound in the work refer to a shared mediated imaginary of the moon derived from the Apollo mission broadcasts. The snowy abstracted visuals originally generated on analogue video synthesisers and the audio constructed from hundreds of samples transmitted from lunar missions key into an audiovisual collective consciousness (Goodings, Lewis & Brown, Steven & Parker, Martin 2013) built from repeated exposure the snowy indistinct Apollo video broadcasts over a half century.
The context for the reception of the work was designed from the outset to facilitate an ambient lived embodied experience in place and environment. Crucially, the Rhodes Connection (as the venue is called) is a space that a local community that lives in nearby unit blocks frequents daily for study, socialising and play. As such this audience encountered daily the slow-moving transformation of the work throughout the lunar cycle. While frequent visitors were well used to the installation site as a location for art and media works, the very nature of this piece - where repeated exposure would reveal slow changes even to those paying little attention to the screen - was unique. The tidal inlet viewable from in front of the screens then invited a connection between this and the usually unnoticed daily and monthly movement of the waters metres away from the installation.
The conventions of speed and reception for public screens tend to be either fast paced at advertising/broadcast speed or static and informational (Miranda 2012). Libration Song uses the language of slow cinema (Weerasethakul 2010) wherein the viewer comes to contemplate duration and the passing of time as core elements of the work in addition to the locative bodily immersion of being aware of time based processes playing out slowly around the viewer. These latter aesthetic tools are derived from architectural and land art practices that frame or highlight larger environmental forces or movements in-situ, the light and portal works of James Turrell being key pieces (Turrell 2007) in the canon of pieces operating in this way. In media art and in music key examples of time awareness as content include the ‘639 year As Slow As Possible’ organ work by Cage (Cage 1987) and 24-hour synchronised film The Clock (Marclay 2010) by Marclay.
Libration Song combines these elements and extends this language by incorporating location and time-based media simultaneously. As argued by prominent locative media art practitioner Janet Cardiff (ref) locative media itself is a hybrid language where the in-situ, felt environment and the provided media experience combine to provide a third space where meaning as a combination of those elements is produced. While the moving image was also theorised in the early days of cinema (Birdsall 2012) as being akin to opera combining artforms to produce a Gesamtkunstwerk or total artwork. In harmonising these elements with the thread of much longer embedded environmental time spans in a particular location this work invites an interactant to incrementally consider a position as co-inhabitant of the natural world and the immediate physical surroundings. Key to this effect for the lay viewer, the non-art audience who formed the target audience for this work, is the experience of lived duration with the slow changes in the piece. Many of the regular visitors to the space in which the work was installed reported appreciating and investigating the ideas behind the work further after they had noticed it shifting slowly over time. This can be considered in contrast to how screen-based media art might be experienced by a general audience where attention may at first be given to gauge whether the work is of any interest then ignored thereafter if not. The key innovation and research achievement of the work is in finding a form to embody a grounded site responsive media work that prompts an audience to participate both imaginatively and viscerally in environmental changes and processes far beyond customary attention spans and then relate that experience to the immediate place in which they are experiencing the work.
Libration Song video from installation six hours duration (Caines 2020)
If the development and execution of this piece can be thought of as the early stages in developing a new affective hybrid language, further avenues for honing these ideas include extensions of visual music, hyper-extended narrative and the use of site specific environmental data in ways that go far beyond the data visualisation tropes we have become used to. These tropes have been particularly damaging to media art attempting to make poetic use of environmental and contextual data particularly in terms of audience expectation and reception. The question ‘what does it do’ as a response to work that makes use of technical and data driven elements has become an ossified cul-de-sac blocking understanding and engagement with work that can extend and expand our conception of how culture and environment interact. In foregrounding the poetics of how data sets and technological sensing can extend our sense of time and the unseen processes around us these media artworks can join practices of cultural production and engagement with larger macro movements in the natural context. In doing so hopefully these linkages can both provide a distance for appreciating the longer time scale cycles of the natural world and indeed how these cycles relate to our own lived and subjective time. As well as enabling the cultural engagement in these concepts to enter the creative narrative of arts production and reception including the role of the arts in societal understanding.
Conclusion
In thinking about both of these projects, we can discern the distinct outline of a shared approach relating to using notions of grounded place, both actual and virtual, as ways of revealing new approaches to groundedness. As two practitioners coming out of the media arts traditions of user interactivity and locative media you can also discern our direct lineage within the traditions of that flow from the concepts of the Memory Palace (Cicero 90) and underpin the spatial and conceptual metaphors of forms across gaming, narrative genre and even network and internet infrastructure models.
Taken together, our approaches to mediated place both as metaphor and as a tool for revealing environments in-situ enable an expanded understanding of the grounded through using a hybrid approach to contemporary expression of these ancient traditions. In expanding the conceptual framework of our mediums of virtual reality and locative situated media to reach toward expanding these ideas we provide a framework for addressing ideas normally outside the scope of the forms. Using hybridised creative approaches in this way also allow understanding and provisional forms of knowledge to arise between the cracks of hybridity making possible valuable new perspectives on the core lived experiences of how we make use of ground and environment culturally and emotionally.
References
Birdsall, C. (2012). Cinema as Gesamtkunstwerk, Nazi Soundscapes: Sound, Technology and Urban Space in Germany, 1933-1945, Amsterdam University Press.
Cage, J. (1987). As Slow As Possible, Halberstadt Germany, 639 year organ composition.
Cardiff, J. (2005). The Walk Book Thyssen-Bornemisza Art Contemporary Vienna.
Cicero, M. (90AD). Rhetorica ad Herennium. Earliest surviving historical mention of the method that goes back to pre-history. Accessed 22/3/21 https://www.laits.utexas.edu/memoria/Ad_Herennium_Passages.html
Davies, C. (1998). ‘OSMOSE: Notes on Being in Immersive Virtual Space.’ Digital creativity (Exeter) 9 (2) : 65–74.
Dirlik, A. (2011). ‘Globalization, Indigenism, Social Movements, and the Politics of Place.’ Localities 1 : 47-90.
Glaser, B.G, Strauss A. L. (1987). The discovery of grounded theory: strategies for qualitative research. New York: Aldine de Gruyter.
Goodings, L., Brown, S. and Parker, M. (2013). ‘Organising images of futures-past: remembering the Apollo moon landings;. International Journal of Management Concepts and Philosophy 7. 263 – 283.
La Burthe, A., Colinart, A., Spinney, J., and Middleton, P. (2016). Notes on Blindness [Virtual Reality]. France, Archer's Mark, Arte France and Ex Nihilo. https://www.arte.tv/sites/webproductions/en/notes-on-blindness/
Marclay, C. (2010). The Clock, 24hr time synchronised film.
Marker, C. (1983). Sans Soleil [Video] [Video]. The Criterion Collection, Kanopy.
Miranda, M. (2012). ‘Uncertain Spaces: Artists? Exploration of New Socialities in Mediated Public Space’, Scan Journal, ed John Potts. http://scan.net.au/scan/journal/display.php?journal_id=101
Mori, M., MacDorman, K. F. and Kageki, N. (2012). ‘The Uncanny Valley [From the Field],’ in IEEE Robotics & Automation Magazine, 19 (2), pp. 98-100 (June), doi: 10.1109/MRA.2012.2192811.
Nintendo (2017a). The Legend of Zelda: Breath of the Wild [game] (Nintendo Switch, Wii U).
Nintendo (2017b). Super Mario Odyssey [game] (Nintendo Switch).
Oliver, J., Wild, K., Honegger, S., Malcom, I., Blundell, A., Halliday, J., Harrigan, M., Taylor, D., and Markwart, C. (2003). Escape from Woomera [PC game - incomplete] (Microsoft Windows).
Ryan, M.L. (2015). Narrative as virtual reality 2: revisiting immersion and interactivity in literature and electronic media, John Hopkins University Press, Baltimore.
Strauss, A.L., Corbin, J.M. (1990). The basics of qualitative research: grounded theory procedures and techniques. Newbury Park, CA: Sage.
Turrell, J. (2007). Dividing The Light, Pomona College, California, permanent installation. 2007
Varda, A. (2000). Les Glaneurs et la Glaneuse [video] Potential Films, Kanopy.
Varda, A. (2008). The Beaches of Agnès [video] The Criterion Collection.
Weerasethakul, A. (2010). Uncle Boonmee Who Can Recall His Past Lives, Kick the Machine.
