Landscape Composite: Remediating Landscape Paintings through Visual Effects

DOI: https://doi.org/10.33008/IJCMR.2020.26 | Issue 3 | April 2020

Sam Wilkins

Bath Spa University

Abstract

This multi-piece portfolio examines the development of ‘Landscape Composite’ (2016-) a series of single screen video composites that remediate, through visual effects techniques, existing landscape paintings in the pastoral tradition of the 17th, 18th and 19th century. This portfolio will present the development of the works culminating in the exhibition ‘Post Pastoral’ at Centre Space Bristol (2020).

It will consider the theoretical context surrounding the work, present the methodology in creating the works and consider the intermedial links between painting and visual effects by examining specific works in detail. In terms of process, ‘Landscape/Composite’ applies visual effects techniques to remediate and animate landscape paintings to create ultra high definition tableaux vivant video works presented on large scale video screens. It uses Terry Gifford’s (1999) Post Pastoral idiom as the contextual frame for the remediation. The project provides an original contribution to the emerging trend of artists’ use of visual effects techniques to reexamine painted images of the landscape and reconsider our cultural preconceptions of the landscape and concept of the Pastoral and the picturesque.

https://postpastoral.tumblr.com/

Research questions and aims

The Pastoral, as a concept, has been explored in literary and art theory but has not been explored as rigorously within new media and moving image making. Landscape paintings and landscape aesthetics are and continue to be used to drive new media and video art creation. This project aims to ask how can the remediation of pastoral landscape paintings using visual effects be used to examine our relationship with the pastoral? The project aims to examine how novel methods of remediation in the form of moving paintings or animated tableau vivant can be used to encourage an audience to examine or question their relationship with these traditional landscape images. It also aims to explore the interdisciplinary approach exploiting literary theory relating to the pastoral as the contextual frame for the remediation. The method used is a growing area in which established works are animated to recontextualise them and it is in this method that I situate the analysis.

Watermarked clip of ‘Landscape with Nymph and Satyr Dancing after Claude Lorrain’ (2020) Single Screen UHD Video

Context, compositing and the landscape

In his 2017 lecture on the Pastoral Malcolm Andrews undertakes an analysis of the 1943 poster of a shepherd in a landscape by Frank Newbould. He examines how the image was constructed using two distinct images; a photograph of the south downs and an image of a shepherd with his dog taken from the Times newspaper. He states:

The splicing together of the two different landscapes strengthens both theme and composition. It conveys a sense of a timeless rural world set in sharp relief from catastrophic convulsions (2017: 2).

The image contains a shepherd, a small country cottage, rolling hills and sprawling downland and 'composes a kind of omnium gatherum of iconic English landscape features' (ibid.). What he calls ‘splicing’ we would now term compositing; a term that is used by visual effects companies and image manipulators, students with Photoshop or HTML5 image manipulation software to combine images together. In this process they make a technical act - combining the images either realistically or not and an ideological act - bringing to pieces of media/text that were never linked together to create new meaning.

The technique and process is not new and intertextual play and remediation (Bolter and Grusin, 1999) continues to grow both as an academic field and creative process. Modern computers and software have made Hollywood level visual effects processes available to artists and amateurs alike and these ‘splicing’ techniques can now be performed with moving images with relative ease. Within film the manipulation of the landscape is common place with skies replaced, hills and mountains changed and made more dramatic and they use the language of the Capriccio [1] landscape painting regularly without critical thought or examination. In tandem artists now use these same techniques to create much more critically engaged reexaminations of landscape images.

Certain paintings are remediated regularly. In her 2014 article for Screen, Agnes Petho examines the prevalence of intermedial recreations of Bosch’s ‘Garden of Earthly Delights’ she writes:

a number of experiments that interpret this as a challenge to bring properly to life the already lively picture or to animate the motionless figures. This challenge has been both artistic (to create and recreate similarly intriguing images of in-betweenness) and technical (to explore possibilities of media transpositions) (2014: 473).

Emily Allchurch utilises a similar method to recreate landscape paintings using photo manipulation software. She uses the capriccio technique to create fantastical landscape and architectural composites to explore modern cities and vistas, remediating Breughel’s ‘Tower of Babel’ (1563) in her series ‘Towers of Babel’ (2005 -2018).

In the work ‘In Pursuit of Venus-Infected’ (2017) by Lisa Rheihana a twenty five metre long multi screen projection uses this approach to remediate the nineteenth century wallpaper - the famous multi-blocked ‘Les Sauvages de la Mer Pacifique’(1806) as designed by Jean Gabriel Charvet. She animates the work and remediates through the postcolonial lens, by filming tableau vivant of the native Moari figures against green screen and then compositing them into one giant gently scrolling film that examines the colonial bias of the original work and creates a new technical achievement that was well received at the 2017 Venice Biennale.

Landscape, the pastoral and remediation/intermedial approach

On defining the pastoral, Terry Gifford (1999) identifies distinct idioms in examining texts made of the pastoral; pastoral (idyll), anti-pastoral and post-pastoral. He finds that works in the post-pastoral mode critically engage with and reflect on both how the pastoral has been used to present these ’idylls’ (pastoral) and works that represent the reality of the hardships and strife of actually working in the rural environment (anti-pastoral) and finds writers and artists who create works that examine the tensions that lie in these representations.

There is an interesting tension in taking painting works that sit firmly within the idyll tradition of the 18th and 19th century and using the post pastoral approach to remediate them for new audiences. These paintings as acknowledged in the introduction to the Art and the Environment Conference (2017):

The British countryside, largely mediated by the visual representations of eighteenth-century landscape painters, has now become artistic heritage, part of a national identity.

By utilising Gifford’s idiom this re-examination of the landscape paintings can be framed in a way that exploits Petho’s challenge by exploring both the technical act of animating the painting but also in the artistic act to create challenging and ‘intriguing images of in-betweenness’ which I will now examine in more depth.

Method

Utilising Christopher Frayling’s model for art and design practice research (2003) each work begins with research ‘into’ [2] art history with the identification and examination of specific landscape works. For this series works are generally British (but not always, I also choose works that were popular with British buyers and present often Italianate idylls, capriccios and rural fantasies that were popular at the time) and usually of the 18th and 19th century and sit in the idyll tradition. Aesthetic analysis is made through content analysis through the post pastoral lens.

The research ‘through’ practice then begins. Each work begins as a high resolution download. The work is digitally ‘cleaned’ using photo manipulation software, where all elements such as people and animals are removed. I then begin a process of iterative experimentation of compositing elements onto the work. The elements are taken from online video searches, still images from newspapers and online search engines and animated 3D models are made and animated . They are then cut out from their backgrounds using chroma key techniques, luma key techniques, rotoscoping and manually cutting them out using the digital manipulation software. These are then composited onto the painting, animated, scaled and colour corrected so they ‘sit’ in the works using standard compositing techniques, elements such as weather conditions are also added and animated.

Analysis of the work 'Wivenhoe' (2016)

Watermarked clip of ‘Wivenhoe’ 2016 Single Screen HD Video

The first work created was ‘Wivenhoe’ (2017) which was screened at ‘We Make Stuff, Holburne Bath’ (2017) and ‘Imagined Faraway Lands, Caraboo Bristol (2019) and uses the John Constable painting of Wivenhoe Park (1816). The work was painted for the landowner Major General Francis Slater-Rebow and shows his estate in Essex and was an important early commission for Constable. The estate now belongs to the University of Essex and has been repurposed into a modern university campus. Constable’s idyllic view contains some men fishing and Slater-Rebow’s niece riding a pony and trap. The differences and contradictions represented by what the image was and what this particular landscape now is intrigued me. I was particularly interested in the repurposing of large country houses and this drove the choices of the narrative elements for the tableau vivant. I populated the scene with many of the aspects that have been preoccupying me; joggers, sleeping migrants, a wind turbine, tourists, businessmen and women, maypole dancers amongst other narrative elements. The work was made with a free reign and it allowed me to combine the animation and compositing techniques and began the process of the contextual frame of interrupting the idyllic view.

Watermarked clip of ‘Reaper’ (2016) Single Screen HD Video

Analysis of the work 'Reaper' (2018)

Reaper is the second work created in the series. In the work ‘Reaper’ (2018) Benjamin William Leader’s painting of February Fill dyke is used. It was painted in 1888 and was relatively poorly received at the time, but has over time developed a following and has become a popular painting of the British idyll tradition. Kenneth Clark in ‘Landscape into Art’ makes a comparison between the work of Leader, held by the public a ‘perfect truth to nature’ he compares it to Monet, Sisley and Pisarro (working at the same time both in France and here in England) and sees a contrast between true and false naturalism. Clarke (1984) states:

even in reproduction we see there is none of that unity of atmosphere, the general envelope of light (to use Corot’s word) which is the essence of true naturalism. But then there is no unity of any kind, Nature has not been perceived as a whole, but described piece by piece. Leader still thinks of the world as made up of a number of things which have to be treated separately (170).

What Clark sees as a weakness - I propose as an interesting metaphor of how we perceive the landscape now - Nature has not been perceived as a whole but described piece by piece. What Clark describes is what in visual effects is termed a ‘composite’. It is this idea of ‘false naturalism’ that holds my attention as I would argue all composite works are false naturalism (even within high budget movie making, realistic composites are made to drive either narrative or aesthetic visions that are in themselves ideological acts).

'Reaper', although confused overall in its symbolism and use of metaphor began the important examination of the idea of compositing as both a technical and an ideological act. As Lev Manovic states:

Although digital compositing is usually used to create a seamless virtual space, this does not have to be its only goal. Borders between different worlds do not have to be erased; different spaces do not have to be matched in perspective scale, and lighting: individual layers can retain their separate identities rather than being merged into a single space; different worlds can clash semantically rather than form a single universe (2001: 18).

'Reaper' examines this false naturalism in various ways, especially the blunt replacement of a bird with a full size MQ-9 Reaper Drone deliberately drawing attention to its uncanny falseness. The drone is lit using the painting itself, and sits well in the painting (is realistically composited) but presents two aspects of the idea of viewing and being viewed and as Manovich states creates an interesting clash of semantic worlds.

Reflection and development

Both works were presented at the ‘We Make Stuff’ (2017) public symposium at the Holburne Museum in Bath. Having presented them, received feedback and reflected on both the method and the content I began to question certain aspects; both the symbols used, the theoretical frame and the method of creation. At this point I paused the practice elements and having recognised certain contextual weaknesses in relation to the literature I returned to Frayling’s research ‘into’ art. This pause in practice became a key moment and by exploring and expanding the reading in a much more considered and interdisciplinary way the practical research could then continue to develop and certain motifs and themes began to emerge.

Kenneth Clark’s ‘Landscape into Art’ (1984) identifies specific modes of presentation of landscape art around themes; fantasy, topography, symbol amongst others. Clarke identifies the symbolic mode, commonly used in mediaeval paintings which show within a single landscape painting multiple temporal narratives or multiple moments from individual narrative in a single work. This helped cement the aesthetic approach by situating the new video works firmly in an aesthetic space that has a fixed viewpoint of a landscape that contains multiple simultaneous temporal narratives within that space. This was most common in medieval paintings and can be seen here in The Thebaid, Fra Angelico, (c1420).

https://commons.wikimedia.org/wiki/File:Angelico,_tebaide.jpg

Having reviewed the narrative elements of Reaper and Wivenhoe and incorporating the reading on pastoral and landscape art I realised that there were certain ways I could both approach the method and the content. In their article ‘The Role of Landscape Art in Cultural and National Identity’ (2020) Wen and White comment “In the majority of (John) Constable’s most-recognized works there are a number of human figures carrying out everyday tasks; it could be argued that Constable’s art combines landscape and genre subject matter. But it is a ‘tamed’ countryside that is depicted” (p11). The use of figures performing tasks in various thematic acts to interrupt and dislocate the viewer of the ‘tamed’ landscape in the post pastoral mode was thus identified. These are:

Joggers, cyclists and walkers. These represent the urban ‘other’ within the space - the urban interloper using the space as a leisure playground. Critical creative examination of this can be seen in the work of Simon Roberts in his series We English(2007-08), In Ben Wheatley’s film ‘Sightseers’ (2012) and Hope Dickson Leach’s ‘The Levelling’ (2016) and Ross Raisin’s novel ‘God’s Own Country’ (2008).

The farmer/shepherd and sheep/livestock. Men and women at work in the countryside are nearly always used in Arcadian works and would form a key symbolic point and are essential to the idealised Arcadian view. However by exploiting the anti-pastoral mode they can be updated and exploited in new ways and sit uncomfortably alongside the other themes.

Folk rituals, eerie folk, paganism. This represents an interest in ‘folk’; folk traditions and folk horror and includes archetypes of the urban young’s projection and desire for a return to traditional country values. These include elements such as the Green Man, the may pole and maypole rituals, music festivals and other traditional festivals such as Beltane and solstice celebrations; what writers such as Robert McFarlane have described the growth of the ‘Eerie Pastoral’.

Immigrants, gypsies, peasants and homeless. These are a different form of ‘other’. The portrayal of homeless, vagrants, travellers and gypsies can often be seen in the works of Constable and others, but nearly always in nostalgic and non intrusive ways. By using these images it enables the exploration of themes around migration and became the core driver for the work ‘The English Garden’.

Analysis of the work 'Flood' (2020)

‘Flood’ 2020 Single Screen UHD Video

“Dedham Vale with the River Stour in Flood by John Constable 1814-17” forms the basis for the work Flood. The painting was chosen as I was looking for a representation of a river in flood to create a work loosely based on the 2014 Somerset floods. Even though in flood the Dedham Vale is still presented in ‘idyllic’ terms, no one seems to be adversely affected, the cattle graze, unperturbed in the foreground. I wanted this work to explore two aspects in relation to Gifford’s modes: that of the anti-pastoral; the people affected and the hard work that then goes into managing an active flood. More broadly I wanted to address floods to interrogate the post pastoral mode to encourage the viewer to consider the way climate change is affecting our perception or more importantly interrupting our view of such pastoral scenes.

Individual narrative elements that I had previously collected were utilised such as the story of Somerset’s ‘King Canute’ who attempted to build a moat around his newly built house and it is this house that sits in the middle distance, the earth movers working in vain to stop the flood. The workers in the foreground building the sandbag walls were played by myself and were shot using green screen.

The work functions in an interesting space between the post pastoral/anti pastoral modes as it encourages or confronts the viewer with the hardship and drudgery of the current climate situation. It also asks questions around land management and ownership with the homeowner now having to build his own moat and rampart (in the 21st century) to try and protect his property. The importance of the development in the practice here was the simplification, or focussing, of the use of the metaphor and symbols used and a concentration on the theme which allowed space for such complex themes to be explored in a much more considered way.

Analysis of the work 'The English Garden' (2020)

‘The English Garden’ 2020 Single Screen UHD Video

Jacob Philipp Hackert - Landscape with Motifs from the English Garden in Caserta (1797) forms the basis of the work ‘The English Garden’. Returning to Gladwell’s quote of works creating an “omnium gatherum of iconic English landscape features' and Clark’s ‘false naturalism’ this painting created in Italy, by a German of a garden started by an Englishman (Lord Hamilton) for the Italian royal family represents for me a perfect example of the composite landscape. It contains an imagined view of a re-imagined landscape in the English garden style with fantastical capricci features.

For the work the foreground lounging ladies and cattle have been replaced by a composite digital matte painting of the ‘Jungle’ camp at Calais. All of the tents featured have been taken from press images at the moment of the clearance of the camp in February 2016. It combines the idyllic view with the interrupted view and uses subtle animation of nearly every aspect from the trees swaying the wind to the fire burning in one of the tents. The combination of the original Hackert image with the specificity of the familiar yet unfamiliar view of the tents (viewers have sometimes mistaken it for a music festival) the new medium creates a sense of what Petho terms intermedial ‘in-betweenness’. She writes “Speaking of intermediality in Deleuzian terms is facilitated by the fact that intermediality has usually been described as a ‘dynamic relationship’ between media rather than a simple assemblage of media coming into contact. Theorists often describe it as a performative act, a process and a ‘displacement’, a ‘dislocation’, ‘transitory’ or ‘interim’ experience” (p474). This metaphorical and literal displacement is at the core of my current approach by this time. By concentrating on specific visual themes, in this case migration and geographical displacement and combining this with the intermedial remediation methodology it creates and supports the metaphorical space for the interim experience.

Conclusion

Post Pastoral Exhibition Still

The works culminated in the exhibition ‘Post Pastoral’ (2020) at Centrespace Gallery Bristol. Shown together the works showed both the development of the method and also the culmination of the theoretical/contextual post pastoral frame.

The method of using visual effects to remediate paintings can be effective. It is clear and obvious that artists will continue to find inspiration in art historical painted images, but the use of visual effects to do so is growing rapidly and worthy of further study and consideration. New and novel production methods allow audiences to engage with this and this method allows both audiences and makers to read and make new visual realities through the remediation techniques. By combining this method in an interdisciplinary manner the meaning and ‘baggage’ that comes with landscape painting around nation and identity formation, hegemony and false imagined idylls can be explored in bold and relevant ways. These then form part of a broader reexamination and recontextualisation of all aspects of the ‘pastoral’ from perspectives of decolonisation and ownership and a reappraisal of such flawed idylls.

Comments from viewers of the exhibition support this and points such as the works were 'making me think about what was consciously left out of the original images', while another commented 'It makes me really aware of the idea that even now we are clinging to this idea of Britain or what the landscape, countryside means . . . . there is something quite beautiful about them as well, even if the themes are quite sinister, pointing to the darker underbelly of society now as opposed to then, but how they are still joined.'

The continued use of intermedial method and theory to drive practice has real benefits. It is possible to use the literary theory of the pastoral to create and examine visual art works and keep addressing the concept of the pastoral in a considered way, while using the intermedial approach to explore sense feelings of displacement, dislocation and create challenging and exciting interim experiences. To develop the project, avenues of collaboration are now being sought with institutions both city and rural with landscape painting in their collections. The method can also be expanded outside of the pastoral contextual frame to either drive practice or be used as an analytical tool.

References

• Andrews, M., 2017. English Landscape: The Picturesque - Professor Malcolm Andrews. [online] Youtube. Available at: <https://www.youtube.com/watch?v=Wa8bw_ARgf0> [Accessed 10 November 2019].

• Bolter, J. and Grusin, R., 2003. Remediation. Cambridge, Mass: MIT Press.

• Chatel, L., Gould, C. and Mesplède, S., 2017. In: Art and the Environment in Britain, 1700-Today. [online] Available at: <https://artenvironuk17.sciencesconf.org/> [Accessed 7 July 2019].

• Clark, K., 1984. Landscape Into Art. New York [etc.]: Harper & Row, p.170.

• Frayling, C., 1993. Research in Art and Design. Royal College of Art Research Papers, 1(1).

• Gifford, T., 1999. Pastoral The New Critical Idiom. London: Routledge.

• Manovich, L., 2001. The language of New Media. Cambridge, Mass.; London: MIT

• Petho˝, Á., 2014. The garden of intermedial delights: cinematic ‘adaptations’ of Bosch, from modernism to the postmedia age. Screen, 55(4), p.473.

• Wen ,X., & White, P., 2020. The Role of Landscape Art in Cultural and National Identity: Chinese and European Comparisons. Sustainability School of Design, Shanghai Jiaotong University, Shanghai & Department of Geography, University of Sheffield (p11)

Artworks

• Allchurch, Emily. 2005-2018. Towers of Babel [online] Available at: https://www.emilyallchurch.com/gallery/babel-towers/ [Accessed 08 July 2020]

• Angelico, Fra c1420. The Thebaid. Jpeg image of Tempera on wood painting, https://commons.wikimedia.org/wiki/File:Angelico,_tebaide.jpg

• Constable, John. Dedham Vale with the River Stour in Flood from the Grounds of Old Hall, East Bergholt. Jpeg image of oil painting. https://commons.wikimedia.org/wiki/File:Dedham_Vale_with_the_River_Stour_in_Flood_from_the_Grounds_of_Old_Hall,_East_Bergholt_by_John_Constable.jpg

• Constable, John. 1816 Wivenhoe Park. Jpeg image of oil painting. National Gallery of Art, Washington DC. https://en.wikipedia.org/wiki/Wivenhoe_Park_(painting)

• Hackert, Joseph Philip. 1797. Landscape with Motifs from the English Garden in Caserta. Jpeg image of oil painting. https://www.lempertz.com/en/catalogues/lot/1153-2/2503-jacob-philipp-hackert.html

• Williams-Leader, Benjamin. 1881. February Fill Dyke. Jpeg image of oil painting. Birmingham City Art Gallery.https://artuk.org/discover/artworks/february-fill-dyke-33694

• Reihana, L., 2017. ‘In Pursuit of Venus-Infected’ 24ft Video projection (Available at: http://www.inpursuitofvenus.com/infected-1[Accessed 6 Jan,2020)]

Notes

[1] A Capriccio in painting combines a landscape with often idealised and fantastical architectural features to create a new and imagined fantastical landscape.

[2] In his 2003 work ‘Research in Art and Design’ Christopher Frayling identifies three distinct research methodologies in Art and Design which form a useful structure for approaching practice research

Dear Carnegie Hall: Utilising Digital Technologies to Unlock Multiple Perspectives of the Past

DOI: https://doi.org/10.33008/IJCMR.2020.26 | Issue 3 | April 2020

Ruth Farrar

Bath Spa University

Abstract

Accounts of history inherently vary. This single-piece exploration investigates how digital media technologies – specifically, an app – can be used to capture the complexities of understanding a history’s multiple perspectives and perceptions. Dear Carnegie Hall is an interactive storytelling app commissioned by Carnegie Hall in New York to commemorate the music venue’s 125th anniversary. This project was a collaboration between researcher and app producer Ruth Farrar and Barney Heywood and Lucy Telling from Stand + Stare: an interactive design company based in the UK.

In terms of process, Dear Carnegie Hall applies app, image recognition and augmented reality technologies in conjunction with archive material and messages from patrons, backstage staff and audience members. The project provides an original contribution to the emerging trend of museums and galleries mediating digital technologies in apps to create new modes of understanding history. Such apps typically provide a one-way channel of sharing history from the organisation’s app to attendee (see ‘British Museum Visitor Guide’ app (2016) and ‘Uffizi Gallery’, Florence (2017)). In contrast, ‘Dear Carnegie Hall’ explores how the affordances of an app can provide for a more democratic approach to narrating the stories of history. As well as the app curating stories of Carnegie Hall’s past, the user is also able to record an audio postcard (Farrar, 2015) of their personal story and experiences of Carnegie Hall, which in turn aimed to position the app as that which broadened the diversity of the venue. Dear Carnegie Hall thus demonstrates how using new digital technologies can encourage a sense of play with the seemingly fixed stories of history, which led to its users commenting on a deeper understanding of the organisation’s history.

Figure 1: A selection of postcards designed for the Dear Carnegie Hall app (2015). Image Credit: Barney Heywood (2015).

Background research

Dear Carnegie Hall (2015) is an interactive storytelling app commissioned by Carnegie Hall in New York to commemorate its 125th anniversary. I was invited to create this commission after Carnegie Hall’s Head of e-Strategy discovered my work in The New York Times and then came to experience my sound art installation From Austria to America (2013). My commissioned sound art piece creatively captured multiple oral histories of Austrians emigrants in America. As a practitioner, I am drawn to the creative challenge of capturing and exhibiting multiple histories in one artefact. Dear Carnegie Hall provided me with an opportunity to investigate documenting multiple perspectives on a larger scale. I collaborated with industry partners Lucy Telling and Barney Heywood from interactive design company Stand + Stare on this app project. Dear Carnegie Hall invites a participant to unlock 12 stories from its past. These stories move beyond a typical primary focus on stage performers to explore Carnegie Hall’s rich history from diverse perspectives and gain a deeper understanding of this iconic institution.

Research questions

Accounts of history inherently vary. This is further complicated when a practitioner is tasked with equitably representing multiple subjective histories in one artefact. One creative technique is re-telling a classic history from a different perspective ranging from Jean Rhys’ Wide Sargasso Sea (1966) novel decolonising the story of ‘the mad woman in the attic’ from Charlotte Brontë’s classic Jane Eyre (1847) to Hidden Figures (2016) film retelling NASA’s history from the perspective of a team of pioneering female African-American mathematicians. They played a vital role in astronaut John Glenn's launch into orbit yet were notably absent from its original history.

However, deconstructing a history to retell a new history is in itself ‘fixing’ the history again within the confines of a book or a film artefact as it returns to focusing on history from a singular angle or perspective. In Dear Carnegie Hall, I was interested in how new media technologies offer a practitioner new approaches in documenting multiple histories in one artefact. My primary research question was how can new media technologies – specifically, an app, image recognition and augmented reality technologies – be used to capture the complexities of understanding a history’s multiple perspectives and perceptions? Also, how can new media technologies be used to deepen an understanding of an organisation’s history? From a social constructivist perspective, in what ways can researchers shape new media technologies to help archival material come to life for new audiences? [1]

Methods

To understand how the app operates and shapes image recognition technology and augmented reality, please view the app demonstration video below.

Figure 2: A video demonstration of the app’s features

In two field trips to Carnegie Hall, we examined its archive’s research material, gained feedback on our proposed new media technologies and interviewed 12 people with different connections and stories of Carnegie Hall such as interviewing Carnegie Hall patron Jim Stewart at the New York Times building. Our method for interviewing stories included inviting participants to bring an object connected to a memory of Carnegie Hall. One interviewee Bernard Kaplan brought an old ticket stub from a performance he experienced at Carnegie Hall in the 1970s. The physical object proved an effective method in helping unlock his memories of Carnegie Hall’s history. Each interview was edited to found archival material connected to Carnegie Hall shaped as a postcard from the past. [2]

In post-production, I reshaped a feature from a previous project I originally created for a website installation piece, Audio Postcards: Brooklyn (2013), into the app. This key method enables the app user to experience 12 postcard stories and then create their own postcard by selecting a postcard cover from archival material and then in-app audio recording a postcard message sharing their memories of Carnegie Hall, which can then be emailed to a recipient. This method was used to openly encourage further diverse perspectives of Carnegie Hall’s history.

Research findings

My research as practice approach revealed three key insights. Findings were informed by the following data evaluations: my practitioner’s journal; reflective discussions with industry partner Stand + Stare; feedback from Carnegie Hall staff in USA; quantitative and qualitative feedback from the app’s user testing group of 15 UK-based participants; theming feedback about the app from press coverage and social media and feedback forms from a knowledge exchange interactive keynote session I led in Japan.

1) This app is original and different to current apps used by museums, galleries and cultural organisations in that it does not provide a one-way ‘fixed’ history of an organisation but instead openly invites, enables and validates multiple diverse historical perspectives.

The project provides an original contribution to the emerging trend of cultural organisations mediating digital technologies in apps to create new modes of understanding history. Current apps on the market typically provide a one-way channel of sharing history from the organisation’s app to attendee (see The British Museum Visitor Guide app (2016) and Uffizi Gallery, Florence (2017)). In contrast, Dear Carnegie Hall explores how the affordances of an app can provide for a more democratic approach to narrating the stories of history. The user experience of the app encourages a user to explore twelve different stories about Carnegie Hall’s history. These include Carnegie Hall’s Director of Archives Gino Francesconi’s retelling of Andrew Carnegie’s story, Artist Liaison Debby King sharing backstage experiences with stars such as Frank Sinatra and Jenna-Marie Sparacio, a youth orchestra student, shared how the organisation shaped her: all different yet equally validated experts in helping a user understand Carnegie Hall’s rich history.

This invitation to be an expert is further extended when users are invited to share their memories of Carnegie Hall thus opening out the organisation’s history beyond internal representations. This sense of play elicited from making a digital postcard helped created a welcoming invitation and validation that their story matters too in understanding Carnegie Hall’s history. When sending digital postcards, users had the option to grant permission for their story to be saved by Carnegie Hall for archival purposes. The app became a significant method of gathering stories about the organisation, which may ordinarily have been missed.

2) This unique app makes visible the remediation process of documenting an organisation’s history using new media technologies. This invited participation process enhances engagement and deepens a user’s active understanding of an organisation’s history.

Documenting history inherently involves a shaping process. A deconstructionist perspective of history acknowledges the ‘unavoidable, impositionalist role of the historian’ in creating meaning from texts (Phillips, 2012). I am also aware of my role as researcher and practitioner in making meaning from and remediating archival material, editing audio interviews and shaping digital technologies for this app.

For Bolter and Grusin, remediation rests upon the ‘double logic’ of different degrees of immediacy and hypermediacy (1999). Placing and scanning the physical postcard in the app’s viewfinder unlocks a history and provides the user with ‘window through’ of immediacy to understand Carnegie Hall’s history. However, this action also makes the user aware of the ‘window at’ visible medium of the app as they experience the image recognition technological process used to remediate archival material bringing the postcard to life via the app. Hypermediacy is also at play because the app’s multimedia elements form an invitation for the user to create their own postcard. This process intentionally fragments the self-contained unified perspective of each postcard’s history in the app to expand upon and add more perspectives of the organisation’s history.

The unconcealed remediation of the app invites participation. I discovered this increases a user’s engagement. Through play, users actively learn about Carnegie Hall’s history such as giving them the option to choosing which postcard to listen to next or the option to make their own postcard increased users’ deeper understanding of Carnegie Hall’s history. Illustrative qualitative data statements include:

‘it brought the history to life’

‘the creative little details such as the archivist’s letter helped me understand the bigger historical context’

'Really engaging! Loved how I can also add my own story to Carnegie Hall’s history. Making my postcard helped me rediscover fond memories of Carnegie Hall’ (2015).

This app is a creative statement on the role we play in shaping, remediating and retelling history. Users decide which of the 12 stories to experience and how they remediate archival material and record their memories of Carnegie Hall in their own digital postcards. While a film may conceal its remediation, this app’s remediation is visible and promoted as it acknowledges history is not ‘fixed’ but open to being shaped and remediated.

3) ‘Dear Carnegie Hall’ provides a new technique for documenting multiple perspectives of the past and its significance has since influenced industry and academia

I and my industry partner Stand + Stare retained the intellectual property rights for the app’s design and development. When Stand + Stare was tasked with the challenge of documenting multiple historical perspectives, they applied our app’s infrastructure to a new industry setting for further impact. The result was a funded commission: Stand + Stare’s app Illuminators of Aberdeen (2016) unlocking the stories of six innovators from Aberdeen’s past for Spectra: Aberdeen’s Festival of Light.

I also shared our original techniques in a knowledge exchange invited keynote at The International Academic Forum (IAFOR) Conference in Kobe, Japan. My interactive keynote session influenced an attendee: Professor Norman Ralph Isla from the Philippines and his teaching practice. When teaching students about a complex topic, which required students to understand multiple perspectives of the past, he used insights and a reshaped shorter version of our app’s methods using image recognition and smartphone technologies in the classroom. He found this method measurably increased students’ engagement and deepened their understanding of a historical topic.

Figure 3: Sharing our app’s techniques in Japan influenced teaching practice and enhanced student engagement and learning in the Philippines. Image Credit: Professor Norman Ralph Isla (2018).

Our original aims with Dear Carnegie Hall were to help people beyond the organisation to understand its history. However, a further application of this app’s techniques emerged when a globally renowned tech company became aware of the app. Due to a non-disclosure agreement, I am unable to go into the finer details. However, they can see how this original technique of sharing multiple perspectives of the past can be applied to educate new employees about an organisation’s history in industry.

Ultimately, this process has confirmed that when a researcher shapes new media technologies in a unique way, it leads to increased participant engagement and a deeper multi-layered understanding of an organisation’s rich layered history. The techniques employed in this app effectively provide a solution for the challenging yet important task of documenting multiple diverse perspectives of the past.

Acknowledgements

This project was commissioned by Carnegie Hall. Many thanks to Carnegie Hall for their generous support including Seamus O’Reilly, Debby King and Carnegie Hall’s Director of Archives and the Rose Museum: Gino Francesconi. It was also a pleasure collaborating with Lucy Telling and Barney Heywood from Stand + Stare on this project.

References

• Bolter, J, and Grusin, R. (1999) Remediation. Understanding New Media. Cambridge Massachusetts: MIT Press.

• Brontë, C. (2019). Jane Eyre. London: Penguin Random House.

• Farrar, R. (2013). From Austria to America. [Installation Exhibition]. The Austrian Cultural Forum, New York. 19 – 21 September 2013.

• Farrar, R. (2013). Audio Postcards Brooklyn. [Website Installation]. Available at http://audiopostcardsbrooklyn.com (Accessed 16 November 2017).

• Farrar, R., Telling, L. and Heywood, B. (2015) Dear Carnegie Hall. [App].

• Hidden Figures. (2016). Directed by Theodore Melfi. [Feature film]. USA: 20th Century Studios.

• Rhys, J. (2016). Wide Sargasso Sea. London: Penguin Classics.

• Phillips, M.G. (2006). Deconstructing sport history: a postmodern analysis. Albany: State University of New York Press.

• Pinch, T.J. and Bijker, W.E. (1984). The Social Construction of Facts and Artefacts: Or How the Sociology of Science and the Sociology of Technology Might Benefit Each Other. Social Studies of Science, 14(3), pp. 399-441.

• Stand + Stare. (2016). Illuminators of Aberdeen. [App]. Available at https://apps.apple.com/gb/app/illuminators-of-aberdeen/id1077822220 (Accessed 5 March 2017).

• Vusiem Tour Guides Ltd. (2016). British Museum Visitor’s Guide. [App]. Available at https://apps.apple.com/gb/app/british-museum-guide/id551275212 (Accessed 10 September 2017).

• PharoSuite. (2017). Uffizi Gallery. [App]. Available at https://www.uffizigallery.app/# (Accessed 16th December 2017).

Notes

[1] Pinch and Bijker’s social constructivist SCOT critical framework eschews a scientific objective ‘real truth’ of a technology (1984). Instead, they argue a technology’s development is shaped by its users. This approach appealed to me as a user of image recognition and augmented reality technologies shaping its developments and pushing forwards its capabilities in the context of app development for cultural organisations. We developed and shaped these new digital technologies to avoid a singular history of Carnegie Hall in order to generate and celebrate multiple perspectives of the past.

[2] The app can be used without a physical postcard. However, as previously mentioned, interviewees found the method of bringing a physical object helpful in unlocking their memories of Carnegie Hall. Moving forwards with this method, the printed postcards act as a tangible physical device connecting past documenting memories methods (a physical postcard) with current technologies (a digital postcard) as a useful way to spark a user’s own memories of Carnegie Hall. The printed postcards also served as a creative technique to generate awareness about the app. The postcards were displayed in Carnegie Hall’s foyer and shared via an international postal marketing campaign providing an accessible way for all age demographics regardless of technical ability to become aware of the app and to reflect on Carnegie Hall’s rich history from multiple diverse perspectives.

Language in a meme economy

DOI: https://doi.org/10.33008/IJCMR.202016 | Issue 5 | October 2020

Ami Clarke

Abstract

Artist Ami Clarke traces some of the complexities, multi-temporalities and scales, that coalesce around some new, and some very old power relations, that come of, and are revealed by, the technologies associated with the interdependent ecologies of social media, finance, and the environment.

Her work traces hyper-speculation that came of the semiotic boom, where the loss of the referent in both language and the economy is shared across the trending behaviour of neoliberal/free market dynamics in finance, as well as emerging media ecologies.  Here, inconsistencies in claims of ‘fake news’ amidst rights to ‘freedom of expression’ converge in the shortcomings of colonial practices of extraction, and new hyper-networked digital colonialisms, as the futures markets meets behavioural futures across the interdependencies of a reputation economy that comes of online news and social media, and the forms of finance driven by this.

Pre-empting many of the conditions brought into sharp focus by the pandemic, she touches upon how the insurance industry reveals the catastrophic flaw of investing in the neoliberal myth of the market, as ‘unprecedented’ events become increasingly every-day. A state of contingency, that no longer promises an opportunity to ‘write the future’, but instead, is felt through the mechanisms of disaster capitalism as churning markets across both the financial sector, and the mediasphere, as a means by which to game not only who has the authority to speak, but democracy itself.

Introduction

Cries of ‘fake news’ characterise these very particular times, but fake news is anything but new, with words and language having long been weaponised. Whilst researching the Leveson Enquiry in 2011/12, it became apparent to what degree the British press, politicians, and police, were all in each others pockets, very publicly pouring doubt on anything approximating a free press. What also became clear was that the British press was 80% owned by right wing billionaires with a recent analysis in 2019 by Open Democracy showing that UK media have a concentrated ownership structure ‘with six billionaires owning and/or having a majority of voting shares in most of the national newspapers’. In light of these already highly compromised relations, I began an in-depth enquiry regarding the changes in news production that came of the drift online by traditional journalism, and what that meant for the news, as well as the new forms of reportage opened up by this.

Low Animal Spirits was a work made in collaboration with Richard Cochrane in 2014, that took its cue from the then oft-mentioned loss of the referent in both language and the economy, speculated about wildly after the economic collapse of 2007/08. It deployed a high-frequency trading (HFT) algorithm written by Cochrane who was once a Vice President of Goldman Sachs, that ‘deals’ in words sourced from global news feeds for virtual ‘profit’, whilst speculating on their usage. The analysis produces new phenomena in the form of speculative headlines tweeted from the twitterbot @LowAnimalSpirit. The projected visualisation shows what is about to trend (not what is already trending), with a glimpse into the HFT algorithms buying and selling activity, as it accesses 994 English Language global news feeds ‘live’, and acts upon the data as if it were trading in the global market place, analysing words in terms of the potential for a virtual ‘profit’ to be made.

Image 1

And we chose to apply HFT to the news because it seemed to epitomize the speculative, short return, and unfettered free market ideology of neoliberalism, in an activity closer to ‘sifting’ than market making in and of itself.

And, because, quite unlike poetry - news matters precisely because of its indexical link to the truth.

A social production interwoven with a history of technological innovation, from the era of the printing press to present day hybrid strategies that include both print and digital platforms, that has an underlying requirement for a need for language to be tethered to an instance based on fact, and consequently a focus on how truth may be defined.

As Wikimedia, the new media theorist Mercedes Bunz, and Hannah Arendt all attest, access to knowledge is a basic fundamental right with reliable information known to be a key driver for social and economic development, “with the evaluation of truth and information shifting from the beliefs of the middle ages, to reason, during the Enlightenment, precisely because information and knowledge become an essential means of production, underpinning economics itself’” (Bunz, 2013). A taxonomy underwritten by the extractive relations of colonialism and slavery, that established a very specific order, in which ‘the construction of Europe was based upon the concept of racial superiority”, as a ‘discrete, racially pure entity, solely responsible for modernity” through the very conceptualisation of blackness (Jameson, 1983). The socio-political and economic technologies of capital production would later go on to inscribe the binary relations necessary to underwrite capitalism further, with the unpaid labour of 50% of the population.

Shoshana Zuboff’s fieldwork, today, shows how the new knowledge territories emerging alongside the capacity to analyse processes and behaviours, also resulted in political conflict over the distribution of knowledge, as: “Surveillance capitalists … declared their right to know, to decide who knows, and to decide who decides. Meanwhile, the constantly changing landscape of digitalization has meant that “Algorithms are [also] frequently updating the fact, with the outcome that it is being altered endlessly” suggesting that, whilst “the digital fact has never been more accurate, it also has never been less durable” (Bunz, 2013).

“it becomes obvious that the digital fact is relating to truth in a new and different way than the fact of the industrial age, for truth and facts aren't the same but share a rather complex relationship newly adjusted by technology” (Bunz, 2013).

The means by which this comes about, and ‘who’ gets to contribute via ‘the constantly updateable fact’, whether via citizen journalism, twitter, facebook, medium, buzzfeed, and so on, begins to challenge some of the ‘absolutes’ that come of empire, through a newly vocalized multitude of voices, as “the shape of the truth seems to be related to the technical means we use for approaching it” (Bunz, 2013).⁠ Finance afforded a glimpse of a highly volatile, and paradoxical model of mass-behaviour, with the figure of homo economicus already a deeply in-debted subject “as (increasingly) the lifeworld became a system for the notation of market trend data”⁠ (Philipp Ekardt , 2014) within the highly quantified protocols of platform capitalism.

Image 2

Finance had long been the site of cognitive bias research with the title Low Animal Spirits pointing to the paradox that Keynes discovered during the Great Depression, in which an inherently herd-like mentality to the market was simultaneously reliant on the concept of the heroic individual spontaneously acting of their own ‘free will’. The term 'animal spirits' was drawn from the Latin 'spiritus animals' to describe an elusive energy, or spirit, at the time thought of as a fluid that drives human beings, associated with Hume’s concept of spontaneous motivation. Gambling and games of chance, with all the romantic heroism of the autonomous subject, pitted against mathematical determinism, drove the quest for ever better odds, as the C18th study of celestial mechanics developed complex new mathematical tools, for deriving probable outcomes. Across several different branches of science, visual perception was overtaken by what Maxwell termed as ‘the true logic of this world ... the Calculus of Probabilities, which takes account of the magnitude of the probability’. It is significant, he writes, that: ‘this branch of math ... is generally thought to favour gambling, dicing, and wagering, and therefore highly immoral’ (Bender and Marrinan 2010, 178 footnote 79). The arena in which these ideas arose, was of course heralded for the very concept of ‘free will’ coming into being, and as Hayles notes; an evolving subjectivity that emerges through market relations.

Images 3, 4 & 5

Breaking News Flash Crash

The financial sector was also of interest because they really weren’t shy at forging ahead with automation in ways that displaced even the most trusted wisdom of human operators.

Breaking News Flash Crash depicted the moment that the Associated Press's twitter account was hacked with the tweet: Breaking: Two Explosions in the White House and Barak Obama is injured - with the press reporting "Hackers caused a mini stock market ‘flash crash’ this afternoon after sending a bogus tweet from a hacked Associated Press (AP) Twitter account". The market recovered particularly quickly in this instance, as there were no other corroborating stories, and the Associated Press were quick to shut down their twitter account. The entire event lasted less than 5 minutes and it’s worth noting that what happened was not due to a broken part, a mis-behaving algo, or a computer running crazily out of control, but precisely because the algorithms operated as designed to do, at speeds far beyond human cognition.

In a 2015 analysis of the flash crash event Tero Karppi and Kate Crawford draw attention to the Dataminr software that mines Twitter’s ‘firehose” and delivers what is deemed relevant into the hands of traders. “Dataminr combs through 340 million daily tweets on Twitter and its algorithms quickly seize on abnormal and actionable signals that can be analyzed and confirmed as a relevant event for a client. This could be anything from an assassination or general instability in certain countries to government sanctions, natural disasters or on-the-ground chatter about products or trends.” Citing journalist Michelle Price in the Financial News, they write that this sophisticated scoring of the relationships between words in play, can uncover grades of expressed ‘emotions’ and produce more than just a sentiment analysis of Twitter data: Here, value is accrued through an opaque, but meaningful process of assessment, in that the analysis in no doubt fuels decisions made by financial operators happening at speeds of data processing far beyond human capacity. A phase transition in cultural research, social scientists now analyze patterns in the massive datasets used to study emotional sentiments on Twitter, to deconstruct narrative tropes in the media to allegedly identify anger, fear, disgust and sadness. ‘Emotion detection’ has grown from a research project to a $20bn industry.⁠

Karpi and Crawford suggest that: digital innovations generally, and software code specifically, are codes also in the sense of being able to shape human conduct. “Thus, computer code is seen as performative, in that it creates specific types of social (but not necessarily human) beings (Introna, 2011; Mackenzie and Vurdubakis, 2011) or as governmental (Thrift and French, 2002), because it shapes human conduct. (Karppi and Crawford 2015, 49).” They argue that: “Twitter and social media are becoming more powerful forces, not just because they connect people or generate new modes of participation, but because they are connecting human communicative spaces to automated computational spaces in ways that are affectively contagious and highly volatile”.

The Eye That Remains Of The Me That Was I

Error-Correction: an introduction to future diagrams developed from a fascination with diagrams as a capacity to draw together complex entangled threads that could reveal something of the processes, conditions, and relations of power flowing through today’s human/technological assemblages. A writing that emerges as a script reflecting on the influence of calculus, in which each articulation is just one of many takes, constantly re-edited, that references and includes openly appropriated texts, contemporary commentary, news items, anecdotal evidence; culminating in an interrelated convergence of many interwoven threads, whereby the voice (through language), is constituted between someone else's thoughts, and the page. The work attempts to acknowledge the multitude of influences contributing to how thought might be arrived at, with an emphasis on any subject to speak of (as questions of authorship, also arise) emerging in synthesis with their environment.

I was interested in the complex ways that the body receives and processes multiple sense data, in everyday human technological assemblages, and took a cue from the physician and physicist Herman von Helmholtz’s research into mathematics of the eye, that led to probability theory, who suggested: “human perceptions, so prone to error, are at best, an approximation, an estimation even, that 'operate(s) within the protocols of instruments” His premise was that human eyes have: “a hard-wired, involuntary drive to minimize perceptual errors - and discovered error-correction in the very nerve endings of our bodies” (Bender and Marrinan, 2010).

Here, error⁠ is no mistake - but the driver of probable outcomes.

I was also interested in the way that the body was in question in this assemblage, acting at several simultaneous scales and temporalities, whilst the script denied any primacy of the voice, or identifiable subject, which is all the while being constituted through other people's words.

Error-Correction met Low Animal Spirits as a new assemblage in a serendipitous and entirely unexpected moment, and the scripting of language and code conjoined in a suspect ‘liveness’ of performance, that included the eye that remains of the me that was I - myself as ‘automated reader’.

Image 6

A nonlinear text, that doesn’t sit adjacent to what came before, as proximity no longer presupposes contiguity, and causality is no longer possible in the ordinary sense of the word. Reliant on syncopation and semantic rhythm to drive the score, the axiom, or recipe (Cramer, 2012) that dictates the form, is betrayed by a pornography of language (Rule & Levine, 2012), as earworms seep, until they crystallize in thought. Any subject to speak of emerges from this assemblage via an untrustworthy body with faulty equipment, produced in synthesis with it's environment, that feels through prosthesis, with a body that matters, without mattering.

Error-Correction takes it’s title from the theory of least squares by Adrien-Marie Legendre and Carl Friedrich Gauss (1805), and its subsequent application in celestial mechanics ‘formulating true statements in advance of experience’ in the work of John Couch Adams and Urbain Le Verrier. The mathematician Pierre-Simon Laplace at the École Polytechnique, championed the possibility ‘to determine the probability of causes lying behind events by working backwards from observations’ (Bender and Marrinan, 2010, 160) in his 1812 Théorie analytique des probabilities. The script speaks of the development from perspectival drawing, to projected geometry, and further, beyond the human sensorium via mathematics to calculus, leading to probability theory and being able to speculate the future.

It suggests that just as Euclidean geometry was surpassed by calculus, perhaps within the current diagram of power, another strategy might be necessary, oscillating somewhere between remaining visible, whilst also hidden, within the potential of becoming multiple, as it became clear that the protocols of platform capitalism relied entirely on a ‘freedom’ of expression drawn out by highly addictive and sophisticated scripting mechanisms, as you uploaded exactly what was on your mind, 24/7. Over the past few years these ideas have become less conspiracy theory, more mainstream media, via the revelations regarding Cambridge Analytica, amid a more widespread acknowledgement of, if it’s ‘free’, you are the product.

As ‘freedom of expression’ gets caught up in protocols that grant access to platforms that rely inherently on ‘you’ as the product, a currency to data emerges, that buys participation through the protocols of platform capitalism. An outrageous paradox, in which the important drive to become ’visible’ in terms of political representation, becomes distorted, when data, extracted willingly, or otherwise, leads to wrongful or reductive analysis and categorisation, adding further to unprecedented levels of surveillance and contributing to often systemic practices of discrimination. A phenomena not without considerable historical precedent, with Adolphe Quetelet (among others) applying statistical analysis to social data, producing an estimation of the ‘average man’ in his writing in ‘Sur l’homme’ in 1835, rendering statistical quantification as a form of power, through taxonomy and scientific categorisation.

The critical theorist Simone Browne author of Dark Matter: On The Surveillance Of Blackness, notes that in simple terms, ‘biometrics is a technology of measuring the living body’, and that most surveillance, as David Lyon suggests, is: “practiced with a view to enhancing efficiency, productivity, participation, welfare, health or safety,” leaving social control “seldom a motivation for installing surveillance systems even though that may be an unintended or secondary consequence of their deployment.” Browne points out that “Fiske argues that although Michel Foucault and George Orwell both conceptualized surveillance as integral to modernity, surveillance “has been racialized in a manner that they did not foresee: today’s seeing eye is white.”“

“How to act in a techno baroque condition” critical theorist Paul B Preciado asks, the likes of which today are characterised as ‘modulative’ (Deleuze, 1992) that “work through the manipulation of the flows which move bodies, and the thresholds across which they must cross” (Coley, Lockwood, 2012) as data becomes ‘a strategic asset and a behavioural surplus, underwriting in turn, a monetary surplus for the likes of Google, Microsoft, Amazon with a colonising ruthlessness’ (Zuboff 2019). Questions regarding the currency of data emerge with platform protocols, just as data as a currency might still unfold. Much like the wages for housework movement noted, historically, it really doesn’t matter if you want to think of human interactions and emotional responses as quantifiable, or not, it’s happening anyway.

Image 7

Image 8

Repetition: A Radical Unoriginality

Driven by an increasing urgency to think about what it might mean to ‘act’, within increasingly pre-emptive outcomes, with the media-sphere fast on its way to becoming as volatile as the markets, Ami Clarke: Author of the Blank Swan, was a conceptual work as publication, that emerged via the French Lebanese financier and philosopher Elie Ayache’s book ‘The Blank Swan’. A book on derivatives, where he suggests that ‘writing’ has an equivalence to ‘pricing’ due to the contractual nature of the derivative that ties the past to the future through writing. He asks if there were a technology that might be available “to get inside the very process of history and do something more active than to watch passively as history unfolds: altogether different from the conceptual activity consisting in predicting and outguessing history.”

He draws on Borges story of ‘Pierre Menard; Author of the Quixote’, a fictional writer and critic who spends his time writing chapters of the C17th Don Quixote, several centuries on from when the text was published. Interpretations of this have tended to focus on how ‘reading’ brings about difference, through a Barthes like emphasis on the true locus of ‘writing’ as reading. Conversely, Ayache’s focus interpellates Borges’ fiction with the apparatus of the derivative contract that implicitly relies on writing, bringing about a different emphasis on the ‘writing’ – of a previously existing text – as a truly contingent act.

Taking him up on his challenge, with each word that I wrote of ‘The Blank Swan: Chapter 4, Writing and the Market’, one dismal winter break with a snivelling wretched cold: there was simply nothing that could guarantee that any given word would necessarily follow the next.

The artist Elaine Sturtevant back-dates her artists book: Sturtevant: Author of the Quixote, published in 2009, via a letter written to Borges in the introduction, to 1970, around the time of her early practice of making works of other artists works. As Patricia Lee notes, thereby “pushing the codification of artists to specific signifiers” in relation to the structures and systems of art. Reducing the artists’ work to a sign; a brand: an easy meme producer, percolating myths of genius, and so on, that could be seen to have more to do with the market, than whatever other values might be claimed for art at any given time in history.

Image 9

Sturtevant’s emphasis on “the brutal truth of the work is that it is not a copy” is shared in Ayache’s thinking when she claims “the dynamics of the work is that it throws out representation” (P. Lee 2016). Ayache writes “Only through the writing/trading performance and not through the realization of a theoretical stochastic process, that is framed in representational thought, can the writer or trader of contingent claims exceed the saturated context and move to the next – i.e. he can trade.” What this brings forth in Ayache’s terms, is the trading room and a performative capacity that is singular and non-reproducible.

A capacity to write the future, seemingly shared across the blockchain in so far as a technology that exceeds probability through the time-stamp. Driven by the same writing technology as the contract in the derivatives market, Benjamin Bratton notes, blockchain amounts to little more than insurance. Developed notably from a libertarian perspective to facilitate trade online without the necessity of a bank as an external source of trust, blockchain also brought forth the most volatile of markets, with Goldman Sachs in on the action from early on, and the historical trader pitting themselves against the mathematical determination of the markets with all the heroics of high animal spirits, soon to follow. As Hayles notes, similiarly “Ayache’s vision of the market’s ontological power is a neoliberal fantasy run wild, fuelled by Quentin Meillassoux’s (2010) philosophical argument for the absolute nature of contingency and applied by Ayache to finance capital” (Hayles, 2017, 148)

Florian Cramer picks up something of Sturtevant’s critique in his essay Crapularity Aesthetics, 2019, describing how “the signature dichotomy of contemporary art is kept in place, by a series of measures that keep it simultaneously abundant and artificially scarce: (a) through the economics of the single-copy artwork in gallery art (b) through the illusion of autographs in Zombie Formalism and its meta-joking, digital institutional critique derivatives, or (c) through the limitation of the number of mutations and the use of digital crypto signatures and owner authentication stamps on memetic blockchains, such as in the proverbial Rare Pepes that, according to their creator Altpeter, demonstrate the ability to enforce digital scarcity for the first time”.

Blockchain, is, after all, only a tool, and just as a pencil is only useful in so far as what you choose to write with it, so is the potential of decentralised ways of working. The most urgent and compelling aspect of the equation, for me, then, is who gets to write the future in this new calculus.

The Marketplace Of Ideas

Events in recent years have heralded in several seemingly unimaginable futures, unless of course, as Whitney Phillips points out in “This Is Why We Can’t Have Nice Things”, you were already in a position of being aware of the discriminatory nature of western culture, often articulated through casual ‘banter’, as its wont to be called, happening on a daily basis.

In 2013, the Amsterdam based design collective Metahaven asked:

Can Jokes bring down governments?

whilst in 2017 we might have asked:

Can jokes bring down democracy?

A seam of cruel jouissance that comes of hunting in packs was given expression alongside the emphasis on the individual as consumer par excellence, drenched in narcissism and resentment through the mechanisms of social media, the dynamics of a meme economy, and a trickle down theory of economics, that does anything but. High on animal spirits it seeped through the retreat from empiricism to the post-truth political landscape of truthiness – that describes “the belief in what you feel to be true rather than what the facts will support”.

New (old) strategies of ‘disinformation’ also took hold, an old tactic, as mentioned, but one now built on unprecedented access to data, alongside psychological profiling made possible by platform capitalism’s inherent business model, borne of data analysis, psychological profiling, and targetted advertising. Chris Wyllie, the Cambridge Analytica whistleblower, points to how these strategies differ from general election campaigning, saying: “…Obama didn’t win because of disinformation. The fundamental difference between standard political targeting and information operations is this idea of informational dominance.”

This quote is particularly interesting if considered alongside the observation made during the presidential campaign that Trump acted much like a hungry algorithm, deliberately (in so far as intention goes with him) searching out weak points to attack. Rupture appears purely for the sake of it, saturating the churning infosphere and thus enabling an affective capture in a highly competitive economy of attention. The analytics firm Mediaquant estimated that between October 2015 and November 2016, Trump received $5.6 Billion dollars in “free” earned media from this strategy, three times his nearest rival: “The more outrageous his words, the more coverage he received’.

Rupture, or transgression, perhaps, is of particular interest as a tactic that resonates throughout the disruption of language; of logos, and the undermining of ‘meaning’ through various avant-garde art movements invested in it’s radical political potential, that can be traced through the non-sense of DADA to the no sense of the cut-up, where nuclear dust radiates through mimetic media, to more recently a psyops driven ‘un-sense’ (Srnicek, 2018) - whereby, I would extrapolate, the complete lack of meaning, evident in the first place, is actually quite difficult to subvert. There’s little point in making ‘non-sense’ of Donald Trump - he’s doing very well on his own.

Perhaps this points to the fact that previous avant-garde strategies that deployed rupture and transgression as a tactic, and not without considerable animal spirits, work only in relation to more entrenched societal behaviours of the time, historically, that meant that the tactic of ‘shock and awe’ wasn’t an everyday occurrence, prior to the ontological collapse of semiotic capture. Perhaps more pertinent to the post-causal, ‘post-truth’ dimension of politics, would be that the persuasive rhetoric of logos is distorted through the outrageous claims of advertising - in itself profoundly productive through psycho-analytical readings of the populace. The practice of publishing entirely untruthful claims, tailored to target very specific vulnerabilities, does the work of seeding the lie, regardless of the later retraction. Much like the press acts as Trumps marketing team, equally out-moded tactics of over-identification also serve only to spread the contagion that much more effectively, within a furiously fast economy of attention whilst normalising and relativising both hate speech and aesthetics.

The liberated first person narrative, meanwhile, is co-opted into complex behaviours of addictive upload, reliant on the construction of ‘you’ as the product, whilst cognitive, non-cognitive, conscious, and unconscious, bots, recogs, and other automated voices, driven by machine learning and Artificial Intelligence - contribute to a cacophony of ‘un-thought’ (Hayles, 2019). All, or any of these, describe Literat and van den Berg in their paper “Buy memes low, sell memes high” (where the community on Reddit’s MemeEconomy centres around the appropriation of stock market jargon to discuss memes as investment-worthy commodities), might then be a means to contribute to the collective construction of value in the digital age, rife with exclusionary dynamics, whilst boundary policing, identity constructing,⁠⁠ and accumulating cultural capital all the while, that far surpasses the logic of creating error, glitches, gaps, and rupture, purely for the sake of it.

Civil

In an attempt to counter the rise of ‘fake’ news, the founders of Civil, a site for journalism built on Ethereum, as a “marketplace for great journalism’, operates as a peer-review economy with the CVL token used in transactions as a means to “economically incentivize and promote good behaviour that helps the platform grow and thrive, whilst keeping bad actors off it”. Most importantly it recognizes the damage to journalism by the lack of an adequate business model as advertising revenue drifted online. They write that they’re ‘committed to introducing a new funding model …to focus on journalism, not satisfying clicks-over-quality mandates from third parties like advertisers and publishers’. If nothing else, a micro-payment system might evolve. More tellingly, the multiple modes of verification, reputation building and so on, implicit in Civil’s manifesto, attempt to address a waning grasp on reason and truth, core values of the enlightenment, closely aligned with individual liberty, and contributing factors to what has been reductively called the ‘post-truth’ landscape, but not all. As a system that attempts to deal with the drift in journalism from the one to the many, to the many to the many, which has an opt-in at point of entry, CIVIL risks being a silo of truth, amongst a sea of churning indifference.

And herein lies the problem.

The magic sauce that seeps through online civic life as jouissance ripples through the networks, means that many people don't much care for whether it's true or not, whilst investing in a freedom of expression born of victimhood, that’s curiously unaware of it’s own privilege. Conservative newspapers have been particularly quick to decry the post-truth landscape, whilst investing in a very particular sort of truth, notably, that props up their own agenda. An insidious argument that utilises analytical philosophy with claims of what amounts to the ‘truth’, as a means of gaming who has the authority to speak, when in actuality the “hate speech of the few silences the voices and threatens the lives of the marginalized many.”⁠ Here the inconsistencies in claims of ‘fake news’ amidst rights to ‘freedom of expression’ converge in the shortcomings of older colonial practices of extraction, and new hyper-networked neo-colonialisms. 

Image 10

Magic Sauce Meets Secret Sauce

The Underlying attempted to grasp, fleetingly, something of these complexities as they converge in the upcoming environmental challenges ahead, that come of, and are revealed by the interdependencies between the mutual ecologies of the reputation economy, that comes of online news and social media, and the forms of finance driven by this.

The contractual condition of the derivative, and insurance, central to risk management, was key to thinking about environmental concerns, as a means to consider how models of probability reveal the catastrophic fault lines of capitalism in relation to the environment, as ‘unprecedented’ events become increasingly every-day, and a state of contingency becomes a modus operandi. 

The work co-opted the financiers tool of live sentiment analysis regarding mentions of BPA’s (Bisphenol A) in online news production, and social media, to consider how surveillance, rather than a rogue element of capitalism, enmeshes with the effects of market forces upon the environment, happening at a molecular level.

Image 11

Visitors to the VR work Derivative arrive just outside the iconic striped postmodern landmark of 1 Poultry, immersed within a dusty crystalline maze, in which familiar landmarks merge with multiple fractured views reminiscent of popular ecological disaster sci-fi’s such as Bladerunner 2049 and Netflix series Mars.  Asking just how 'virtual' the effects of the markets are on the environment, as data from sentiment analysis software influenced the sandstorm polluting the landscape at the financial heart of the British nation state, born of the ongoing financial oppression of tax evasion and offshore banking. The work emphasised an important aspect of the climate challenges ahead will be to address present inequalities borne of colonialism.

Particles escaped from the virtual landscape as an enormous drift amassed up against the windows of the gallery, where outcrops of spawn-like eyes: The Prosthetics, reminiscent of surveillance capitalism, remained evocative of collective potential. Lag Lag Lag reminiscent of financiers monitors, enmeshed human cognitive as well as non-cognitive processes, blurring human/animal in-distinctions with soft computing, the molecular structure of Bisphenol A, and live data production.

Image 12

Co-opting these technologies, our pricing model utilised a financial quantitative model with BPA sentiment analysis, weather futures contracts, the FTSE, and local pollution data from the longitude and latitude of the gallery to contribute to a speculative view into the rise and fall in reputation of the top 100 companies responsible for over 70% of emissions, as public opinion turns, and insurance companies lose their appetite for underwriting companies dealing in the production of pollutants. Oldridge points to three key areas identified by the Bank of England’s Prudential Regulation Authority:

The first is physical impact risk, relating to the increasing frequency and severity of

extreme weather events with substantial costs of repairs rendering places prone to

these already uninsurable (home insurance in areas liable to flooding).

A second area of risk, also happening already, relates to legal liability, as litigation

cases related to the climate are beginning to become reality - potentially leading to fossil fuel divestment.

The third area of risk relates to investment and stems from the fact that the insurance industry is one of the world’s biggest institutional investors in fossil fuels.

As public opinion turns, it increasingly renders investments stranded assets, as coal, gas, and oil, suffer sudden and unexpected drops in value - rendering the bulk of the insurance industry’s investments worthless. As ‘unprecedented’ events become increasingly more every-day, so too, the historical loss records that traditionally guide underwriting and pricing risks, start to lose their value. Whilst, in the meantime, ex-derivative trader Jen Elvidge assured me of the multiple ways you can hedge the impending climate crisis, generating huge profits in the short term.

This is where the current moment crystallizes around wealth derived from a relation that inscribes ongoing inequalities. Where the magic sauce of memetic media meets the secret sauce of right wing billionaires, underwriting political campaigns to facilitate a wholesale move to the hard right, as various survival strategies emerge as the planet continues to heat up, facilitated to a greater or lesser degree by the ‘freedom’ that blockchain supposes in terms of citizenship, as progressives and the alt-right meet, and Davos man doubles down for the long game, to ride out the winter of democracy altogether.

A state of contingency that no longer promises an opportunity to ‘write the future’ but instead, is felt through the mechanisms of disaster capitalism as churning markets across both the financial sector and the mediasphere, as a means by which to game not only who has the authority to speak, but democracy itself.

Image 13

I’d been reading Sylvia Winter’s writing, and it struck me anew, how important it was the stories people were telling right now. Derivative draws from the popular imaginary of blockbuster film productions, but located amongst the City of London’s financial district, for something more akin to ‘Bladerunner 2019: the burnout’ in the year the first film was set. The replicant in Ridley Scott’s Bladerunner had been important to me growing up as a construct - one that was left ambiguous - as Deckard’s status is never revealed, whilst the 2049 film lost this nuance almost entirely. Fascinated by Bisphenol A as oestrogen left me during the menopause, it took me back to adolescence where I was prescribed synthetic oestrogen at the Tall Girls Clinic: Department of Growth and Development, Great Ormond st Children’s hospital, in a desperate bid to limit my growth. As I emerged via biochemical prosthesis, I was aware of a plasticity in stark contrast to any biological determinism.

If nature is unjust – then change nature

Here, the alienation inherent in being a cyborg (replicant, or post-human), as a machine aware of being a machine (Preciado) leads to an understanding of identity as a construct, that hence can be constructed anew, whilst foregrounding how technologies such as synthetic hormones, for example, lead to a writing technology of choice.

“We need to become aware of the technologies that we take that are producing

the subjectivities that are being produced through them”

Conversely, and, at the same time, Preciado’s reworking of Foucault’s history of sexuality (for Foucault is no feminist, let us not forget), describes the necropolitical regime as it enters biopolitical forms of government, with the regime of hormones in the pharmaco-pornographic era making progesterone and oestrogen the best sellers in the history of pharmacology, and key elements of global capitalism. In this update to Foucault’s panopticon, molecular entanglements enmesh with Deleuze’s modulatory control of data, as sentiment/emotion analysis mines Twitter and online news, uncovering grades of expressed emotions as well as importance and social meaning in order to predict the present, and thus transform social media signals into economic information and value.

Image 14

Covid19 has drawn into sharp focus all these conditions and worse, with existing inequalities, bias, and discriminations coming to the fore fast. A state of contingency – a chaos of churning markets and mediasphere – providing maximum opportunity “to exact ‘emergency measures’ with minimum scrutiny for oversight - in the pandemic version of “disaster capitalism”” writes Rachel Shabi in her article” The Pro-Privatization Shock Therapy of the UK’s Covid Response.

The ‘unprecedented’ nature of the pandemic reveals the structural weakness of a neoliberal ideology, that has shewn away the state as insurer of last resort for the sake of the myth of the market. Through decades of deregulation, austerity, and the lessening of governmental obligation to those it is supposed to serve. Whilst advances in technology draw out existing bias and discriminations like a poultis, the crisis reveals how these are underscored by economic realities, that to a large extent, determine who bears the brunt of the pandemic.

But, whilst the future might seem like it’s coming up incredibly short, what this could also suggest is an end to doing things in these ways, as the pandemic forces many to recognize that problems of this magnitude can only be solved collectively, amidst calls for a radical overhaul of capitalism that prioritises people’s welfare, to such a degree, that it is entirely unrecognizable as capitalism at all.

Footnotes

[1] Media Reform Coalition: New Report, Who Owns the UK Media in 2019. The Media Reform Coalition produced its first comprehensive report on media ownership in the UK back in 2015 when it argued concentrated ownership was a significant problem for any modern democracy. Four years later, it has produced an updated version that suggests that, not only does concentrated ownership persist but that the problem may be getting worse.

[2] For more information and critique on HFT - see: Low Animal Spirits by Ami Clarke - Journal Of Visual Art Practice, 2016. VOL. 15, NOS. 2–3, 145–160 http://dx.doi.org/10.1080/14702029.2016.1228759

[3] “In this way they have come to dominate what I call “the division of learning in society”, which is now the central organising principle of the 21st-century social order, just as the division of labour was the key organising principle of society in the industrial age.” Zuboff – Surveillance Capitalism.

[4] Reported by The Washington Post, NY magazine, and Andrew Yoskowitz in After Dawn news.

[5] The first time the work was presented at DRUGG (Diagram Research Use and Generation Group) diagram conference, at the Slade - was in the form of an audio recording, captured ‘live’ on the platform at Hackney Downs. The work in part, is my struggling to read the script as the audio-scape of trains / rail announcements carry on regardless, around me. Someone asked if I’d chosen to do it in this way because I’d wanted to remain hidden - which was interesting to me, as I hadn’t realised at this point that this was what I was doing.

[6] An online app – one of the many written ‘takes’ of the script Error-Correction, produced for the exhibition Snow Crash, Banner Repeater, London, 2014

[7] There seems to have been little governmental interest in scrutinising the role that tech plays in undermining and gaming democracy – until absolutely forced to do so. One might wonder if this might be due to, who, thus far, this has benefitted. Meanwhile there are many who have been critically engaged with these concerns for at least a decade and more.

[8] “Insidious - The Body Mass Index. Invented nearly 200 years ago by Adolphe Quetelet, during the early 19thC identifying characteristics of l’homme moyen – the average man - whom, to Quetelet, represented a social ideal. Quetelet believed that the mathematical mean of a population was its ideal, and his desire to prove it resulted in the invention of the BMI, a way of quantifying l’homme moyen’s weight. Initially called Quetelet’s Index, Quetelet derived the formula based solely on the size and measurements of French and Scottish participants. That is, the Index was devised exclusively by and for white Western Europeans. By the turn of the next century, Quetelet’s l’homme moyen would be used as a measurement of fitness to parent, and as a scientific justification for eugenics — the systemic sterilization of disabled people, autistic people, immigrants, poor people, and people of color. While Quetelet’s work was used to justify scientific racism for decades to come, he was clear about one aspect of the BMI: it was never intended as a measure of individual body fat, build, or health. For its inventor, the BMI was a way of measuring populations, not individuals — and it was designed for the purposes of statistics, not individual health”. Article by Your Fat Friend, medium.com.

https://elemental.medium.com/the-bizarre-and-racist-history-of-the-bmi-7d8dc2aa33bb

[9] Browne writes also of how the artwork Keeper Of Keys machine (kk) developed by Marc Böhlen (aka RealTechSupport) in the context of the Open Biometrics Initiative, could engender a critical biometric consciousness, beginning from the position, that: “understands all body data as probabilistic. By taking seriously the idea that identification and verification of fingerprint biometric data through computational means relies on probability—that a match is more akin to an approximation than a confirmation—the Open Biometrics Initiative designed the kk to subvert the notion that biometric identification technology is infallible.” Browne notes, interestingly, that the kk is “designed to re-imagine, beyond the confines of security and repression, notions of machinic identity control and biometric validation.”

[10] Zuboff writes further: Google the company, preoccupied with growth, handled data as a strategic asset: where the imperatives of expansion suggested that it ought to be shared with other tech companies, it did so without hesitation, giving access to Microsoft, Amazon, Yahoo, and even Apple (though Apple denied their involvement). Under capitalism, who gets to appropriate behavioral surplus is of only secondary importance; what matters is who gets to appropriate actual surplus—and to thereby remain in the position of doing so over the long term.

[11] See also N K Hayles’ writing on Ayache’s work in her book Unthought (2018) – pages 145 - 147

[12] Neural 60 Interview with Ami Clarke, Sept 2018.

[13] Can Jokes Bring Down Governments? Memes, Design and Politics by Metahaven 2013.

[14] Emily Nussbaum’s “How Jokes Won the Election” article: how many of Trump’s outrageous statements during the campaign had the structure of jokes. A lot of people have made a lot hay out of the “seriously”/”literally” dichotomy breaking down, but Nussbaum focuses instead on how it limits your ability to react, especially when you’re the subject of the “joke”: The political journalist Rebecca Traister described this phenomenon to me as “the finger trap.” You are placed loosely within the joke, which is so playful, so light—why protest? It’s only when you pull back—show that you’re hurt, or get angry, or try to argue that the joke is a lie, or, worse, deny that the joke is funny—that the joke tightens. If you object, you’re a censor. If you show pain, you’re a weakling.

[15] Back in 2004 Ron Suskind’s article Without a Doubt in the NY Times magazine describes the moment an anonymous Bush aide told him that “guys like me were 'in what we call the reality-based community,' which he defined as people who 'believe that solutions emerge from your judicious study of discernible reality.' “I nodded and murmured something about enlightenment principles and empiricism. He cut me off. 'That's not the way the world really works anymore,' he continued. 'We're an empire now, and when we act, we create our own reality. And while you're studying that reality— judiciously, as you will—we'll act again, creating other new realities, which you can study too, and that's how things will sort out. We're history's actors ... and you, all of you, will be left to just study what we do”.' The most outstanding example of course, the faulty case for war in Iraq, presented to the U.N., to Congress and to the American people, which the press had failed to detect, debunk, or resist.

[16] Stephen Colbert coined ‘truthiness’ in 2005, for an episode of Comedy Central’s “Colbert Report.”

[17] Conspiracy theories bloom wildly in this volatile churning vacuum, as truther communities gather together via ‘outsider status’, often holding several contradictory beliefs all sharing certain overarching similarities: the disdain or disinterest of mainstream institutions. The pandemic lays bare the strange convergence of environmental activists and alt right truthers, forming bizarre alliances via the anti-vax movement, all bound up with freedom of speech.

[18] 3.5 million Black Americans were profiled and categorised as ‘Deterrence’ by Trump campaign – voters they wanted to stay home on election day. Revealed: Trump campaign strategy to deter millions of Black Americans from voting in 2016 - 28 Sep 2020 report by Channel 4 News Investigations Team.

[19] Ioana Literat & Sarah van den Berg’s paper “Buy memes low, sell memes high: vernacular criticism and collective negotiations of value on Reddit’s MemeEconomy, Information, Communication & Society”, in 2017, describe how in a particularly self-reflexive corner of Reddit called r/MemeEconomy; “thousands of so-called meme traders spend their time discussing the ebbs and flows of the ‘meme market’. Members post requests for meme ‘appraisals’, and commenters respond by sharing their insider knowledge of meme trends and advising on whether it is safe or risky to ‘invest’ in particular memes. Numbering over 250,000 members and quickly growing, the community centres around the appropriation of stock market jargon to discuss memes as investment-worthy commodities. Unlike the stock market, though, no real money is involved; rather, the reward lies in being able to craft and participate in collective assessments of value, while cementing members’ status as meme insiders or connoisseurs.”

[20] Reports by the Southern Poverty Law Centre show – an American nonprofit legal advocacy organization specializing in civil rights and public interest litigation.

[21] The underlying references the material assets, or rather, the price – ie the ‘performance’ of the underlying, that drives the derivatives markets.

[22] Bisphenol A molecule, the synthetic oestrogen and pollutant, produced alongside the manufacturing of plastic since the 1930’s, that has seeped into the entire planets water supplies

[23] Shoshana Zuboff positions the ‘problems’ she encounters within Surveillance Capitalism as a rogue condition in an otherwise tolerable capitalist system, rather than focussing on the predative extractive relations that capitalism is predicated upon.

[24] The work points to the ancient semi-alien entity lodged inside the British nation state in the form of the City of London Corporation – that exists outside of parliament’s normal legislative remit – an offshore city within a city – that facilitates a third of all tax evasion in the world today. Dragging the past into the future, the works point to Britain as by far the biggest enabler of global corporate tax dodging, ground-breaking research by the tax justice network: The Corporate Tax Haven Index finds. The research highlights a widely acknowledged, but fundamentally unsolved situation that points to “the role of the UK and its network of Overseas Territories and Crown Dependencies in undermining the ability of other countries, including some of the poorest in the world, to provide for the most basic rights of their citizens.”

[25] with the help of ex-derivatives trader Jen Elvidge and programming by Rob Prouse.

[26] The environmental law firm Client Earth has already targeted the UK's biggest pension funds with a warning that they could face legal action unless they properly take account of risks to their investment portfolios posed by climate change - approximately 900 other climate related lawsuits are now underway in 24 countries, according to a survey by the United Nations Environment Program.

[27] as can be seen with the volatility in the markets generated by Brexit, and Covid for that matter - as hedge fund managers the world over make millions out of our volatile times.

[28] On Being Human As Praxis – editor Katherine McKittrick published by Duke University Press.

[29] A quote by the Xenofeminism group Laboria Cuboniks – tempered by the considerations focussed upon in After Geoengineering, Climate Tragedy, Repair, and Restoration by Holly Jean Buck, and others.

[30] Crucially, the work embraces complexity, and emphasises a shift to thinking about potential ways to shape things differently, that refutes any notion of natural versus artificial, or indeed, any binary at all, prioritising a hybrid approach in all things. Accompanied by a necessary scrutiny of the contributing factors to specific assemblages emerging throughout history, that construct gender, race, class, sexuality, and ableist bias’ and discriminations – troubled with a crucial criticality that questions the humanist project for only ever having afforded some humans rights, and not others.

[31] “Accordingly, for Preciado agency is accessible through prostheses: we are constructed through drugs, objects and representations, but we can also construct ourselves through them.” Transitional States – Hormones at the Crossroads of Art and Science.

[32] Public Lecture by Paul B Preciado. Testo Junkie: hormones, Power, and Resistance in the Pharmacopornographic Regime. Wellcome Collection talk 5th June 2018 – my notes. Transitional States: Hormones at the Crossroads of Art and Science.

[33] former Conservative Secretary of State for Business, Energy and Industrial strategy: Greg Clarke, Channel Four News, April, 2020.

[34] This follows Guardian data analysis which shows that across the UK, people from black, Asian and ethnic minority backgrounds are overrepresented in the sectors hit worst by the coronavirus crisis. Art & Design 5 August 2020 - Inside the Southbank Centre’s “laughable and terrifying” mass job cuts and shift to “start-up” culture. Europe’s largest arts centre will look to further commercialisation to recover from the coronavirus crisis. By Ellen Peirson-Hagger

https://www.newstatesman.com/culture/art-design/2020/08/southbank-centre-mass-job-cuts-redundancies-start-up-culture?fbclid=IwAR2npUp1HS6ZdRnvsal69WkPv3uKA-ai11BmrnjOGwBli9x1qcbwP8k3rC0

BAME workers disproportionately hit by UK Covid-19 downturn, data shows. Exclusive: Guardian analysis finds BAME people overrepresented in sectors hit worst by coronavirus crisis. Niamh McIntyre, Aamna Mohdin and Tobi Thomas

Tue 4 Aug 2020 https://www.theguardian.com/society/2020/aug/04/bame-workers-disproportionately-hit-uk-economic-downturn-data-shows-coronavirus

Bibliography

• Bender, John, and Marrinan, Michael. The culture of diagram. Stanford University Press. 2010. ISBN: 9780804745055

• Brown, Simone. Dark Matters: On the Surveillance of Blackness Paperback – Illustrated, 2 Oct. 2015 Book Pages: 224 Illustrations: 20 illustrations Published: October 2015. Paper ISBN: 978-0-8223-5938-8 / Cloth ISBN: 978-0-8223-5919

• Bunz, Mercedes. The Silent Revolution: How Digitalization Transforms Knowledge, Work, Journalism and Politics without Making Too Much Noise October 2013 (p 41) Clarke, Ami. Neural 60 Interview with Ami Clarke, Sept 2018. http://neural.it/issues/neural-60-blockchain-the-trust-catalyst/

• Coley. Rob and Lockwood. Dean. Cloud Time. The Inception of the Future. Zero books. 2012. ISBN 978-1-78099-095-8

• Cramer, Florian. Crapularity Aesthetics. https://makingandbreaking.org/article/crapularity-aesthetics/

• Cramer. Florian. Words Made Flesh - Code, Culture, Imagination. Media Design Research Piet Zwart Institute - institute for postgraduate studies and research Willem de Kooning Academy Hogeschool Rotterdam. 2012.

• Deleuze, Gilles. Postscript on the Societies of Control October, Vol. 59. (Winter, 1992), pp. 3-7. Stable URL: http://links.jstor.org/sici?sici=01622870%28199224%2959%3C3%3APOTSOC%3E2.0.CO%3B2-T

• Ekardt, Philipp. Vogl, Joseph. In the Pull of Time - a conversation between Joseph Vogl and Philipp Ekardt on speculation. 108 Texte Zur Kunst Verlag GmbH & Co. KG. 2014.

• Hayles. N. Katherine. Unthought. The Power of the Cognitive Unconscious. The University of Chicago Press, Chicago and London. ISBN-13: 978-0-226-44788-9

• Jameson, Cedric J. Black Marxism. 1983. University of North Carolina Press. First published 1983 by Zed Press, Caledonian Road, London. ISBN: 0-8078-4829-8.

• Jones. Ed. Five reasons why we don’t have a free and independent press in the UK and what we can do about it. Britain's press is controlled by the same networks of people as run everything else. Is it really free? Open Democracy. 18 April 2019 https://www.opendemocracy.net/en/opendemocracyuk/five-reasons-why-we-don-t-have-free-and-independent-press-in-uk-and-what-we-can-do-about/

• Karppi, Tero and Crawford, Kate. Social Media, Financial Algorithms and the Hack Crash. State University of New York at Buffalo. Microsoft Research and MIT Center for Civic Media.

• Lee. Patricia. Sturtevant - Warhol Marilyn Part of Afterall Books / One Work. Paperback 6 x 8 1/2 inches, 104 pp., 24 colour illus., 2016. ISBN: 9781846381638

• Literat. Ioana & van den Berg. Sarah. (2019) Buy memes low, sell memes high: vernacular criticism and collective negotiations of value on Reddit’s MemeEconomy, Information, Communication & Society, 22:2, 232-249, DOI: 10.1080/1369118X.2017.1366540

• Preciado, Paul B. Public Lecture. Testo Junkie: hormones, Power, and Resistance in the Pharmacopornographic Regime. Wellcome Collection talk 5th June 2018 Transitional States: Hormones at the Crossroads of Art and Science. My notes.

• Merlan. Anna. Vice magazine. 20th July 2020. The Conspiracy Singularity Has Arrived. With the pandemic and a global uprising against racial injustice to be explained away, conspiracy communities are bleeding into each other, merging into one gigantic mass of suspicion. https://www.vice.com/en_uk/article/v7gz53/the-conspiracy-singularity-has-arrived

• Metahaven. Can Jokes Bring Down Governments?: Memes, Design and Politics. Publisher: Strelka Press (1 July 2014) Language: English ISBN-10: 099291468X ISBN-13: 978-0992914684

• Rule, Alix & Levine, David. International Art English. On the rise—and the space—of the art-world press release. “International Art English” was produced by Triple Canopy as part of its Research Work project area, supported in part by the Brown Foundation, Inc., of Houston, the National Endowment for the Arts, and the New York City Department of Cultural Affairs in partnership with the City Council. https://www.canopycanopycanopy.com/contents/international_art_english

• Shabi. Rachel. The Pro-Privatization Shock Therapy of the UK’s Covid Response. New York Review of Books. July 6 2020.

• Wynter. Sylvia. On Being Human As Praxis. Editor Katherine McKittrick. Published by Duke University Press. 2015.

• Zuboff, Shoshana. The Age of Surveillance Capitalism: The Fight for a Human Future at the New Frontier of Power. ISBN-10: 1781256845. 2019. ISBN-13: 978-1781256848

Image citations

Image 1

The accompanying twitterbot @LowAnimalSpirit tweeting live speculative headlines from the HFT algo’s ‘portfolio’ as it speculated on what was about to trend. The analysis produces new phenomena in the form of headlines generated with the help of Natural Language processing algorithms.

Image 2

Low Animal Spirits

Most, probably, of our decisions to do something positive, the full consequences of which will be drawn out over many days to come, can only be taken as a result of animal spirits – of a spontaneous urge to action rather than inaction, and not as the outcome of a weighted average of quantitative benefits multiplied by quantitative probabilities...Thus if the animal spirits are dimmed and the spontaneous optimism falters, leaving us to depend on nothing but a mathematical expectation, enterprise will fade and die; – though fears of loss may have a basis no more reasonable than hopes of profit had before. The General Theory of Employment, Interest, and Money by John Maynard Keynes (1936) (VII chapter 12).

The projected visualisation is a glimpse into the HFT algorithms buying and selling activity. The work produced some unexpected results made visible via the visualisation process - where behaviour changes in the algorithm can be seen at a millisecond level as the words appear to ‘swarm’, producing a topology of value that points to the propensity for contagious viral behaviours. And never the more so than in a period of such Low Animal Spirits, post financial crash of 2008/9.

Photo Headstone to Hard Drive Central Saint Martins, University of the Arts London, 2015.

Images 3, 4, 5 (clockwise)

Headline from review of Low Animal Spirits by Phoebe Stubbs for ARTSLANT magazine.

Breaking News Flash Crash at End User, Hayward Gallery. Photo credit: Tomas Rydin.

Screen grab of the Associated Press twitter account hack 2013.

Image 6

‘Low Animal Spirits’ and ‘Error-Correction: an introduction to future diagrams’ performance for 'Liquidity: Art & Capital' - ICA London, 2016

Image 7

‘The eye that remains of the me that was I’ (Error-Correction: an introduction to future diagrams (take 3))

by Ami Clarke. Promo poster 2014. A whispered visual/audio work made available through a downloadable app that provides the ideal conditions for the work to be listened to – one to one, on headphones, kept in your pocket a little like a paperback. 

Image 8

‘Ami Clarke: Author of the Blank Swan’

installation for Annihilation at Lethaby Gallery, University of the Arts London, 2016.

Image 9

Ami Clarke, Author of the Blank Swan (2016) Ami Clarke, 2016 Published by Banner Repeater. Cover printed by Aldgate Press, ISBN 978-0-9929176-3-0 Screen capture of 3D render of clay cuneiform table, with astronomical procedure text for Jupiter: Mathematical rules for the area of a trapezoid.  British Museum COL: http://bit.ly/26LAyRI Model by British Museum Photographic Department. Elaine Sturtevant: Author of the Quixote (2009). Elaine Sturtevant, 2009 Editor: Udo Kittelmann Publisher: Walter Koenig

ISBN: 9783865604729 Installation for Annihilation at Lethaby Gallery, University of the Arts London, 2016.

Image 10

‘The Underlying’ by Ami Clarke, including Derivative (Virtual Reality, with live sentiment/emotion analysis re BPA's), Lag Lag Lag (video interface with live sentiment/emotion analysis re BPA's), and The Prosthetics (prosthetic optics, blown glass).

Exhibition: The Underlying by Ami Clarke: arebyte gallery, London, 2019. Commissioned by arebyte. Photo credit: Chris McInnes

Image 11

Derivative (VR work)

(Unreal Engine, Oculus Rift headset, 2 x Oculus Touch controllers, 2 x Oculus sensor, 2 x AA batteries, PC computer, HDMI cable and monitor, cable to connect to raspberry pi's processing sentiment analysis.)

Exhibition: The Underlying by Ami Clarke: arebyte gallery, London, 2019. Commissioned by arebyte.

Image 12

Lag Lag Lag by Ami Clarke (8 screen video interface, with live sentiment/emotion analysis re mentions of BPA on twitter and online news)

Exhibition: arebyte gallery, London, 2019. Commissioned by arebyte. Photo credit: Chris McInnes

Image 13

As the remnants of hurricane Ophelia made themselves known via the orange skies above London in 2017, everyone made the same Bladerunner joke. Screengrab of twitter feed 2017.

Exhibition: arebyte gallery, London, 2019. Commissioned by arebyte. Photo credit: Chris McInnes

Image 14

The Prosthetics (prosthetic optics, blown glass).

The Prosthetics drew reference from the Fates, the three sisters forced to share one eye between them. Suggestive of the surveillance that drives data analysis, they also point to the limited resources of a dwindling biosphere, but also to the collective approach necessary to face the challenges ahead.

[15hibition: The Underlying by Ami Clarke: arebyte gallery, London, 2019. Commissioned by arebyte. Photo credit: Chris McInnes[