New Sites of Memory Making: Augmented Reality and Holocaust Memory
By Prof. Victoria Grace Richardson-Walden
Augmented Reality or ‘AR’ is still an emerging field in Holocaust memory. We explore examples from AR practice and theory and propose five recommendations for its future development.
What do we mean when we talk about ‘augmented reality’ or ‘AR’ projects? Blancas et al. (2021) bring together several definitions which emphasise that AR does not refer to a singular medium or technology:
Augmented Reality (AR) enriches the physical world with digital information, annotating reality and supplementing it with additional information (Feiner et al., 1997). A classical definition considers it a form of Mixed Reality (XR) in which a real-world view is supplemented by synthetic sensory input (Milgram & Kishino, 1994). For some authors, AR should fulfil at least three properties: combining real and virtual, interactive in real time, and registered in three dimensions (Azuma, 1997). An ideal AR system would make users believe the virtual and real objects coexist in the same space, blurring the frontier between real and virtual (Billinghurst et al., 2002).
It might be best then to define augmented reality as:
a particular mode of mediation, which produces a specific relationship between the body and the lived-world with and through media.
If broadcast and print media assumed an ocularcentric way of coming into contact with the mediated world, media theorist Mark Hansen (2006) recognises that current discourse in VR and by extension also AR has returned to an acknowledgement that the world is three-dimensional and experienced multisensorially, i.e., by the whole body. Hasen argues this shift towards ‘Mixed Realty’ eschews ‘representationalism and embrace(s) (…) a functionalist perspective rooted in perceptuomotor activity’ (p.3). That is to say, AR is post-representational – it is about spatiality, experience, and the body coming into relation.
Whilst the ‘augmented’ in AR suggests that digital technologies might be able to extend or provide better access to some dimension of reality, for Hansen, all reality is essentially mixed reality. And what the new mixed reality paradigm foregrounds, is the significance of the body as a site of meaning-making.
Therefore, when we talk about AR, MR or XR experiences, we should think about these as media-enabled bodily encounters with the world. In the context of Holocaust memory, this means thinking about how we might use media to produce experiences with this past that are multisensory and centralise the body as the place where meaning is made.
Essentially then, just as Hansen’s provocative subtitle argues that ‘all reality is mixed reality’, we might argue that ‘all Holocaust memory is already mixed reality memory’. Nevertheless, just as with wider broadcast era media, Holocaust museums, memorials and memory culture have tended to prioritise ocularcentrality. We go to museums to look, not touch. Whilst memorial sites often allow us to explore space with our bodies, they are annotated with text we should stop to look at or historical remains are interpreted by exhibitions including maps, facsimiles of documents, photographs, and objects behind glass.
AR in Holocaust Memory Practice
To date, there are only a few examples of experiences described as ‘AR’ for the sake of Holocaust memory. These include Sobibor AR (LA Holocaust Museum); the Oshpitzin App (Auschwitz Jewish Centre – which only using explicit AR in V2.0); Tower of Faces App (United States Holocaust Memorial Museum); The Liberation App and ARt (Dachau Memorial); Remembering the City (an Italian-produced Stolpersteine app); and some of the projects of the Future Memory Foundation, for example the reconstruction apps at Bergen-Belsen and Falstad memorials.
Many of these AR projects take the sites of former concentration camps as their learning space.
The Future Memory Foundation projects have VR or AR view options. They allow memorial site visitors to explore exhibition content such as images and testimony relevant to the place in which they stand. In the VR view, the visitor will see a 3D model of the site fill the screen on an iPad. In the AR view, they can see buildings only against today’s landscape.
The Oshpitzin app has not always used so-called ‘AR technologies’, but regardless it has always been an augmented reality experience. In its earliest form, when AR technologies were not specifically integrated, users would follow a map to a particular site and then open an information panel with photographs and a description (in text and audio). The user could then try to find the exact location where the historical photographs were taken (read more in Walden 2019).
Liberation and ARt allow visitors to hear audio, read text, and see photographs in the former, and artwork in the latter, in-situ. As one watches other visitors take selfies at the former gates of Dachau, holding one’s mobile phone up towards the gatehouse, the Liberation app will present a photograph of survivors hanging out of the window over what they otherwise see.
Liberation is now only available as a web version.
Similarly, although not at a former concentration camp site, Remembering the City enables pop-up information including audio, images, text and maps related to the people commemorated on Stolpersteine in Venice to appear in the vicinity of the stumbling stones.
Whilst Sobibor AR does not take visitors to the physical former site of Sobibor, it digitally recreates the physical 3D model long featured in the museum, which was created by survivor Thomas Blatt. The project was one answer to the LA Holocaust Museum’s question ‘How can a museum bring its unique collection to life and make Holocaust education innovative and accessible to students worldwide?’ A question that became particularly urgent to them during the Covid-19 Pandemic. Whilst claiming to be ‘AR’ and the model does appear in front of the user’s own lived-world context, it does not need to – it could work just as well in VR.
The app in the Tower of Faces installation at the United States Holocaust Memorial Museum takes a slightly different approach, using AR to connect individual stories. The famous memorial installation is a three-floor-high tower which commemorates the massacred community of the Lithuanian town Eisiskes. The tower is a dark space of black walls, covered in multiple photos of individuals or small groups. A handful of iPads are now available to visitors, who can point one to any photograph and immediately receive more information about the people and culture represented in the image.
Afterwards, virtual arrows appear making visible connections between this image and another from the town, subtly demonstrating that this was a living community. The individual photographs are no longer monads but part of the entangled life of Eisiskes.
AR in Holocaust Memory Research
To date, most research into the use of augmented reality for the sake of Holocaust memory has come from the computer sciences and, besides those by the Future Memory Foundation and Universitá Ca’ Foscari Venezia, tend to focus on prototype rather than publicly-released projects.
Whilst these studies claim to evidence the efficacy of AR in this context, none of them test their projects against any parameters relevant to Holocaust memory and education contexts. There has been much debate about whether Holocaust education research has an agreed set of values by which effectiveness can be measured (see last week’s blog), however such programmes tend to focus on:
- Historical knowledge
- Memorialisation
- Commitment to action in the context of civil/human rights
- Critical thinking and critical engagement with the past and present
In contrast, the few studies into AR’s efficacy in this context tend to consider ‘memory’ to be testable in terms of immediate recollection and later retention of facts (Oliva et al, 2015; Pacheco et al., 2015; Challenor et al., 2023). Long-term memory can be defined as only a week after the experience (Pacheco et al., 2015). In contrast, when we are thinking about collective Holocaust commemoration, long-term refers to decades into the future.
Whilst the Future Memory Foundation team argue that Holocaust education needs new, empirically grounded approaches (Blancas et al., 2021), they do not use Holocaust pedagogical measures in their study, but rather broader memory retention markers. They also situate their project in relation to the broader domain of Historical and Cultural Learning (HCL) (Verschure and Wierenga 2021), rather than the specifics of Holocaust Education. Similar claims from Pittarello et al., (2022, 2023) do not align with the testing parameters focused on ‘raising citizens’ awareness’ (2022, p.2).
Terms like ‘memory’, ‘empathy’ and ‘overwhelm’ have very different meanings in quantitative measurable studies and Holocaust memory.
What is clear at the current moment is that theoretical ideas about augmented reality, AR in Holocaust memory practice, and computer sciences driven research into using AR for Holocaust education conceptualise what AR is and what it is good for in this context extremely differently.
How can we reconcile these distinctions to create meaningful AR experiences for Holocaust memory? How do we measure their success appropriately?
Five Recommendations for Future Development
- Build interdisciplinarity teams from the humanities, sciences, education and museum teams, who co-create from initial aims through to mixed method, long-term impact studies. Importantly, define terms together.
- Build impact studies that focus on the four tenets of Holocaust education: historical knowledge; memorialisation; moral actions; and critical engagement.
- Move beyond thinking about ‘AR’ simply as augmented spaces or creating mobile or distant ‘tours’, and explore different augmented reality approaches beyond the Holocaust context to define principles for why and when it might be useful.
- Think about AR as putting the body first, and design augmented reality experiences from this perspective. What is the user able to do with their body? What meaning is produced through their actions? What/how do they interact with virtual and physical artefacts?
- Test, test, and test again. Engage users in the development of AR experiences from the earliest ideation through paper, analogue and digital prototypes.
