Explained: Augmented reality, and how artists and institutions used it to make culture an at-home experience during the pandemic

The past decade has seen the art world embrace Augmented Reality (AR) technology in various forms. From individual artists to galleries and institutions such as museums and public art endeavours, AR has been used not only to enhance viewers' experiences of the art (say, by including historical or factual information, exposition etc) but also by adding layers to the interactive experience (movement, graphics, animation being added to the visible artwork).

AR is different from Virtual Reality in that it augments/enhances or adds to a pre-existing reality. While VR requires the use of specialised gear like goggles or a headset for the user to be immersed in a fictional world, AR can be experienced through merely a smartphone, or a specific app. If you've ever wished the art in our Muggle world was more like that from the Harry Potter universe — subjects moving across frames, always in motion — AR makes that possible.

The Pokémon Go craze — the app was downloaded over 11 million times globally — showed just how ubiquitous AR tech could be, and how much its popularity could cut across age, geography and other demographic parameters. It's hardly surprising then that the art sector too has employed AR in a variety of ways.

Tech giants step in

Google, Facebook, Apple and Microsoft have all liaised with artists and in some cases, institutions such as the Tate Museum, to curate AR art projects. Apple’s [AR]T initiative, curated by the New Museum, saw it conduct public art walks in Central Park, New York City, where viewers were led to various locations around the park where specific artworks had been installed, and used their phones to experience the AR elements such as speech bubbles, animated sequences, text and audio elements like a poetry reading.

However, criticisms have been levelled against these endeavours by tech companies because they are often branding-driven. For instance, the Apple [AR]T initiative required users to visit an Apple store, use an iPhone and Apple's wireless headphones in order to experience the public art walk around Central Park. As writer Ben Davis concluded, the viewer was not experiencing art and AR so much as they were experiencing art/AR through the corporate framework of Apple. In other words, the product and not the art was the focus.

Institutions and AR

Leading museums and galleries around the world have been making use of AR so that their visitors can experience significant collections in entirely new ways. For instance, the National Museum of Singapore crafted an interactive AR experience around images from the William Farquhar Collection of Natural History Drawings, rendering animations in 3D:

An innovative use of AR tech was demonstrated by Art Gallery Ontario and the digital artist Alex Mayhew. When visitors positioned their phone or tablet screens over a classical artwork, an iteration by Mayhew would transpose the same scene and subjects into a 21st century setting (books, for instance, might be replaced with smartphones, a pastoral landscape with traffic and pollution).

At the Smithsonian Institution in Washington D.C., an app called Skin and Bones could be used by visitors to recreate creatures in AR by pointing their phones at their exhibited skeletons/fossils.

Some concerns were expressed over the potentially isolating/exclusionary pitfalls of incorporating AR tech in museums: would older viewers be able to use the technology? Would viewers become isolated as they experienced the AR elements of works rather than enjoying the exhibits as a collective activity? These concerns have, however, been proven to be unfounded.

Dedicated apps, and how the pandemic changed AR use in art

While fixed installations, public art tours and museum/gallery exhibits had been making use of AR for a while, the COVID-19 pandemic and the resultant lockdowns, social distancing and self-isolation presented a new reality for the art world to grapple with.

A name that often crops up in this connection is "Acute Art", an app start-up founded in 2017. Curator and critic Daniel Birnbaum began collaborating with artists to create more AR and VR works from 2019; when Acute decided to focus on its AR app to reach viewers at home, the events following March 2020 made "this move seem prescient," as writer Samantha Culp notes. The app reportedly has had over half a million downloads, and enables AR elements of storytelling, audio and cinema over works by artists.

The University of Chicago is among the institutions that used AR technology to bring art into people's homes during the pandemic. Previously, the university had collaborated with the artist Jenny Holzer on her project You Be My Ally, in which, pedestrians within the university campus could use their phones to project text from Holzer's work onto the campus buildings with the help of a free-to-use app. When the pandemic struck, the university and Holzer made it possible for viewers to project the artworks in their own spaces.

A model that is being increasingly preferred is a hybrid one — site-specific installations and AR-enabled experiences that viewers can enjoy wherever they happen to be. As travel still remains limited, and with headcounts in public spaces being controlled as a safety measure, AR tech that brings art to people is going to continue to grow in impact.