Not in Kansas anymore: Digital tools take immersive experiences to the next level


Adin Heller

Senior Research Executive, New York

Categories Innovation

Digital tools are being used to great effect in the theatre to create highly immersive experiences – but these cutting edge tools could easily extend to retail environments as well.

When you go to see a film or play, the director’s goal is often to transport you from your theatre seat to another time, place, or world. But what if the action on the screen or stage wasn’t the only tool at the director’s disposal?

I’ve recently attended two plays that used digital and audio tools to make the theatre-going experience even more immersive and interactive than usual.

Elements of Oz

This inventive production from NYC-based company 3-Legged Dog is a mash-up of YouTube clips, live action, immediate video play-back, and augmented reality via smartphone. The live action bits largely consist of the (very) small cast reenacting scenes from the 1939 film version of The Wizard of Oz, interspersed with trivia about the film, which in turn is presented alongside clips from YouTube offering cultural commentary and context about the film.

Image courtesy of Elements of Oz
Image from Elements of Oz app

However, the real ‘star’ of the show is the proprietary Elements of Oz app, which the audience is instructed to download prior to the show and have open and running for the duration of the performance. The app allows for augmented reality interaction with the play’s program and with the performance itself, bringing aspects of the story like the tornado and the flying monkeys to life in ways that such a small production wouldn’t normally allow.

The Elements of Oz app uses augmented reality to bring the printed program to life; here, it imposes a whirling tornado over an image of Dorothy and Toto.

While it took some time to get used to sensory input coming from multiple sources at once (i.e. live action, video, and smartphone), Elements of Oz clearly demonstrates how the theatre experience can be made even more immersive and transportive than it usually is.

The app’s augmented reality feature sprouts technicolor flowers around Glinda the good witch

The Encounter

Having previously worked in the field of Audio Branding, I really loved that The Encounter, a one-man show conceived by and starring Simon McBurney, eschews elaborate set design and instead relies on sound to create a rich textural setting, making use of directional microphones, inventive props, and creative sound design. All of these sonic elements are heard through headphones worn by every audience member, which creates an extremely intimate and emotionally rich experience.

Headphones were provided at every seat in the theatre with plenty of technical support on hand for troubleshooting

Not only was I mesmerized by the closeness and ceaselessness of The Encounter‘s augmented audio, but I was also riveted by McBurney’s ability to use sound to help differentiate the various ‘versions’ of himself presented, as well as the character of Loren McIntyre he was playing, in this richly layered story. It turns out that one man and a pair of headphones can speak volumes about the fiction and simulacra that surround us every day.

Simon McBurney interacts with a 360° microphone shaped like a head (Image from

Beyond the stage

While it’s nice to have a bit of fun at the theatre, it’s also important to remember that retail environments aren’t all that different from the theatre stage. Thinking of your store or brand environment as a stage in which consumers and your products play the starring roles can open up a world of possibilities for both large and small scale activations – now more than ever with digital tools lying in wait in almost every pocket or handbag.

One retailer who has embraced digital to create a more immersive experience for its shoppers is Barney’s New York. Their new downtown store makes use of many digital tools, such as beacon technology that makes it easy for customers to enhance their shopping experience with videos, lookbooks and designer interviews from Barney’s editorial site The Window. Additional technology allows salespeople to be more attuned to their customers needs and wants, resulting in more personalized customer service.


Indeed, there is room for great creativity when it comes to the ways that retailers and brands can deploy digital and immersive tools like those those used in Elements of Oz and The Encounter to engage consumers in personal, emotional, and meaningful ways.