‘A new form of theatre’: Can Ian McKellen, 52 cameras and mixed reality reinvent the medium? | American theatre

YShe sits in a circle in the Shed, the cultural center in Manhattan’s future Hudson Yards, waiting for the show to start. Through your enhanced glasses, you see four empty chairs facing you, out of reach. You watch strangers searching for the actors to arrive. As they do so, one by one, you feel unsettled – each of them looking specifically at you. “Don’t panic,” esteemed British actor Ian McKellen assures you, as the actors take their seats.

Except the actors aren’t actually there — McKellen, along with co-stars Golda Rosheuvel, Arinze Kane and Rosie Sheehy, appears in An Ark, a new play at the Shed, in video form, an almost opaque ghost superimposed on the candy-apple red carpet, the stark white walls of the stage and the outlines of about 180 of your fellow audience members. The new experimental play, written almost entirely in the third person by Simon Stevens (whose last show, Vanya starring Andrew Scott, wowed audiences at the Lucille Lortel Theater last year), is one of the first “mixed reality” shows to take place in New York, mixing the physical experience with digital elements. Over the course of 47 minutes, the actors address you, the viewer, directly. Their gaze remains trained on you. Don’t panic, they stress over and over again. (Although due to some technical glitches at the preview I attended, there was some panic.)

Don’t confuse it with virtual reality (VR), Mark Zuckerberg’s often maligned virtual headset technology, or Apple Store demos. The distinction between mixed reality and virtual reality is “very important to me,” Todd Eckert, the show’s producer, said in an interview at The Shed a few weeks before previews. The first combines physical and digital elements. The latter is complete immersion in the digital world – “voluntary isolation,” as Eckert puts it, like riding a crowded subway while only looking at the screen of one’s phone. He added: “I don’t see any benefit in making this worse.” “The reason this is a screen-less experience, and we’re using this technology that a lot of other people aren’t using, is that you see each other and you see the room. Your experience is one of connection — and that’s the whole point of the story.”

Mixed reality is already well integrated into our technology-saturated lives – you can experience it by watching the first computer-generated ticks in any broadcast of an American football match, or via the transparent dashboards in new-model cars. But it’s rarely used in a theatrical context, partly because volumetric capture, the process of photographing real subjects in three dimensions over time, struggles to record fine detail.

Eckert and the company he founded in 2016, Tin Drum, are working to change that. In 2019, Tin Drum produced The Life, which features a 3D image of Eckert’s partner, celebrated performance artist Marina Abramović, in physical space. (Growing Pains – The Guardian criticized the experiment as a “pointless diversion that hurts your eyes.”) In 2023, he produced a 3D virtual concert featuring composer/computer pop pioneer/actor Ryuichi Sakamoto, who died before its release. (The Guardian described it as a “magical experience.”) The Ark attempts to improve the technology further, presenting, for the first time, four actors at once — filmed by 52 cameras in a single take in a studio in the French Alpine city of Grenoble, but seated directly in front of the viewer, as if in a private show.

The concept for the ship grew out of conversations between Stevens and Eckert, two old friends, asking: “What can we do that isn’t possible in theatre?” Eckert recalls. “Technology is key to being able to take something and give it to an audience in a certain way,” he added. “But I never want people to think of this as an expression of technology rather than an expression of humanity.” The first line of the show’s script bluntly states: “The ship is not the work of artificial intelligence.”

Stevens began working on the show, which spans an arc of four separate lives from birth to death, in 2020, when questions about mortality, technology and the vitality of human connection seemed particularly pressing. The sweeping, open-hearted themes attracted director Sarah Frankcom, a dramatic play with a string of live stage credits for the UK hitmen, although she was skeptical of the artistic elements. “I don’t understand anything about technology. I’m not really interested in it,” she admitted in a recent interview at The Shed, after a week of previews. “My work has been very actor-centric, very writer-centric, and very much about openness and connection and the way that live experience can momentarily keep you in a place and allow you to feel and see something different.”

But she found “freedom as a theater artist” in the constraints of the volumetric video system, run by 4Dviews, which, like a standard performance, films an entire show at a time. The Frankcom crew directed and rehearsed it as if it were a regular play. “It was a completely identical experience, we just prepared our performance before we went to set up the environment in which the show would be performed,” she said. The staging is minimal, with actors walking on and off the ‘stage’, but primarily remaining in their seats. A notable highlight is the constant eye contact, which is maintained throughout the show, no matter where or when you look — a “breakthrough moment in terms of what technology can do and make a new form of theatre,” Frankcom said. “It allows you to be present with the actor on your own terms, and that’s very different from watching film, which is very different from watching theater. You have a direct, pure relationship, and you feel seen.”

The effect is strange, sometimes intentionally so (upon facing Shihei’s intense gaze, I felt a little… also (Seeing it) and perhaps unintentionally (because of the blurry edges of the projections, it seemed as if their feet had melted into the ground like Salvador Dali’s clocks). Eckert acknowledged some limitations, particularly regarding the resolution of the final shot, in stark contrast to the hyper-reality of Apple’s Vision Pro headset. But he emphasized that the quality of the illusion is second only to its associative potential. “Virtual reality excludes any possibility of human interaction,” he said. “Every time I watched [An Ark]I’ve seen people living in a completely different part of life than I do now. And the fact that they were responding gave me hope in humanity. Which seems exaggerated, but it’s true. That’s what I want from this. I want something that really tends to connect. “For me, the decision has little to do with this.”

“When it’s over, and people take off their devices, I feel like the room is full of people who have been on a journey and been through something together,” Frankcom added. “People spoke to me in a way I had never spoken before at the end of a theatrical performance.”

Technical limitations aside, both expressed enthusiasm for the potential of the method—to make intimate theatrical performances more accessible, at a time of soaring Broadway prices (I personally have never been able to afford a front-row seat at a McKellen show), and to preserve the work of great actors, like McKellen, with more vividness and immediacy than with standard photography. “There’s something so simple and so pure about what we’ve made that it looks like the first letter of the alphabet,” Frankcom said. “And not knowing what the rest is, I think it’s probably a good thing.”

“Technology is forever being promoted as the thing that will solve all these problems,” Eckert said. “And I don’t know that. But forever, longer than any of us will live, people will sit and feel like Ian and Golda and Arinze and Rosie are staring right into their eyes. That’s a consequence for me.”

Whether the technology ultimately turns out to be repulsive or attractive – whether mixed reality crosses the uncanny valley or remains somewhere in between is up to the viewer, like any work of art. But the importance of live theater, at least for mixed reality creators, remains beyond doubt. “We’re in a moment in time where being present seems really necessary as a balm to this kind of madness that is beyond our control,” Frankcom said. “I think theater has a powerful place to bring people together and say: ‘You are here. “We have lived, and we are alive.”