The News
An off-Broadway theater in New York has a new way to attract audiences: live, AI-powered translations that make performances more accessible to non-English speakers.
Attendees for Perfect Crime, performed at The Theater Center and the longest-running play in New York, can scan a QR code to search an online translation platform. They then select one of 60 languages and listen to the live translation through headphones during the show.
The murder mystery about a psychiatrist accused of killing her husband can be heard in Arabic, Afrikaans, Polish, and other languages, reflecting one of the many ways AI is becoming a part of daily life. The translation’s text also appears on the user’s phone for individuals who are hard of hearing or wish to follow along that way.
With a majority of Perfect Crime’s attendees being tourists visiting New York, roughly 25 to 30 people use the service across eight shows each week, allowing for the theater to sell tickets to those who may not have bought them without the offering, according to Catherine Russell, general manager of the theater who stars in Perfect Crime. Theatergoers have selected a wide range of language options, with no specific language dominating, she said.
Perfect Crime actors wear microphones that capture their voices, which are fed directly into the translation system to ensure no side conversations or audience noises are picked up. It’s free for theatergoers.
Broadway shows have largely recovered from the pandemic slump, in which theaters were closed for 18 months in New York. Off-Broadway productions and regional theaters, however, continue to struggle filling seats and turning a profit. Russell hopes offering the technology will draw in a new audience that hasn’t typically attended English theater.
“A lot of people don’t go to the theater because it’s not accessible to them,” she told Semafor. “If they don’t understand English well, they don’t see it as an option.”
Russell also hopes the translation service will give her show a leg up over the nearby musicals, which tend to be more popular than plays. The technology will eventually reach musicals and Broadway, she said, but it will take a while. “Off-Broadway can be more nimble. We can do things that Broadway can’t,” she added.
In this article:
Know More
After testing a variety of AI-powered translation services, Russell selected a product offered by Wordly, a Silicon Valley-based startup. Its technology works by picking up voices and transferring them through a soundboard to a computer accessing Wordly’s cloud. The system then transcribes the audio into text and translates it into the desired language using a series of transformer models — the specific model depends on the language. Within three to five seconds, it delivers the result to users.
Clients employing the technology can also customize their translations so names of products or people — like the company “Apple” or the name “Rose” — aren’t translated into the objects when mentioned.
Wordly’s typical customers include industry associations putting on conferences and global business looking to streamline their internal meetings, Chief Marketing Officer Dave Deasy told Semafor. Recently, it has been working more with churches, schools, and city councils. For example, when fires tore through California at the start of the year, Los Angeles County used Wordly’s technology to translate its press conferences, Deasy said.
The Theater Center developed a partnership to use Wordly’s technology at no cost. Its service runs at about $100 per hour, so a theater running a similar two-hour performance would pay roughly $200 per show, Deasy said.
Wordly says the translations are accurate 95% to 100% of the time. It is working on expanding its language offerings — French Creole and Indigenous languages are on the list — and adding a variety of male and female voices (right now, there is only one option for each).
Worldly’s translation does not yet mimic the speaker’s voice undulation, pitch changes, and emphasis choices, so for theatergoers, it won’t sound as melodic or intentional as the actor on stage. In Semafor’s testing of it, the technology does sound natural. It comes up short of Sesame’s recent industry-wowing conversational voice demo, but outperforms the robotic-sounding Siri.
Room for Disagreement
Some are wary of incorporating AI into the arts, including Bliss Griffin, a New York-based theater consultant at her own company who also serves on the board of an industry research and advocacy group, the Theater Advocacy Project.
Playwrights spend a lot of time grappling with specific word choices, regionalisms, and idioms, and when translated by AI, some of that may be lost, she told Semafor. “The kind of word-for-word translation that a machine is likely to give is not a high-quality or inclusive experience, because word-for-word does not always get at the heart of things,” she added.
Another option is for shows to provide a recorded voiceover of the production that is translated professionally and can better convey the nuances of the dialogue, she said — although that also presents issues with timing.
Perfect Crime’s Russell said that while an AI-generated translation may not be perfect, the essence of the show remains.