What is the future of AI in film?

Artificial intelligence is at the centre of a new controversy facing some of the most-promising Oscar contenders this year after production members of Golden Globe-winners The Brutalist and Emilia Pérez confirmed it was used in both films to enhance performances.

For the hotly-tipped The Brutalist, editor Dávid Jancsó said that he used AI to perfect lead actors Adrien Brody and Felicity Jones’ Hungarian dialogue. In Emilia Pérez, re-recording mixer Cyril Holtz confirmed at Cannes last year that he improved the singing performance of lead actress Karla Sofía Gascón with AI.

The Brutalist director Brady Corbet has defended the decision.

“Adrien and Felicity’s performances are completely their own,” he said. “Innovative Respeecher technology was used in Hungarian language dialogue editing only, specifically to refine certain vowels and letters for accuracy. No English language was changed.”

Additionally, Corbet clarified that it “was a manual process” done by both film and Respeecher teams meant to “preserve the authenticity of Adrien and Felicity’s performances in another language, not to replace or alter them and done with the utmost respect for the craft.”

Corbet’s comments have him straddling the battle lines drawn up in recent years over film’s use of AI. In 2023, for the first time since 1960 both the Writers Guild of America (WGA) and SAG-AFTRA went on strike simultaneously.

In one of the longest union strikes in Hollywood’s history, a key issue was studios using AI. “There was a lot of concern around the implications of AI in Hollywood and how employers might try to use it to undermine pay and working conditions,” Erica Knox, senior research and policy analyst at the WGA tells Euronews Culture.

The WGA and SAG-AFTRA went to the studios with proposals to protect their writers and actors from losing work to AI. Studios initially refused to negotiate on the topic, hence the strikes.

By the end of the WGA strikes in late September 2023, the union had won provisions to regulate the use of AI.

“What these proposals really do, at the heart of it, is they prevent employers from using AI to replace writers or lower their pay,” Knox explains. AI can’t be treated as if it’s a human writer, meaning any contribution to – and therefore compensation by – a human writer can’t be diminished by the inclusion of AI.

Writers also must be informed that they are working on material that has used AI and can’t be compelled to use the tool in their writing process. Similarly, the actors union SAG-AFTRA eventually negotiated in November 2023 that actors must consent directly to the use of any AI-based replica of them for a performance beyond what was originally scripted for them.

While the strikes were ultimately a victory for the unions, they didn’t entirely stipend studios from the use of AI in productions. It’s here that the current battle of AI use in film emerges.

Fighting against AI post-strikes

From the WGA’s standpoint, for example, much of the issue comes from the ways AI uses copyright protected material in their machine learning models. The union fought hard to create the copyright protections for writers’ works, yet companies that are outside of the Hollywood system scrape that data wholesale.

“We see this as a massive theft of our writers’ work,” Knox says. “Just generally speaking too, human creativity is a key ingredient of all the film and TV projects that we all love. Undermining that or removing human creativity means it’s less meaningful and enjoyable for everyone.”

It’s a point that was keenly felt by director duo Scott Beck and Bryan Woods when making last year’s Heretic. In the film’s credits, there’s a line that reads: “No Generative AI was used in the making of this film”.

Woods told Variety that he considered AI “an algorithm jumbling a bunch of shit together and then spitting it out as art. It’s not human and it’s borderline theft on some level.”

“It’s important for people to start talking about the need for human intersection in art, business and every facet of this life, because we’re on the precipice of every job on planet Earth being replaced overnight. It’s going to happen so fast. And it’s easy for it to happen in the arts. We’re in a business that is exceptionally greedy. Decisions are made for the bottom line and not for the good of the artistic process,” Woods continued.

On the flip-side, many studios have thrown themselves at AI tools believing them the future of the profession. In September last year, Lionsgate announced a deal with AI firm Runway to train a new model that can “augment their work”. Later that month, Avatar director James Cameron joined the board of directors for Stability AI, an AI company known for its text-to-image model.

AI tools for filmmakers, by filmmakers

Defenders of AI say that the technology has the ability to work in tandem with creatives to enhance and not subtract from people working in the arts. One such defender is British filmmaker Scott Mann.

Mann has directed multiple Hollywood projects, most recently 2022’s Fall. He’s also the co-CEO and founder of Flawless AI, a generative AI start-up intent on revolutionising film editing.

His first interest in AI was founded through seeing a terrible foreign language dub of his film Heist (also known as Bus 657).

“Robert De Niro and Jefferey Dean Morgan gave great performances and seeing those ruined in the old dubbing process, I didn’t realise the amount of compromises that went into that equation,” Mann tells Euronews Culture.

Mann sought out solutions and founded Flawless. The first product for Flawless was an AI tool that could improve the foreign dubbing process, reshaping actors’ mouths and performances to fit the new language’s words.

That product, TrueSync, was followed up by DeepEditor. With DeepEditor, filmmakers can film normally and then “do things like edit performances together, move cameras about and do different adjustments.”

“That for me is like the ultimate filmmaking tool. I’ll be able to make films like 10 times faster, 10 times better. That’s what this has always been about,” Mann explains.

If this all sounds like the comments of a typical AI CEO with speed and profitability as their only motivation, Mann assures that this is all about art.

“Why am I a filmmaker?” he asks. “It’s because I’ve pursued the idea that through art you can have an impact on the human condition. And filmmaking is the most technologically advanced form of art we have today.”

Mann, and Flawless he argues, aren’t trying to deprive anyone of the work making films. He wants it to be democratised. Filmmaking is a prohibitively expensive artform, financially and temporally.

With reduced timings and costs, “you start getting into a cycle of a more investable film industry model where you can afford originality again, because we haven’t been able to afford it,” Mann says, echoing the common criticism that Hollywood studios have increasingly only invested in hugely expensive guaranteed bankable titles.

The unions’ complaints though were that these tools limit the use of creativity. From his background as a filmmaker, Mann argues tools can still help that and not just ceaselessly create derivative works. “You have to build tools that reward originality,” he says.

This is where one of Flawless’s most interesting products comes in. Artistic Rights Treasury (ART) is a tool Flawless developed alongside the unions to build in agreements that limit the use of an actor’s performance to just the production they’ve agreed to work on. First created before the strikes, it mirrors much of what SAG-AFTRA fought with the major studios to have.

“You should own the rights to the things that are kind of going into these training models,” Mann tells Euronews Culture. An actor agreeing for their performance to be used in one film shouldn’t give studios the right to create other works with it.

“It’s all got to be handled and consented through the core owners of the materials, the original performers.”

Mann’s artist-first approach to an AI filmmaking company feels refreshing in a landscape where tech bros gleefully applaud the potential for these tools to strip the arts of their employees. As AI improves, it’s undoubtedly going to increasingly be used in films, one of the most high-tech artistic mediums.

However, as The Brutalist and Emilia Pérez have shown, it won’t be without controversy.