AI voice cloning technology is being utilized by celebrity estates to generate revenue from deceased Hollywood icons.

• A "professional voice clone" can be created using just 30 minutes of audio from a Hollywood actor.
• The reading app, ElevenLabs, has secured funding from prominent Silicon Valley venture capital firms and has struck deals with the estates of renowned actors such as Burt Reynolds, Judy Garland, James Dean, and Sir Laurence Olivier.

New business models are addressing concerns about unauthorized AI impersonation by allowing stars from Hollywood's golden age to be reborn through celebrity estate AI voice cloning deals.

The audio technology startup ElevenLabs, backed by venture capital firms such as Andreessen Horowitz and Sequoia, has struck deals with the estates of renowned actors like Burt Reynolds, Judy Garland, James Dean, and Sir Laurence Olivier, for its IconicVoices tool that generates AI-powered voices for users to listen to via an audiobook app.

Since its launch in 2023, ElevenLabs has been producing audio for books and news articles, video game characters, film pre-production, and social media and advertising. The company has already collaborated with major publishers such as the New York Times and Washington Post, and recently joined Disney's accelerator program.

To create a professional voice clone, you need approximately 30 minutes of high-quality audio, according to Sam Sklar, a member of ElevenLabs' growth team. The voices are generated from a celebrity's catalog, and once created, they can be used to read text content such as articles, PDFs, ePubs, newsletters, and other text-based materials. However, the voice and content cannot be exported, and all listening must be done through a reading app.

Users can listen to James Dean narrate articles within the app, but they cannot access his voice for any content outside of the app.

AI-generated voice content could become less contentious and more controlled with the help of certain types of deals. Google Play and Apple Books already use AI-generated voices to some extent, but there are significant challenges in replicating human voice pacing, intonation, and emotion.

Scarlett Johansson accused OpenAI of ripping off her voice after she rejected offers to license it, leading to concerns about the use of celebrity voices in the AI industry.

Sklar emphasized the importance of safeguarding against the risks associated with synthetic media, stating that they take the safe use of their tools very seriously. Measures include active moderation of content, accountability enforceable with bans, and special provisions for safeguarding the impact of AI voice on the 2024 election.

The current generation of actors is still anxious about the use of AI in generating voice content. Voice actors for video games have raised concerns, and last year's film and television strike had significant roots in anxieties over the use of AI. However, the use of iconic voices sold by estates is a market niche that potentially avoids these pitfalls, representing a new income stream from AI rather than a lost income stream because of AI.

The use of soundalike celebrity voices in advertising has been a contentious issue for decades, with examples such as Frito Lay's use of a Tom Waits soundalike in 1988 and Waits' own refusal of advertising deals. AI has made it easier to create soundalikes, but recent lawsuits against AI startup Lova for uncompensated use of voice actors in generating its AI voices serve as a reminder that the world of AI voice generation may remain a complex and potentially litigious one. Lova has denied the claims in the suit and also highlighted its revenue-sharing model for actors whose voices are cloned.

Steve Cohen, a partner at Pollock & Cohen representing voice actors in an unrelated lawsuit, stated that it is challenging to evaluate the safeguards in place without examining the specific language of the IconicVoices contracts.

The IconicVoices tool of ElevenLabs acquires permissions and manages the usage of voices in a specific manner.

"Cohen stated that permission, compensation, and control are the key factors."

New, clearer laws may discourage people from misusing their voices, but not for hardcore offenders, said Cohen. As Bette Davis once said in "All About Eve," "Buckle your seatbelts; it's going to be a bumpy ride."

The level of speech quality from ElevenLabs' text-to-speech tools is now indistinguishable from real human speech, according to Sklar.

The quality of AI is determined by the models it is trained on, and the actor's voice datasets are included in the training process.

"According to Nauman Dawalatabad, a postdoctoral associate at the MIT Computer Science and Artificial Intelligence Laboratory, neural models acquire their abilities by imitating and memorizing the subtleties and patterns found in their training data. The effectiveness of the model is highly dependent on the quality and variety of the training data."

Movie stars' vocal delivery could enhance AI mimicry and learning by providing high-quality voice datasets for training and fine-tuning large models, as Dawalatabad stated. However, he cautioned against using "sounding human" as the sole benchmark for the AI voice field, as it could perpetuate an adversarial relationship between human and synthetic voices.

The Audio Publishers Association's executive director, Michele Cobb, stated in an interview with CNBC last year that AI technology can enhance workflows and is not a new tool for voice talent, producers, and publishers, who use it to enhance their quality control in post-production.

AI voice licensing could ease the burden on voice actors, Dawalatabad stated, without replacing them entirely. They play a crucial role in the process by providing correction or enhancement to aspects such as intonation, warmth, and emphasis, which remain challenging to replicate.