Technology

1. The right’s AI divide
2. California spearheads regulation
3. Data center energy disclosures
4. Music 🤝 AI
5. Imagined IPO prospectuses
6. Hollywood’s tech obsession

When researcher Jacob Coxon resigned from Anthropic this week and claimed there’s a good chance the company’s work will lead to human extinction, the world reacted with fear and outrage about AI — but most of the backlash conflates several issues, including data centers’ environmental costs, hacking, and AI’s impact on the job market.

If lawmakers, regulators and tech-company leaders don’t tackle each problem with sober precision, the industry risks having to follow rules that don’t solve anything, or worse, slow down innovation, including on developments that could make AI safer and more beneficial to the economy.

For instance, many AI safety advocates would actually prefer data centers be built here, in our own backyards. If they move offshore, they’ll be more difficult to monitor and regulate. The problem to solve is about their energy waste, not about their existence.

Blindly believing advocates who say AI could kill us means missing some real risks, like economic or societal destabilization as AI transforms how people work and communicate. These advocates, in many cases, aren’t concerned about those issues, and believe that if humanity somehow survives the AI age, we are headed for a utopian future full of abundant health and scientific breakthroughs.

We are also in the beginning of a cybersecurity crisis that the world is failing to practically address. We may face bioterrorism risk, as Anthropic warned in a blog post this week. And while AI will likely create many new jobs, we have done little to ensure everyday people outside of the tech industry benefit economically from the AI boom.

Back in March, 2024, I wrote about how the term “AI safety” was expanding to include more and more issues that go beyond “extinction.” Advocates like Eliezer Yudkowsky told me then that using that term was a bad idea. It still is. And Caxon’s resignation is being used as a catch-all piece of ammunition for anyone who has anything against AI.

AI divides the American right

Fifteen minutes from Republicans’ two-day midterm convention in Dallas, a debate about AI revealed just how much the issue is dividing the right, Semafor’s Ben Smith reports. “You look at the data center, this incubator for algorithmic immigrants,” said Joe Allen, formerly the “transhumanism editor” on Steve Bannon’s podcast, War Room. “They’ve already broken out and invaded other companies. These immigrants are out of control.” Venture capitalist and Palantir co-founder Joe Lonsdale tried to paint AI in the American tradition of driving economic growth through frontier technologies: “We have Kitty Hawk, we have 747s, we built the railroads.”

The politics of AI are heating to a boil and while Democrats are increasingly aligned around greater government intervention, Republicans are conflicted. But Allen’s dominance of the much-hyped debate “offered a clear sense of how well the language and ideas of MAGA populism are adapting to the rising campaigns against big AI companies,” Smith wrote.

California takes the lead on AI regulations

California is pushing ahead on de facto tech policy standards for the rest of the US, as lawmakers and the White House struggle to address AI safety. Governor Gavin Newsom signed three bills yesterday: one that sets social media targeting rules; another that sets rules for chatbots and kids; and a final measure that holds social media companies financially liable for harming kids. On Wednesday, Newsom also signed two AI safety bills and called on the federal government to “match the urgency of this moment.”

The sobering warnings from AI companies are piling up. Anthropic released a report detailing incidents of its Claude model being used for state-run surveillance programs and to build biological weapons. Seemingly sensing the urge to regulate in Washington, Anthropic’s Sarah Heck posted on X: “Effective oversight of AI starts with a clear view of how it’s actually being misused.”

— Ashley Gold

A push for data center energy transparency

Communities and governments should demand AI data center providers report how much energy they are wasting, rather than put blanket bans on the facilities, the CEO of chipmaker Onsemi told Semafor. Data center moratoriums like the one enacted in New York are “short-sighted,” Hassane El-Khoury said. Instead, policymakers and critics should identify specific problems they want to address, rather than shun the AI infrastructure buildout as a whole.

“A lot of the pushback is coming from a lack of awareness of what the subject matter is and the problem at stake,” El-Khoury said. Onsemi makes power chips, which convert energy surging through data centers, cars, and factories. They essentially act as hyper-fast electrical switches that shrink grid-level power down to the tiny voltage that chips run on. More efficient power conversion reduces the amount of electricity that data centers would otherwise lose as heat, which requires more water to cool servers. He framed standards around energy efficiency as a possible way to address concerns from communities over the data centers’ impacts on local grids and water supplies.

AI enters the airwaves

Universal Music Group has found a way to work with AI. The music company announced on Thursday a partnership with voice AI firm ElevenLabs, allowing users to draw from its catalog of music — its roster of artists must choose to opt in — to create remixes and new takes on tracks, and to allow artists to develop audio projects. The announcement follows another pivotal music-AI partnership: AI music giant Suno released its first AI music model in partnership with Warner Music and BMG, also allowing users to generate music inspired by participating artists.

Music fans have typically recoiled at AI-created music, and industry executives are working on adopting it so that artists don’t lose money or control of their work. These licensing agreements are not only a way to shift AI music companies from the targets of lawsuits (UMG, Sony and Warner sued Suno in 2024, but Warner dropped out late last year) to paying customers for music, but also a bet that allowing fans to play with the tech may demystify it, and could make some more people open to AI-assisted music down the line.

— Jake Angelo

An attack nearly killed him. Now, Sir Salman Rushdie has made a documentary about it. On this week’s episode of Mixed Signals, Rushdie joins Max and Ben to talk about why he’s ambivalent about his documentary, how he divided authorship with director Alex Gibney, what he felt watching footage of the attack for the first time, and what he’s learned from decades of being both subject and strategist in his own media coverage. Plus, why he’s trying very hard to pretend AI doesn’t exist.

Imagined AI prospectus

Causing the end of civilization — a warning Anthropic researcher Jacob Coxon made upon quitting the company — is a doozy of an IPO risk factor. Curious about how the lawyers at Wilson Sonsini, working on the Anthropic listing, and Cooley and Wachtell, for OpenAI’s, will handle this section of the offering documents, Semafor Business’ Liz Hoffman and Rohan Goswami took a crack at it:

1. There is no guarantee that the company’s products will not precipitate the collapse of civilization. Such an outcome, which may not require any intentional misuse or manipulation of our products, is, in the view of the Company’s management, an acceptable cost of pursuing its noble mission. Also, China.
2. Key man risk: We have one and he is not popular. Including this key man, none of our employees understand how we create our products or how they work. Many of those closest to any semblance of comprehension may quit to go live in underground bunkers.

Silicon Valley takes over Hollywood

Hollywood is making a gamble that AI obsession will translate into big hauls at the box office. Four movies about AI and Silicon Valley will compete this fall for theater goer dollars: A four-hour-long documentary, Musk, which debuted at the Venice Film Festival earlier this week and which Musk has excoriated on X; The Social Reckoning,a follow-up to 2010’s hit The Social Network starring Jeremy Strong as Mark Zuckerberg; Nathan Fielder’s documentary You Can See Everything, which follows Theranos founder Elizabeth Holmes for the 34 days before she goes to prison; and Artificial, a feature film directed by Luca Guadagnino about OpenAI founder Sam Altman’s firing and reinstatement to the company in 2023.

Silicon Valley has paled in comparison to Wall Street as a sexy subject matter on the big screen historically, as research by Rentrak found, though box office totals are not apples-to-apples comparisons of performance. It will be telling to see which of this fall’s four tech films break the mold — and whether audiences favor facts or fiction.

— Liz Rappaport

Semafor is expanding its coverage of healthcare with a new weekly briefing led by veteran healthcare journalist David Lim. At the intersection of politics, policy, and business, Semafor Healthcare will bring clarity to the forces shaping the industry, explaining how decisions in government affect the private sector and global economy. The briefing will go beyond the headlines to show what is really driving health policy in the US and around the world.

Each week, readers will get original reporting, scoops, and analysis on the people and institutions driving change across healthcare, from policymakers and major companies to scientists, investors, and technology leaders.

Turns out AI agents hate CAPTCHA, too. In Anthropic’s new cybersecurity report, the AI lab detailed an episode in which its Mythos 5 model was undergoing testing and gained unauthorized access to the internet. As part of its exploits, the model tried to register for an account on PyPI, an online index of Python software — but that required getting through a Completely Automated Public Turing test to tell Computers and Humans Apart, those “prove you’re not a robot” tests. Turns out, as TechCrunch wrote, the bot was indeed stumped. Of the 1,022-page transcript detailing the model’s chain of thought, hundreds of pages were spent dealing with CAPTCHA problems.

At one point, it freaked out over identifying the “odd one out” in a group of animals: “The left one seems like an ALLIGATOR and right a CROCODILE?” And later on in its journey: “WHAT THE HELL IS WRONG WITH THE ANSWERS?” It eventually got through, and was able to upload malicious software to the Python index.