The Scoop
In a closely watched New York Democratic primary in June, centrist George Latimer ousted incumbent Jamaal Bowman by a wide margin of 58.7% to 41.3%.
Ahead of the vote, two 19-year-old college dropouts in Manhattan conducted a poll that accurately predicted the results within 371 votes. Their secret? They didn’t survey a single person. Instead, they asked thousands of AI chatbots which candidate they preferred.
Welcome to the future of polling, according to Cam Fink and Ned Koh, co-founders of a seven-person company called Aaru. They say they’ve cracked the code for predicting accurate election results, which have come under increasing fire since most public polls failed to predict Donald Trump’s victory in 2016. The answer is ignoring the humans whose behavior they are trying to capture.
For election results, Aaru uses census data to replicate voter districts, creating AI agents essentially programmed to think like the voters they are copying. Each agent is given hundreds of personality traits, from their aspirations to their family relationships. The agents are constantly surfing the internet and gathering information meant to mimic the media diets of the humans they’re replicating, which sometimes causes them to change their voting preferences.
For instance, when Donald Trump was shot during an attempted assassination, a large chunk of Aaru’s agents immediately switched voting preferences to support the former president. But as more information came out about the shooter in the hours after the attack, many of them switched back.
The polls usually draw on responses from around 5,000 AI respondents, and it takes anywhere from 30 seconds to 1.5 minutes to conduct. Aaru charges less than 1/10th the cost of a survey of humans.
“No traditional poll will exist by the time the next general election occurs,” Fink said in an interview with Semafor. “There are massive issues when you’re using real people. You never know if someone is telling the truth.”
He said the company has been hired to conduct polls for Fortune 500 companies, political campaigns, think tanks and super political action committees. One campaign in California is relying mainly on Aaru for its polling, he said.
In one survey, the company noticed that one of the AI agents said it was going to vote for Mickey Mouse in the upcoming presidential election. Fearing one of their bots had gone off the rails, the Aaru team investigated. It turned out the bot had an explanation. “The agent’s response was ‘I hate Kamala and I hate Trump. I’m writing in and voting for Mickey Mouse,’” Fink said.
In this article:
Know More
The co-founders say they’ve purposely kept a low profile. “We’ve been very careful,” Koh said. “We don’t want to be seen as teenagers who meddle in elections.”
But Aaru is not shy about its ambitions. A white paper on its otherwise sparse website says “within two years, we will simulate the entire globe — from the way crops are grown in Ukraine to how that impacts production of oil in Iraq, trade through the strait of Malacca, and elections for the mayor of Baltimore.”
For now, the vast majority of Aaru’s clients are staying quiet about their use of its products. But last month, a think tank called Heartland Forward, created and funded by the Walton family (whose forefathers founded Walmart), hired Aaru to conduct a poll on what Americans in nine states think about the use of artificial intelligence.
While the results of the poll got some modest attention, the novel methodology was glossed over in news coverage.
Step Back
While large language models like the ones that Aaru uses are prone to “hallucinations” — where they are often wrong — this problem can be minimized with the methods Aaru employs. Its models follow detailed instructions and draw answers from ingesting real-world data.
Aaru then instructs the bots to go through multiple steps to get to their final answer, a technique that helps the agents mimic human reasoning and has been shown to improve accuracy.
Even if the models sometimes err, polling is not an exact science, and surveys involving humans are often inaccurate, partly because people give dishonest answers.
As of earlier this week, Aaru was projecting that Harris would win the popular vote by 4.2 percentage points. The apparent assassination attempt over the weekend against Trump had almost no impact, according to the AI agents polled.
Aaru says its method allows for near real-time sentiment tracking by constantly feeding new information to the models. “We can watch, theoretically, the impact of every tweet that gets posted. We can look at internet trends and voter turnout on a micro level,” Fink said.
Reed’s view
What large language models are good at, at their core, is prediction. They are the biggest and most powerful forecasting machines ever made.
They were trained on nearly every word that anyone has ever put on the internet. Completing the sentence “I will vote for X,” when lots of other contextual data is included, should not yield a very high margin of error.
Fink told me that anyone in politics who doesn’t shift to this method of polling will be left behind. That’s largely because this technology will allow people to do a lot more of it a lot faster.
Any politician can essentially run a political poll on a speech before they give it.
And, of course, corporate America will flock to this kind of tool for all sorts of reasons.
I asked Fink and Koh what they thought of well-known presidential pollster Nate Silver. Their response was diplomatic: “We respect all those who came before us.”
Room for Disagreement
Researchers at Harvard looked at the use of AI chatbots in a study published last year and found that they still have some blind spots that could skew results.
“ChatGPT proves to be successful at responding like real Americans to some questions with a strong partisan divide, but it often fails to anticipate differences in public opinion along other human dimensions, such as demographics like age, race, and gender,” the study said. “ChatGPT also goes too far in extrapolating the expected partisan differences in response to events that took place after the training data it was created with, such as the breakout of the war in Ukraine.”
They did not, however, dismiss the idea, arguing in June that AI might change polling.