No people, no problem: AI chatbots predict elections better than humans

The co-founders say they’ve purposely kept a low profile. “We’ve been very careful,” Koh said. “We don’t want to be seen as teenagers who meddle in elections.”

But Aaru is not shy about its ambitions. A white paper on its otherwise sparse website says “within two years, we will simulate the entire globe — from the way crops are grown in Ukraine to how that impacts production of oil in Iraq, trade through the strait of Malacca, and elections for the mayor of Baltimore.”

For now, the vast majority of Aaru’s clients are staying quiet about their use of its products. But last month, a think tank called Heartland Forward, created and funded by the Walton family (whose forefathers founded Walmart), hired Aaru to conduct a poll on what Americans in nine states think about the use of artificial intelligence. (Heartland forward focuses on states in the middle of the US).

While the results of the poll got some modest attention, the novel methodology was glossed over in news coverage.

While large language models like the ones that Aaru uses are prone to “hallucinations” — where they are often wrong — this problem can be minimized with the methods Aaru employs. Its models follow detailed instructions and draw answers from ingesting real-world data.

Aaru then instructs the bots to go through multiple steps to get to their final answer, a technique that helps the agents mimic human reasoning and has been shown to improve accuracy.

Even if the models sometimes err, polling is not an exact science, and surveys involving humans are often inaccurate, partly because people give dishonest answers.

As of earlier this week, Aaru was projecting that Harris would win the popular vote by 4.2 percentage points. The apparent assassination attempt over the weekend against Trump had almost no impact, according to the AI agents polled.

Aaru says its method allows for near real-time sentiment tracking by constantly feeding new information to the models. “We can watch, theoretically, the impact of every tweet that gets posted. We can look at internet trends and voter turnout on a micro level,” Fink said.

What large language models are good at, at their core, is prediction. They are the biggest and most powerful forecasting machines ever made.

They were trained on nearly every word that anyone has ever put on the internet. Completing the sentence “I will vote for X,” when lots of other contextual data is included, should not yield a very high margin of error.

Fink told me that anyone in politics who doesn’t shift to this method of polling will be left behind. That’s largely because this technology will allow people to do a lot more of it a lot faster.

Any politician can essentially run a political poll on a speech before they give it.

And, of course, corporate America will flock to this kind of tool for all sorts of reasons.

I asked Fink and Koh what they thought of well-known presidential poll analyst Nate Silver. Their response was diplomatic: “We respect all those who came before us.”

Researchers at Harvard looked at the use of AI chatbots in a study published last year and found that they still have some blind spots that could skew results.

“ChatGPT proves to be successful at responding like real Americans to some questions with a strong partisan divide, but it often fails to anticipate differences in public opinion along other human dimensions, such as demographics like age, race, and gender,” the study said. “ChatGPT also goes too far in extrapolating the expected partisan differences in response to events that took place after the training data it was created with, such as the breakout of the war in Ukraine.”

They did not, however, dismiss the idea, arguing in June that AI might change polling.