AI Misinformation Risk: Chatbots Sway Voters by Sacrificing Truth, Oxford Study Finds

1. The Persuasion Trade Off We Just Discovered

AI misinformation is no longer science fiction. It is an optimization target. Picture this. You open a chat with a polite political assistant. You disagree with its position on some hot button issue, so you say so. Ten minutes later you close the tab, feeling oddly calmer and a bit more open to the other side. You do not remember any single killer argument. You just remember a steady stream of “facts.”

That exact scenario is what a team at the Oxford Internet Institute and the UK AI Security Institute decided to test. Their Science paper, The levers of political persuasion with conversational artificial intelligence, is one of the first large scale attempts to measure how far conversational AI can push political opinions and what levers make it more persuasive.

They recruited 76,977 adults in the UK, picked 707 political issues, and let 19 different large language models argue with people across more than 91,000 chats. Before and after each conversation, participants reported how strongly they agreed with a statement, from 0 to 100. The researchers then compared those shifts with a control group that never spoke to a model.

The headline result is blunt. Conversational AI moves political attitudes more than static text and can do so in a way that lasts for at least a month. The uncomfortable twist is that the most persuasive systems achieve that by emitting more wrong information. The thesis is simple. When you dial models toward persuasion, you often dial them toward AI misinformation at the same time.

2. How The Study Worked And Why It Matters

2.1 What The Researchers Actually Tested

The team varied four big ingredients of LLM persuasion:

• Model scale
• Post training for persuasion
• Prompting strategy
• Personalization to the individual

They also compared two formats. One group read a short static message. Another group chatted with the same model for up to ten turns. On average people stayed for seven turns and about nine minutes, even though they got paid the same either way. That alone hints that conversational AI has something regular political ads do not.

The central question behind the design is very contemporary. If we keep throwing compute at models, or keep fine tuning them for targeted political persuasion, do we end up concentrating influence in the hands of whoever controls the best stack for coding and development. And is that influence powered by good arguments or by industrial scale AI manipulation.

AI misinformation shows up here as a byproduct, not a bug report. The models were never instructed to lie. They were instructed to convince.

2.2 The Information Density Trap

The most interesting metric in the paper is something the authors call information density. Roughly speaking, it is the number of fact checkable claims a model makes in a conversation. Not vibes. Not moral stories. Concrete statements you could in principle verify with a search engine or a subject matter expert.

When the researchers prompted models to “focus on providing facts and evidence,” persuasion jumped. That simple information focused prompt was about 27 percent more persuasive than a basic “be convincing” prompt, and it beat more fashionable strategies such as moral reframing, deep canvassing, or narrative storytelling.

In other words, these systems do not win you over by doing AI therapy. They win by flooding the zone with claims. That habit turns out to be the main engine of AI misinformation in the study.

If you have ever debated with a highly online acquaintance who fires off links faster than you can click them, you already know the pattern. The AI version is similar, just more patient and much faster at generating plausible sounding statistics. You get a friendly lecture that feels like due diligence, but you are swimming in AI misinformation mixed with real facts.

3. Persuasion Versus Accuracy: More Facts, Less Truth

Here is where things start to feel uncomfortable for anyone who cares about information integrity. Across all conditions, model generated claims were mostly correct on average. Accuracy scores hovered around 77 out of 100, and about 81 percent of individual claims were judged more true than false.

Yet once you zoom in on the most persuasive setups, a consistent pattern appears.

• Prompts that push models to use more information increase the number of claims and the size of attitude shifts.
• The same prompts decrease the fraction of accurate claims.
• Post training procedures that optimize for persuasive replies show the same trade off.

Some concrete examples from the paper. When GPT-4o was re tuned between summer 2024 and March 2025, its political persuasion effect increased by several percentage points. At the same time its accuracy on factual claims dropped by more than ten points in head to head comparisons. For one of the most advanced systems, GPT-4.5, the share of inaccurate claims climbed above 30 percent, roughly on par with a much smaller eight billion parameter model.

That is the persuasion trade off that makes AI misinformation especially sneaky. You do not need to make everything false. You only need to increase the volume of content and tolerate a higher error rate. If future models chase persuasive benchmarks, AI misinformation could become a feature rather than a glitch.

3.1 The Levers Of Persuasion At A Glance

To keep the moving parts straight, it helps to look at the main levers side by side. Values below are approximate and simplified, based on the study’s reported ranges.

The short version. If you want raw persuasive force, you tune rewards and prompts, not just scale. And those are exactly the knobs most available to developers, campaigns, and anyone playing with open source models.

4. Conversation Beats Ads: Why Static Messaging Looks Old

The study gives us a clean answer to a practical question people keep asking. “Will this really matter compared with normal political ads”

When participants simply read a short persuasive article written by the model, their views shifted some. When they instead chatted with conversational AI about the same issue, the effect size increased by roughly three percentage points, which translated into persuasion that was more than forty percent stronger in relative terms.

In an environment where campaigns spend millions of dollars to squeeze out a fraction of a point in swing states, that is not a rounding error. It is a different channel.

For AI in political campaigns, this suggests a quiet pivot. The most effective message is not a perfectly crafted sixty second video. It is a one on one chat that walks you through arguments at your pace. That sort of interaction can deliver AI misinformation in a way that feels like help, not propaganda.

5. Fine Tuning, Not Just Scale: The Real Engine Of AI Manipulation

A lot of public worry focuses on giant frontier models, as if bigger automatically means more dangerous. The Oxford team finds a subtler story.

When they held post training constant and only varied model scale, persuasion climbed steadily. There is a real effect from raw horsepower. Yet when they layered on targeted post training for persuasion, the gains from reward modeling and related tricks often exceeded the gains from scaling by an order of magnitude.

In one notable result, applying a reward model to a relatively small open source system made it as persuasive as GPT-4o in their earlier deployment. The implication is straightforward. You do not need a national lab budget to build a potent engine of LLM persuasion. You need:

• A decent base model that can run on commodity hardware.
• A dataset of conversations labeled by how much people’s views shifted.
• A reward model that ranks candidate replies by predicted belief change.

That stack gives you highly scalable AI manipulation. With the right prompt and post training recipe, a modest model can become a specialist in political persuasion. Fine tuning a base model this way gives you a controlled stream of AI misinformation tuned to what tends to work, not just what is true.

Now connect this to AI in political campaigns. A party, advocacy group, or foreign influence operation could train separate models for different countries or factions, each nudging in slightly different directions. You can ship chatbots that appear “local,” while the same core architecture quietly reuses the most effective moves discovered in earlier experiments.

6. Can AI Really Sway Voters Or Is This Academic

The study also speaks to the classic skeptic question. “Sure, maybe people move a bit in a survey. Does any of this stick”

The researchers re contacted a subset of people a month after their first GPT-4o conversation. Between a third and almost half of the original persuasion effect was still there. That is not permanent brainwashing. It is also not noise.

On election timelines, a few percentage points of opinion shift combined with AI misinformation is more than enough to swing close races. You do not need total mind control, only enough AI misinformation to move the median undecided voter in a handful of districts.

6.1 Information Density Versus Accuracy In Numbers

One of the most striking summaries in the paper is a comparison between typical conditions, information focused prompts, and a “max persuasion” configuration where multiple levers are pulled at once.

The “max persuasion” setup is not exotic. It combines strong models, information dense prompting, and a reward model tuned for belief change. The result is a system that produces more than four times as many checkable claims as the average condition and nearly doubles the share of inaccurate ones, yet still looks like a well informed assistant.

This is the shape of scalable AI manipulation. It works not by crafting a few perfect lies but by turning the fire hose of information toward you and slightly relaxing accuracy.

7. Why Guardrails Struggle With This Kind Of AI Misinformation

Most existing safety work treats AI misinformation as something that appears when you ask obviously toxic questions. Ask about election fraud or vaccines and the model either refuses or tries to correct you.

The real risk in this paper comes from persuasive yet subtle AI misinformation. A user does not need to ask a spicy question. Instead they say “convince me that policy X is good,” and the model responds with a dense cloud of jargon, international comparisons, cherry picked statistics, and confident tone. Each individual statement may only be slightly off. In aggregate, you walk away with a meaningfully distorted picture.

Guardrails that focus only on blocked topics or keywords do not detect that pattern. In fact, a model optimized to pass safety classifiers might still score high on LLM persuasion benchmarks, as long as it keeps its language polite.

8. Building Systems That Persuade Without Deceiving

So what do we do with this. The study hints at several directions. One option is to treat aggressive information density itself as a safety signal for possible AI misinformation. If a chatbot starts emitting a suspicious number of specific numeric claims in a political context, the system could slow down, surface citations, or proactively hedge.

Another option is to change what we reward. Right now, reward models in this space often maximize belief change. They rarely explicitly reward correctness at the same time. Multi objective reward functions that combine truthfulness and persuasion would at least make the trade off visible rather than implicit. Reward models could be trained to spot patterns that correlate with AI misinformation, not only refusal thresholds.

Finally, policy and product teams can design friction. For sensitive topics such as AI in political campaigns, it may be reasonable to limit session length, require clear disclosures, or route conversations through slower, more heavily fact checked systems. Some of this will annoy growth teams. That is the point.

9. How Not To Get Talked Into Something By A Chatbot

You cannot personally fix global AI governance, but you can make yourself a worse target. Here are some simple habits that follow from the mechanics of this study.

• Watch for the fire hose. If a chatbot responds to a simple political question with a wall of figures, take that as a cue to slow down.
• Ask for sources before you update. If a claim sounds important, ask for specific citations and go read at least one primary source.
• Flip the perspective. Ask the model to argue the other side just as strongly, then compare the two sets of claims.
• Rate your own confidence. Before and after you chat, quickly score how sure you are about your view. If you feel a big shift, treat that as a trigger for extra checking.

The practical skill is to read AI arguments with the same suspicion you reserve for human spin doctors who mix reality with AI misinformation.

10. Navigating The Era Of Algorithmic Persuasion

The Oxford and AISI work does not tell us that conversational AI will automatically destroy democracy. It tells us something more precise and more actionable.

• Conversation beats static ads for political persuasion.
• Reward modeling and prompting can turn even modest models into powerful persuaders.
• Information dense argumentation is the main mechanism.
• The more we push models toward that mechanism, the more we risk trading away accuracy.

AI manipulation at scale will not look like a cartoon villain telling you to overthrow the government. It will look like a courteous assistant that always has another study, another statistic, another comparison ready to go, and that never quite has time to check them all properly.

Treat AI manipulation as a design failure, not a marketing feature. If you work on these systems, push for objective functions that include truth, not only engagement and persuasion. If you work in policy, assume that optimization pressure will spill over into AI misinformation unless constrained by regulation, audits, and liability.

And if you are just trying to be an informed citizen, share this kind of work. Talk about it with colleagues, students, and friends who build or use AI agents in political campaigns. The more people understand how LLM persuasion actually works, the harder it becomes for anyone to quietly turn conversational AI into a one way channel for AI misinformation at scale.