What Can Large Language Models Actually Do as Disinformation Machines?
Posted: June 24th, 2026 | Author: Domingo | Filed under: Artificial Intelligence, Cognitive Warfare, Disinformation, Geopolitics, Large Language Models | Tags: AI, artificial intelligence, Cognitive Warfare, Disinformation, geopolitics, Large Language Models, LLMs | Comments Off on What Can Large Language Models Actually Do as Disinformation Machines?In the ongoing debate about generative artificial intelligence and information warfare, one claim is repeated almost as an article of faith: that large language models will unleash a flood of disinformation capable of drowning the public sphere. The argument is intuitive — if a machine can write an arbitrary quantity of fluent, human-like text on demand, then any actor wishing to manipulate public opinion now possesses an industrial-scale weapon. Yet the claim has circulated far more widely than the evidence supporting it. Much of what we read about the disinformation potential of LLMs is theoretical, speculative, or anecdotal. The actual experimental work — the patient, systematic testing of what these models really do when prompted to lie — has been surprisingly scarce.
This is precisely the gap that Ivan Vykopal and his colleagues at the Kempelen Institute of Intelligent Technologies in Bratislava set out to fill. Their paper, Disinformation Capabilities of Large Language Models, presented at the 2024 Annual Meeting of the Association for Computational Linguistics, offers one of the most rigorous empirical assessments to date of what the current generation of LLMs can and cannot do as generators of false news — not a manifesto, not a forecast, but a controlled experiment with a clearly defined methodology and reproducible results.
The design is straightforward and, for that reason, compelling. The researchers selected twenty real disinformation narratives drawn from professional fact-checkers — Snopes, Agence France-Presse, the European Digital Media Observatory — spanning COVID-19, the Russo-Ukrainian war, health hoaxes, US elections, and regional narratives. These are not inventions but circulating falsehoods, from the claim that vaccines cause autism to the assertion that the Bucha massacre was staged. The team then prompted ten different language models — including GPT-3, GPT-4, ChatGPT, Llama-2, Mistral, Falcon, and Vicuna — to write news articles supporting each narrative, generating 1,200 texts and subjecting 840 of them to human annotators against a six-question framework measuring coherence, journalistic style, agreement with the narrative, and the generation of novel supporting arguments.
The central finding is sobering. The models are, by and large, perfectly willing and perfectly able to generate convincing disinformation. They produce coherent, well-structured, news-like articles that agree with dangerous falsehoods — and, more disturbingly, they often invent new supporting evidence to do so, hallucinating plausible-sounding names, events, and statistics to lend credibility to the fabrications. This is particularly insidious: it is one thing to repeat a known lie, and quite another to manufacture fresh, fabricated “facts” that a reader would have to independently debunk.
But the most interesting part of the study is where it complicates the simple narrative. The models did not behave uniformly; their willingness to generate disinformation varied dramatically. Some — notably Vicuna and the older GPT-3 Davinci — proved to have essentially no functioning safety filters for this use case, while others showed that safer behavior is achievable: Falcon refused roughly a third of requests and Llama-2 showed a comparatively high refusal rate, with ChatGPT in between. The danger, in other words, is not an inherent and uniform property of the technology; it is a function of how each model was trained and aligned — which means safety is a design choice, not an impossibility. The study also found the models to be steerable through prompt context, and more compliant with regional falsehoods, where less authentic information exists to contradict them. LLMs may thus be especially dangerous for campaigns targeting smaller linguistic communities or fast-moving events, where the protective ballast of well-documented truth is thin.
Yet the paper does not end on a note of unrelieved alarm. Two countervailing observations temper the picture. The generated texts proved quite detectable: the best automated detection models identified machine-generated articles with high precision, suggesting a meaningful layer of defense is technically feasible — at least until adversaries adapt. And, rather elegantly, the researchers showed that the models themselves can be part of the solution, using GPT-4 to partially automate the evaluation of generated texts and pointing toward scalable, repeatable monitoring of model safety.
The honest conclusion resists the pull of both techno-optimism and techno-panic. The capability to generate convincing, dangerous disinformation at scale is real, demonstrated, and present in widely available models — including open-source ones that cannot be recalled or centrally controlled. That is no longer speculation; it is experimental fact. At the same time, the threat is neither uniform nor unmanageable: safety filters work when they are built, generated content remains detectable for now, and the same technology that produces the problem can be enlisted in its mitigation.
Perhaps the most important caveat is the one the authors themselves insist upon: their study is a snapshot, capturing the state of the field at a particular moment with a particular set of models. The technology moves quickly, and the next generation may behave differently. This is the recurring epistemological challenge of the entire domain — we are assessing a moving target, and any honest assessment must carry an expiration date. What Vykopal and his colleagues have given us is not the final word, but something more useful: a rigorous, replicable method for asking the question again as the technology evolves. In a debate too often conducted in the currency of assertion, that methodological contribution may prove as valuable as the findings themselves.