Cut Through!
Cut Through!
The battle for reality
0:00
-16:20

The battle for reality

We need to treat our information environment as critical national infrastructure

“Germany calling. This is Germany calling.”

Throughout the Second World War those words echoed through the static of Britain’s airwaves. The clipped tones of William Joyce, better known as Lord Haw-Haw, were broadcast daily from Nazi Germany with the aim of weakening Allied morale.

The source of those messages was obvious, barely even hidden: Joseph Goebbels and his Reich Ministry for Public Enlightenment and Propaganda.

But in a conflict today, would we spot enemy influence so easily? What does disinformation propagated by a foreign state look like in a world of algorithmic recommendations and AI mediation? And as Europe boosts its defence budgets, should we also be spending more on protecting our information environment?

All wars are a behaviour change campaign

Some government departments can be a bit sniffy about communications. They see it as a downstream activity, something a bit beneath the ‘real’ work of policy. More than once I heard: “We can’t have the comms tail wagging the policy dog.”

But one department was different: The Ministry of Defence. For senior military officers, communications wasn’t an afterthought but a critical part of their capabilities.

One senior military commander told me:

“All wars are behaviour change campaigns.”

I didn’t get what he meant at first, but he explained:

“People think you win a war by killing more people than the other side. You don’t. You win a war when the other side no longer wants to continue. War is the art of changing the capability and motivation of your opponent to fight.”

Russian and Chinese military strategists write openly about “cognitive warfare”. They see the minds of their opponents as a battlefield, a military domain to be contested alongside land, sea, air, or space.

This goes beyond spreading propaganda. Cognitive warfare is about eroding public trust, diminishing morale, and deepening societal divisions without firing a single shot. The Kremlin’s strategy does not rest on physical military capabilities alone. It measures its power in its ability to cause us to see the world as Moscow wishes us to see it, and to make decisions in a Kremlin-generated perception of reality.

The idea of cognitive warfare can sound a little like a conspiracy theory in most western discourse. But this isn’t a theoretical practice, it is one which has been utilised consistently since the Second World War.

During the Cold War, Soviet influence operations (known as “active measures”) mixed covert propaganda with political manipulation. KGB agents forged documents and letters, planted press stories, spread rumours and disinformation, made clandestine radio broadcasts, and supported front organisations and local political groups. The aim was to persuade, but also to weaken trust in Western governments, alliances and institutions. One of the most notorious examples was Operation Denver which was a successful attempt to spread the idea that HIV/AIDS had been created by the United States in a biological weapons lab.

As our information environment changed in the early noughties, so did the methodologies pursued by the Russian state. By the 2010s, thousands of people were being employed to create fake social media accounts at troll farms such as the Internet Research Agency, with the aim of promoting the Kremlin’s interests, sowing division and influencing Western elections. As during the Cold War, methods were designed to make hostile or divisive narratives appear as if they came from local voices rather than Moscow.

So, the idea of foreign states wanting to shape our information environment, is not just a theoretical construct, there are plenty of examples of Russia actively attempting to do so.

That intent is unlikely to change. But the technology we use to access information is changing dramatically. That makes foreign influence harder to detect which in turn increases the threat. The perfect influence operation is one in which you don’t even realise you have been influenced.

Where could information be weaponised in future?

There has been plenty of coverage of how AI deepfake content is corroding our information environment. AI is a boon to both the efficiency and credibility of bad actors. The ability of Russian troll farms to wield influence has been put on steroids.

But I want to focus on a less obvious risk.

Today’s information battlegrounds may be deep in the systems that decide what we see and believe - in the black boxes of recommendation algorithms and AI answer engines.

A newspaper editor who was a KGB agent could shape one front page a day. The ability to influence a modern recommendation algorithm could shape millions of individual information environments simultaneously.

A recent academic study examined TikTok content recommendations during the 2024 US presidential election campaign. Researchers created hundreds of accounts that they seeded with different political preferences - some Democrat, some Republican. What they found was a systematic difference in the 280,000 pieces of content that users were shown. Republican-seeded accounts received significantly more material that reinforced their political preference, while Democrat-seeded accounts were shown more anti-Democrat content.

The authors do not suggest deliberate manipulation by TikTok. The reasons for the differences are unclear. But what the study shows is that platforms like TikTok have the potential and the capacity to shape information exposure differently for different groups at scale.

In traditional media, editors wielded enormous influence but audiences retained a lot of agency. People chose what to read or watch next. They could decide whether to listen to Lord Haw-Haw or not. That is subtly different in systems where algorithms decide what content you see next. It becomes difficult for users to easily distinguish between their own preferences and the preferences the system is gradually reinforcing.

A new paper has highlighted a different risk - that state-controlled media might influence AI models themselves. Researchers found that Chinese state-scripted news appeared in one AI training dataset 41 times more often than Chinese-language Wikipedia content. It is not that AI systems have become direct instruments of Chinese propaganda. It is that if the data AI models learn from have been distorted by state-controlled material, those patterns can become embedded in the models themselves.

The paper makes an important observation. The state influence becomes hidden or “laundered” into the machine-generated text that users are likely to see as trustworthy and independent. If you were listening to Lord Haw-Haw you knew the Reich Ministry was biased, but you might believe an AI response to be neutral.

Not every skewed feed is foreign interference nor is every AI answer propaganda. But I do think these example illustrate the potential for models and platforms to become the focus of modern day influence operations.

In an early episode of their podcast The Rest is Classified, security correspondent Gordon Corera and former CIA analyst David McCloskey, considered ways in which a member of China’s Ministry of State Security might think about TikTok. Would it be possible to get the TikTok algorithm to suppress topics negative to China? Which audiences might be most susceptible to content that would undermine UK policy on China?

Just as every company and campaign group is currently thinking about how to influence what AI engines say about them, why wouldn't China’s security services also be thinking about how they might influence what AI summaries say about Taiwan?

We shouldn't wait for evidence of this happening to start thinking about how we might guard against it. Good national security planning is precisely about asking how systems might be exploited or weaponised before a crisis occurs.

Conclusion

In the Second World War, hostile influence had a voice, a transmitter and a known sponsor. In the Cold War, it resided in front organisations and influence operations. In the social media age, it was driven by fake accounts and bots peddling outrage and division. In the age of algorithms and AI, it may increasingly be located deep within the systems and machines that decide what we see at all.

That means treating our information sphere as a piece of critical national infrastructure. There is no point protecting the undersea cables and 5G networks that transmit information, if the content they are carrying undermines our national security.

We ban foreign money from election campaigns because we want the governance of our nation to be determined solely by the citizens who have a genuine stake in its future. We should accept no lesser standard for the platforms and AI systems shaping how citizens understand the world.

That is not a call for censorship, but for treating algorithmic transparency, the sources shaping AI answers and the quality of training data as national security issues.

Europe is now spending more on tanks, missiles, drones and cyber defence. It should. But defence is not only about protecting territory. It is also about protecting the conditions in which free societies make free choices.

That requires proper debates about sovereign AI and how to preserve high quality public service broadcasting we can trust. Both are expensive, but not having them could be much more costly.

Lord Haw-Haw was easy to recognise. The next voice calling may not sound like propaganda at all. It may sound like a trusted recommendation from a friend.

In the conflict of the future, the first casualty may be our ability to decide what is true.

Leave a comment

Share

Thanks for reading Cut Through! Subscribe for free to receive new posts and support my work.

Discussion about this episode

User's avatar

Ready for more?