The battle for truth is changing. Are we ready?
As AI turbocharges online influence, the real threat may not be fake news itself – but how digital persuasion can push some groups from outrage to real-world action.
For decades, influence campaigns relied on a familiar playbook: propaganda, misinformation and attempts to shape public opinion. Those tactics remain, but artificial intelligence (AI) is making them faster, cheaper and harder to detect.
Generative AI can produce vast amounts of content, tailor messages to different audiences and support automated accounts that interact in seemingly human ways. This gives influence campaigns unprecedented scale, speed and coordination. In Venezuela, for instance, pro-government messages were distributed through videos featuring AI generated news anchors – creating the appearance of independent international news coverage in what was synthetic content.
But while much of the public conversation focuses on fake news and AI-generated content, I believe we are asking the wrong question.
The real challenge is not simply whether people encounter misleading information. It is understanding how, when and why online influence sometimes leads people to act.
This is one of the defining security challenges facing modern democracies.
Governments must protect free speech, legitimate political participation and social cohesion while responding to foreign interference, coordinated manipulation and politically motivated violence.
Today, these challenges are becoming more difficult as trust in institutions declines, social media rewards outrage over nuance and existing divisions provide fertile ground for manipulation. Yet responses often remain reactive, arriving after damage has been done.
I’m leading some work that is researching a different approach: combining AI with decades of social and behavioural research to understand how influence spreads through groups and, in some circumstances, becomes real-world mobilisation.
AI changes the scale, not the story
Influence itself is not new and it can also be profoundly good – responding to others is the glue that makes human societies work together. It’s who and how it’s exploited that can be of issue.
You might like
Humans have always influenced one another. Political movements, advertising, activism and public debate all rely on persuasion. AI has not rewritten the basic rules of influence. What it has changed is the scale, speed and coordination with which influence can occur.
Technology has also given researchers access to huge amounts of information, making it possible to spot patterns in the way people use language and track how those patterns change over time. But data alone cannot explain why an influence campaign succeeds with one group and fails with another.
People are not passive recipients of information. We interpret messages through our identities, experiences, relationships and group memberships.
The same content can therefore have very different effects depending on who encounters it, the group they identify with and the social context in which it appears.
This is why prediction must not be confused with explanation. AI can find patterns across millions of pieces of information, but those patterns still need to be interpreted using knowledge of human behaviour – this is where the work we are doing is critical. The strongest approach combines the scale of data science with theories that explain how people think and act.
Influence does not automatically lead to action
Focusing exclusively on misinformation can be misleading because exposure is not the same as mobilisation.
Online influence is widespread, but offline action is comparatively rare. Polarisation does not necessarily lead to violence. Holding radical ideas does not necessarily mean someone will act on them. Influence and mobilisation are not interchangeable.
The more important question is how people move from consuming or sharing content online to acting together offline.
To answer that question, my colleagues and I study several connected but distinct processes, including politicisation, polarisation, radicalisation and activation. These are often treated as though they describe the same phenomenon, but they can represent different pathways of political and social engagement.
Understanding those pathways can help explain why influence campaigns often fail and why, in other circumstances, they contribute to serious real-world consequences.
Our research shifts the focus away from a search for inherently “dangerous individuals” and towards the wider group dynamics that can shape behaviour – and societal consequences.
When we talk about declining trust, we are often describing one group losing trust in another. When we talk about polarisation, we are describing groups moving further apart in their views. Those divisions can then be identified and exploited by actors seeking to deepen existing social fault lines.
Collective action occurs when people act as members of a group to advance that group’s position. It includes many forms of democratic participation, from peaceful protest and community advocacy to campaigning for political or social change. In Australia, outside of the three-year voting cycle, collective action is a key way that everyday people interact with political systems that they are a part of.
Political engagement and protest should not, by themselves, be treated as threats. Peaceful collective action is essential to democratic life. Democracies depend on people being able to express grievances, advocate for change and participate in public debate.
Protest is like a valve – when it is closed off because conventional forms of activism become criminalised, people do not stop caring. Research from our lab shows that criminalising protest can actually be radicalising for some.
The challenge is identifying when authentic participation is being manipulated, artificially amplified or channelled towards harm.
Watching polarisation develop
One strand of our research examines what happens when opposing groups respond to each other over time.
In a study of debate surrounding the Voice to Parliament, we analysed social media posts published over 15 months by prominent supporters and opponents.
Language-processing tools were used to examine anger, negativity, hate and offensiveness, while statistical analysis tracked whether changes in one group’s language predicted changes in the other group’s language the following week.
The results showed that polarisation was not a simple or inevitable two-way spiral. Changes in hostile language among supporters predicted increased hostility among opponents in the following week, but the reverse pattern was not found. In other words, the response was uneven: one group appeared more reactive to the language of the other.
This matters because it shows that division does not grow uniformly across every group or situation. The pathway towards greater hostility is conditional. Understanding which groups are reacting, what they are reacting to and how those reactions change over time may provide a more useful picture than simply measuring the total amount of hostile content.
Detecting change, not just content
This is where the next frontier of research lies.
At Flinders, we are exploring how AI, language models and other computational models can be combined with social psychology to map the role of emotions in confrontational action, identify language linked with growing commitment to political violence and understand how language and movement networks shape one another.
The aim is not to monitor opinions, label lawful dissent as dangerous or suppress legitimate political expression. It is to understand changes in group behaviour, particularly the pathways and possible tipping points that may distinguish widespread online discussion from coordinated manipulation or movement towards harm.
Rather than only counting posts or identifying false claims, this work examines how narratives travel through groups, how people define who “we” are, how perceived threats are discussed and how these patterns change over time.
This could give analysts a more informed basis for identifying emerging risks, while reducing the danger of overreacting to genuine democratic participation.
Democracy’s next challenge
As information environments become more complex, democracies will need more sophisticated responses than removing content or debunking false claims after the event.
The future of democratic resilience depends on understanding not just what information people see, but how identities, emotions, relationships and group dynamics shape what happens next.
AI is transforming the information landscape, but the human psychology underpinning influence remains remarkably consistent. Meeting the challenge therefore requires both strong methods that can detect patterns and strong theories that can explain them.
If we want to protect trust, strengthen social cohesion and safeguard democratic institutions, we need to understand technology and people together.
Professor Emma Thomas is a social psychologist, Matthew Flinders Professor and Australian Research Council Future Fellow at Flinders University. She is also the Deputy Director of the Jeff Bleich Centre for Democracy and Disruptive Technology and a Member of the Flinders Institute for Mental Health and Wellbeing. Prof. Thomas also leads the Flinders Social Influence and Social Change Lab, researching the psychological processes that shape political engagement, collective action and extremism.
Find out more about Flinders Research That Matters. Changing 10 million lives, here.
Want to see more stories from InDaily SA in your Google search results?
- Click here to set InDaily SA as a preferred source.
- Tick the box next to "InDaily SA". That's it.
