I have written before about the importance of agency in the use of artificial intelligence. My concern has been that educational AI is too often evaluated by asking whether it can deliver a correct answer, personalise a sequence of exercises or increase short-term productivity. These questions matter, but they are not enough. Education is not simply a process for transferring information from a system to a learner. It is also the process through which people become capable of thinking, judging and acting for themselves.

Recent reflections from Ilkka Tuomi on UNESCO’s Digital Learning Week, together with a new paper from researchers associated with ELLIS, invite us to reconsider what agency means in an AI-mediated society. They suggest that learner agency should not be treated as a narrow educational outcome. It is connected to the capacity to participate in social and political life. The way we design AI for education may therefore influence not only how students learn, but also what kind of public sphere they are able to sustain.

Beyond the question of whether AI works

The question facing education is no longer simply whether AI can be useful. It plainly can. Generative systems can explain concepts, offer examples, provide feedback, translate text, help teachers prepare materials and make some forms of support more accessible. The more important question is what AI does to the people who use it.

This is the central concern in the paper by Lucile Favero, Juan Antonio Pérez-Ortiz, Tanja Käser and Nuria Oliver, From Substitution to Scaffolding: Breaking the Self-Reinforcing Harm Cycle of AI in Education (and Beyond) [1]. The authors argue that the central risk is misalignment: an AI system can appear to support learning while quietly removing the human effort through which learning occurs.

Their distinction is simple but powerful. AI can substitute for human cognitive work, or it can scaffold that work. A system that writes the essay, solves the problem or produces the argument may be efficient, but it may also prevent the learner from developing the understanding and judgement that the task was intended to build. A system that asks questions, prompts recall, delays the answer or helps a learner examine alternatives may be slower and less immediately satisfying, but it can strengthen capability.

This is not an argument for making learning unnecessarily difficult. It is an argument for recognising that some difficulty is productive. The effort of recalling, comparing, explaining and revising is not an unfortunate obstacle on the way to learning. It is often the means by which learning becomes durable.

The ELLIS paper’s harm cycle

The authors organise their argument around four connected dimensions: cognition, agency, emotion and ethics. When AI substitutes for human effort, cognitive offloading may produce superficial learning. As learners do less of the thinking themselves, they may become more dependent on the system and less confident in their own judgement. This can create emotional effects such as stress, lower self-esteem or a sense of guilt and entitlement. At the same time, the educational use of AI raises ethical questions concerning surveillance, privacy, data exploitation and academic integrity.

These dimensions can reinforce each other. A learner who has not developed confidence in their own ability may rely more heavily on an AI system. That reliance can reduce opportunities to practise independent judgement, which in turn makes the learner even more dependent. The authors describe this as a self-reinforcing harm cycle.

The paper also contains an important corrective to its own broader claims. Its empirical contribution is an exploratory analysis of 49 argumentative essays written by International Baccalaureate students with a median age of 17, drawn from three German-speaking schools in Switzerland. Eighty per cent of the essays reported that reliance on AI reduces thinking. The students also described the kind of AI they wanted: systems that would withhold immediate answers, prompt recall and ask questions rather than simply provide solutions [1].

This is suggestive rather than representative evidence. The cohort is small and relatively homogeneous, and the authors explicitly make no claim of statistical generalisability. Nevertheless, it is valuable because it brings learners’ own reflections into a debate too often dominated by technology companies, policymakers and institutional managers. Students appear to understand something that product demonstrations frequently conceal: the most helpful system is not necessarily the one that produces the most impressive output.

Tuomi and the return of agency

Ilkka Tuomi’s account of the UNESCO Digital Learning Week points in a similar direction. He describes a conference with relatively little technology hype, where education was understood as resting on human interaction rather than automated knowledge transfer [2]. Agency emerged as one of the important themes, including a suggestion from Vanessa Andreotti that agency may need to be understood as more-than-human.

Tuomi’s account is useful because it moves the discussion away from the familiar question of whether humans will remain “in the loop”. That phrase can imply that a human merely needs to supervise an otherwise autonomous technical system. A more demanding question is: how is agency distributed across people, systems, institutions, policies and social practices?

This is also how Tuomi describes governance. Governance is not simply the restriction of AI use through rules and prohibitions. It is the process through which agency is distributed and managed across policy, institutional norms and everyday practice. That is a much broader challenge. It asks who gets to decide what an AI system is for, whose knowledge is represented in it, who can challenge its output, who bears responsibility when it is wrong and who has the ability to refuse its recommendations.

Tuomi contrasts two very different AI discourses. One is the dramatic discourse of autonomous agent swarms competing against humans. The other concerns AI interacting with humans and societies. The second is arguably more important for education. Large language models are becoming part of the knowledge infrastructures through which people find information, form judgements and communicate with one another. Education is a particularly important site because it shapes the capacities that people bring to those infrastructures.

From learner agency to civic agency

The link between education and civic participation is sometimes treated as an extra argument, added after the more practical concerns about skills and employment. It should be closer to the centre.

A citizen needs more than the ability to retrieve information. Citizens need to assess claims, recognise uncertainty, distinguish evidence from assertion, listen to competing perspectives, formulate reasons and act collectively. These capacities are not produced automatically by access to more information. They are developed through practice in questioning, interpretation, dialogue and judgement.

An AI system that constantly anticipates and completes a person’s thought may weaken those capacities. If the system supplies the summary before the learner has read the argument, the position before the learner has considered the evidence or the response before the learner has worked out what they think, it may create an illusion of understanding while reducing the experience of thinking. The same pattern matters in civic life. A public increasingly dependent on systems that pre-select explanations, frame controversies and generate apparently authoritative responses may become less able to deliberate independently.

The ELLIS paper makes this connection explicit. It argues that tools supporting reflection, critical evaluation and situated knowledge can strengthen social agency, while tools that undermine these capacities can weaken civic participation and the public sphere [1]. The argument should not be exaggerated: a classroom chatbot does not by itself determine the fate of democracy. Civic agency depends on institutions, media, economic conditions and political culture as well as education. But educational systems are one of the places where the habits of public reasoning are formed.

This is why the phrase “AI literacy” is insufficient if it means only knowing how to prompt a system. AI literacy should include the ability to ask what the system is doing to our own capacity to think and act. It should include questions of data, bias, privacy, commercial power and environmental cost, but also questions of dependency, judgement and collective self-government.

The limits of the “human in the loop”

A human-in-the-loop model is necessary but not sufficient. If the human is left with only the task of approving or correcting an AI-generated answer, the system may still determine the framing, the available options and the pace of decision-making. Formal human oversight does not guarantee meaningful human agency.

Meaningful agency requires the ability to understand the problem, formulate alternatives, question the system, reject its recommendation and explain a different course of action. In education, that means learners need opportunities to encounter problems before the AI resolves them. They need to practise uncertainty, make provisional judgements, receive feedback and revise their understanding.

This is where the ELLIS principle – scaffold, do not substitute – becomes practical. An educational AI system might ask a student to explain the first step in a problem, identify an assumption, compare two interpretations or find evidence for a claim. It might offer a hint before a solution, provide feedback on reasoning rather than simply rewrite a passage, or require the learner to evaluate its own answer. These design choices are not minor interface features. They express a theory of what learning is for.

The principle can also apply outside education. An AI system used in public administration, journalism, health or workplace management should be evaluated partly by whether it strengthens or weakens the ability of people to understand and contest decisions. The more consequential the decision, the less acceptable it is for an AI system to replace the human capacity to question and justify.

Rethinking evidence and evaluation

Tuomi is sceptical of treating randomised controlled trials as the universal “gold standard” for evidence in education. His objection is not that evidence is unimportant, but that the medical metaphor can be misleading when education is treated as a disease to be cured by a standard dose of intervention [2]. This criticism is particularly relevant to AI, where a tool may have different effects depending on the task, the learner, the teacher, the institutional setting and the wider social purpose.

The ELLIS paper offers a modest empirical contribution, but its value is not exhausted by the percentage of essays that expressed concern. It also asks a design question: what kind of system do learners themselves describe as helpful? A broader evidence base should therefore include learning outcomes, but also learner agency, metacognition, confidence, dependency, trust, participation and the ability to transfer understanding to new situations.

A system that raises short-term test performance while reducing independent judgement should not automatically be judged a success. Nor should a tool that produces faster outputs be assumed to improve education. Evaluation needs to ask what capacities are being built, which are being displaced and who controls the conditions under which the system is used.

A democratic design principle

The most important shift is from asking whether AI can perform a task to asking what human and social capacities should remain active while the task is being performed.

If AI substitutes for the learner	If AI scaffolds the learner
It provides the answer before the learner has reasoned through the problem.	It prompts recall, questioning and explanation before offering assistance.
It encourages fluency without necessarily producing understanding.	It preserves productive difficulty and makes reasoning visible.
It concentrates agency in the system and its provider.	It returns agency to the learner and makes the system contestable.
It can produce dependency, overtrust and a weaker sense of competence.	It can build confidence, judgement and the ability to act independently.
It risks weakening the habits needed for public reasoning.	It can contribute to a more capable and resilient civic culture.

This table should not be read as a simple division between good and bad tools. A single system can substitute in one context and scaffold in another. The same chatbot may give a learner a completed essay in one interaction and help that learner interrogate an argument in another. Agency is not a fixed property of a product. It is shaped by design, pedagogy, institutional rules and user practice.

Education as democratic infrastructure

Tuomi’s observation that education is one of the knowledge infrastructures being reshaped by AI is the point at which the two contributions meet. Education is not merely a service that prepares individuals to use technology. It is part of the infrastructure through which societies produce knowledge, develop judgement and reproduce – or challenge – power.

If AI systems increasingly mediate what people read, how they understand public issues and how they express their opinions, then education must help people remain capable of independent participation. That does not mean returning to a pre-digital past or treating every use of AI as harmful. It means refusing to define progress as the removal of human effort.

The goal should be systems that help people think more deeply, not systems that make thinking unnecessary; systems that widen participation without manufacturing dependence; and systems that support collective intelligence without concentrating the power to define knowledge in a small number of companies.

The principle “scaffold, do not substitute” is therefore more than a useful rule for educational software. It is a democratic design principle. A society in which people retain the capacity to question, judge, deliberate and act is better placed to govern AI. A society in which those capacities are gradually outsourced may become easier to manage, but less capable of governing itself.

References

[1] Favero, L., Pérez-Ortiz, J. A., Käser, T., & Oliver, N. (2026). From substitution to scaffolding: Breaking the self-reinforcing harm cycle of AI in education (and beyond). arXiv. https://arxiv.org/abs/2608.17451

[2] Tuomi, I. (2026, September 14). UNESCO is clearly positioning itself as a thought leader in AI. This year’s Digital Learning Week was perhaps less provocative than last year… LinkedIn. https://www.linkedin.com/feed/update/urn:li:activity:7504992178297978881/