Trustworthy AI begins with human vulnerability, not technical compliance

Artificial intelligence (AI) is being rolled out across society with growing claims that ethical safeguards can make it trustworthy enough for widespread use. However, new research suggests that this approach starts in the wrong place, because trust in AI cannot be understood without first confronting the human vulnerability created by dependence on automated systems.

In "The value of vulnerability for trustworthy AI," published in AI & Society, the author argues that trustworthy AI should be defined by how well developers, deployers and regulators recognize and address the risks these systems pose to human functioning. The paper recasts trustworthiness not as a branding tool for adoption, but as a test of whether AI governance is built to protect the people most exposed to its power.

Global AI policy has embraced trust, but the study says the idea remains too vague

The paper traces how trustworthy AI became a defining slogan of AI ethics and governance. The most influential example is the European Commission's High-Level Expert Group guidelines on trustworthy AI, which presented trustworthiness as the end goal of ethical AI development and deployment. Similar lines appeared in G20 policy statements, OECD recommendations, U.S. executive orders and Chinese governance documents. Across these texts, trustworthiness became the preferred middle ground between enthusiasm for AI's economic and political benefits and concern about social, legal and ethical harms.

The study argues that this policy turn gave trustworthiness enormous visibility without settling what it really means. In these frameworks, trust is typically treated as a condition for adoption. If the public sees AI as fair, safe, explainable and accountable, then AI systems can spread more widely across society. Trustworthiness, in this view, is valuable because it supports public confidence, and public confidence helps secure AI diffusion.

The author argues that this framing creates an immediate problem. Trust is an instrumental value, not a final one. People trust because trust helps them achieve other important goods, from cooperation and care to knowledge-sharing and stable institutions. When policy documents make trustworthiness the ultimate destination, the study says, they risk building ethics on a concept that is itself only meaningful in relation to something deeper.

The paper identifies several lines of criticism that already surround trustworthy AI. One concerns whether AI systems are even the kind of entities that can be trusted in a moral sense. Human trust usually involves expectations of goodwill, understanding and responsibility. Technical systems do not possess those features in the ordinary way. Another criticism is that trustworthy AI blurs the object of moral scrutiny. It becomes unclear whether the demand is aimed at the machine, the developer, the company, the regulator or the wider sociotechnical system. A third criticism is that the language of trustworthiness can divert attention away from human accountability by focusing concern on the technology itself rather than the institutions that build, deploy and profit from it.

The study takes these objections seriously and adds another. Because trustworthy AI is often presented as a practical compromise between ethics and innovation, it can easily drift toward what critics describe as ethics washing. The language of trustworthiness can suggest seriousness while remaining abstract enough to avoid hard political conflict about power, responsibility, regulation and public resistance. In that sense, the paper suggests, trustworthiness has often been asked to do political work without being given enough moral substance.

The study does not argue that trustworthiness should be abandoned. It argues that the concept can be rescued only if it is redefined through vulnerability. Instead of asking whether AI looks trustworthy according to a checklist of values, policymakers and developers should ask what vulnerabilities push people into dependence on AI systems and what new vulnerabilities those systems create once they are in place.

Vulnerability is the missing foundation of any serious idea of trust

Vulnerability, in the study, is not treated as a niche concern limited to a few at-risk populations. It is presented as a general feature of human life. People are vulnerable because they depend on others, on institutions, and on social and material infrastructures to function, flourish and protect their interests. Trust is one way people manage that vulnerability. It allows them to enter social arrangements without constant surveillance or control.

But trust does not remove vulnerability. It creates a new form of it. To trust someone is to place one's interests in their hands and become exposed to betrayal, neglect or misuse. That is why trust has moral importance. It is not simply about prediction or reliability. It is about the expectation that another party will recognize the vulnerability involved and respond to it appropriately.

The study defines vulnerability in relation to a person's functioning, meaning the ways people live, act, move, participate and pursue their lives within social arrangements. A subject is vulnerable when something places those ways of functioning at risk. That risk can be general, because all human beings are finite and dependent, or specific, because certain groups face layered disadvantages due to poverty, discrimination, disability or marginalization.

This distinction matters because most AI policy discussions address vulnerability only in the narrow, particularistic sense. Existing frameworks tend to mention vulnerable groups, usually as populations requiring extra protection from bias, exclusion or harm. The author argues that this is too limited. It assumes that vulnerability is already there before AI enters the picture. It fails to ask how AI systems themselves may generate new vulnerabilities, even for people not previously defined as vulnerable.

That omission, the paper suggests, is now difficult to defend. AI does not merely process information in neutral ways. It increasingly shapes environments in which people work, receive services, encounter information and make decisions. Recommendation systems influence attention and affect. Automated decision tools affect benefits, policing, healthcare and employment. Search systems summarize information in ways users cannot easily avoid. These are not just tools at the edges of life. They are becoming parts of the infrastructure through which people function.

Seen this way, AI can create risks not simply by reproducing old social inequalities, but by reorganizing the conditions of dependence themselves. Users may become more exposed to manipulation, opacity, exclusion or the loss of meaningful recourse. Communities may face new forms of surveillance or automated judgment. Citizens may find that public and commercial systems increasingly mediate core decisions without clearly accountable human oversight. The study argues that these are not incidental failures. They are precisely the kinds of vulnerabilities that any serious account of trustworthy AI must bring into view.

From this perspective, trustworthiness becomes something more demanding than compliance with ethical principles. It becomes a disposition to recognize and address vulnerability well. The paper stresses that this requires more than technical accuracy or informational transparency. It requires a good-faith orientation toward those who depend on the system. In that sense, trustworthiness is tied not only to what a system does, but to how institutions understand their responsibilities to the people exposed to it.

New model of AI governance built around participation, accountability and public protection

Once vulnerability is placed at the center, the study says, the meaning of trustworthy AI changes. It is no longer best understood as a smooth compromise between innovation and ethical caution. It becomes a sociotechnical project aimed at recognizing and addressing stakeholder vulnerability before, during and after AI deployment. That shift has major implications for regulation, institutional design and the politics of AI governance.

If vulnerability is layered, contextual and often invisible from the top down, then developers and regulators cannot identify it adequately on their own. The paper argues that participatory approaches to design and governance become essential, not optional. People affected by AI systems need to be included in the processes that shape those systems, because their lived experience is part of what reveals where vulnerability lies and how it is intensified.

This makes participatory design more than a procedural add-on. In the study's account, it becomes one of the practical ways trustworthiness can be built. Involving users, communities and affected stakeholders helps close the distance between system builders and system subjects. It can expose power imbalances, identify harms early, and force institutions to confront what kinds of dependence they are creating. It also changes the politics of trust. Trust is no longer something institutions try to secure from passive publics through messaging and branding. It becomes something they have to earn by opening themselves to scrutiny, contestation and shared governance.

Digital and technological sovereignty are often discussed as questions of control over infrastructure, supply chains, platforms or strategic innovation. The author argues that a vulnerability-centered view grounds sovereignty differently. Legitimate authority over AI should not primarily be about managing the balance between innovation and rights. It should be about protecting the people whose lives and functioning are increasingly shaped by digital and AI systems. That gives governments a more direct duty: not simply to certify acceptable risk, but to recognize and address the vulnerabilities citizens face when institutions and markets become dependent on AI.

Safety, accountability, transparency and fairness remain important in the paper's view, but they should not be treated as the final architecture of trustworthy AI. They are better understood as tools whose purpose depends on a prior commitment to vulnerability. Without that commitment, the study suggests, ethical principles remain detached from the real question of what and whom AI governance is for.

Even if machine systems become more responsive, adaptive and socially embedded, the study remains skeptical that trustworthiness in the full sense can be delegated to them. Recognizing vulnerability is not merely a matter of processing information. It involves judgment, care, moral attention and a capacity to mean well. Those are features of sociotechnical systems organized by human institutions, not just properties of software. AI may support trustworthy practices, but it cannot replace the need for developers, deployers and regulators to exercise responsibility.

The study suggests that the real issue is not whether institutions can persuade the public to trust AI. It is whether AI is being built, deployed and regulated in ways that justify trust at all. That requires confronting how dependence is structured, how risks to human functioning are created, and how accountability is enforced when those risks materialize.