Technology, Power, and the Ethics of System Design - Sustainable Catalyst

Last Updated May 9, 2026

Technology is never only technical. Every designed system distributes capacities, constraints, permissions, burdens, visibility, and risk. Interfaces guide attention. Defaults shape behavior. Recommendation systems privilege some signals over others. Databases determine what becomes legible to institutions. Automated decision systems alter how authority is exercised. Infrastructures of communication, logistics, computation, storage, and classification reshape the practical conditions of everyday life.

For that reason, Technology, Power, and the Ethics of System Design begins from a central claim: to design a system is to structure power in advance. Design determines what will be easy or difficult, visible or invisible, reversible or locked in, contestable or opaque, supported or discouraged. The ethics of system design therefore cannot be reduced to usability, innovation, optimization, compliance, or risk management after deployment. It must ask how power is embedded in technical architecture from the beginning.

Technologies are not neutral tools operating outside social order. They are sociotechnical arrangements that help organize social order itself. They shape how institutions see people, how people encounter authority, how choices are presented, how data become actionable, how rights are accessed, how burdens are shifted, and how accountability can either be preserved or obscured.

This matters because digital systems have moved from the margins of institutional life to its center. Governments rely on digital infrastructure for service delivery, identity, administration, eligibility, surveillance, emergency response, and public communication. Firms rely on data systems, algorithmic tools, and automation for pricing, hiring, recommendation, logistics, credit, marketing, surveillance, and optimization. Citizens increasingly encounter institutions through interfaces, platforms, rankings, feeds, portals, dashboards, models, and automated workflows.

The central ethical question is no longer whether technology has social consequences. It is how power is embedded in technical architecture, how that power should be governed, and whether designed systems remain answerable to the people and ecological systems they affect.

Main Library
Publications

Article Map
Stewardship & Ethics

Related Topic
Artificial Intelligence Systems

Related Topic
Institutions & Governance

Related Topic
Data Systems & Analytics

Series context: This article is part of the Stewardship & Ethics content pillar, which examines responsibility, dignity, justice, care, ecological accountability, institutional trust, intergenerational obligation, and the ethical foundations of sustainable human futures.

Editorial illustration of technology and system design as power architecture, showing layered digital and institutional structures, human figures, network connections, and handheld interfaces beneath a red padlock. — Technology is an architecture of power. System design shapes access, authority, accountability, classification, visibility, dependency, and human experience long before individual decisions are made.

This article argues that Technology, Power, and the Ethics of System Design should be understood as a study of how technical systems organize authority. It examines why system design pre-structures action, why technology is never neutral, why sociotechnical thinking is essential for serious governance, how classification systems shape legibility and control, why optimization must be judged by more than efficiency alone, why accountability must be built into systems from the beginning, how participation changes what systems are able to know, why public digital systems raise especially demanding questions of legitimacy, and why the material and environmental footprint of digital infrastructure belongs within technology ethics rather than outside it.

Why This Belongs in Stewardship & Ethics

The ethics of system design belongs in Stewardship & Ethics because designed systems now help govern the conditions of human life. They shape how people apply for benefits, access services, receive information, find work, obtain credit, navigate public institutions, communicate with one another, participate in politics, encounter law, and become visible or invisible to organizations with power over them.

Stewardship is not only the care of land, water, climate, biodiversity, and future generations. It is also the responsible governance of institutions, infrastructures, technologies, and design choices that shape human possibility. A digital system can expand access, reduce burdens, improve coordination, and support public value. It can also intensify surveillance, centralize control, automate exclusion, obscure accountability, manipulate attention, and make institutional power harder to challenge.

This means system design is a moral and governance problem before it is a deployment problem. The ethical question does not begin only after a tool causes harm. It begins when designers define objectives, choose metrics, create categories, set defaults, determine what data are collected, decide what users can see, and establish whether affected people will have explanation, recourse, and influence.

A stewardship ethic asks whether systems are worthy of the power they exercise. It asks whether they are intelligible, contestable, proportionate, accountable, participatory, ecologically responsible, and compatible with human dignity. It asks whether technical systems serve public and human purposes, or whether people are reorganized around the convenience, profit, surveillance needs, or efficiency metrics of systems.

The ethics of system design therefore belongs at the heart of responsible governance. To design a system is to help design the world people must live inside.

System Design as the Design of Power

Power in technological systems does not appear only in dramatic forms such as censorship, coercion, manipulation, or surveillance, though it can certainly take those forms. More often, power appears through design decisions that seem mundane: what data are collected, what categories are available, what counts as an error, what metrics define success, what users are allowed to know, what recourse mechanisms exist, and whose needs are assumed to be central.

These decisions determine who can act, who can contest decisions, whose experience becomes measurable, and whose harm remains structurally ignored. In this sense, system design is a way of pre-structuring the field of possible action. It is an exercise of institutional and technical power long before any single user clicks a button or any final decision is made.

This matters because ethical analysis must move upstream. It is not enough to ask whether a deployed system caused harm after the fact. One must ask how the system was imagined, scoped, classified, modeled, tested, financed, procured, governed, monitored, and justified.

System design structures power through several pathways:

permission: what actions the system allows or blocks;
visibility: what the system makes legible to institutions and users;
classification: how people, events, risks, needs, and behaviors are named and sorted;
defaults: what behavior is nudged, normalized, or made frictionless;
recourse: whether users can appeal, correct, exit, or contest decisions;
memory: what records are retained, shared, inferred, or forgotten;
dependency: whether people become reliant on systems they cannot understand or challenge.

To say that design structures action in advance is also to say that design structures politics in advance. It influences what will count as evidence, who will be visible to institutions, what claims will be easy to process, and what forms of appeal will remain possible after automated or semi-automated decisions are made.

Systems do not merely execute institutional purpose. They help define it operationally. That is why system design belongs within ethics and governance from the beginning rather than after technical work is complete.

Technology Is Never Neutral

The claim that technology is not neutral can easily become a slogan unless it is specified carefully. Technologies are not neutral in at least three senses.

First, technologies embody assumptions about users, goals, acceptable trade-offs, and normal conditions of use. Second, they are introduced into unequal social environments in which advantages and vulnerabilities are already unevenly distributed. Third, they often reshape institutions in ways that outlast the designers’ intentions.

A system created to improve efficiency may also centralize control. A platform built for openness may generate new asymmetries of visibility and dependency. An automated tool marketed as objective may reproduce the blind spots of the data, proxies, classifications, or incentives on which it depends. A public portal built to streamline service delivery may shift administrative burdens onto people with the least time, literacy, accessibility, language support, or digital access.

This matters because once neutrality is abandoned as a myth, responsibility becomes harder to evade. Designers, firms, institutions, public agencies, and regulators cannot plausibly treat technological outcomes as if they were natural events. Design choices are normative choices, even when hidden behind engineering language.

To select a metric is to select what matters. To define a label is to define what will count as reality for the system. To choose a default is to privilege one behavioral path over another. To optimize for engagement, speed, throughput, prediction, conversion, or friction reduction is to accept a particular hierarchy of goods.

Ethics therefore does not enter system design as an external moral decoration added after the real technical work is finished. It is already present in the criteria by which the work is defined. The moral question is not whether values are present in design, but which values are present, whose values they are, and whether they can be justified under conditions of unequal power.

A responsible design culture begins by making those values explicit rather than hiding them behind the language of neutrality.

Sociotechnical Systems and the Problem of Reduction

One of the most important advances in contemporary governance has been the insistence that complex digital systems must be understood as sociotechnical rather than purely technical. Risk in such systems does not arise only from code quality, model accuracy, uptime, cybersecurity, or interface design. It also arises from human behavior, institutional context, data provenance, organizational incentives, deployment conditions, oversight structures, procurement choices, and downstream patterns of use and misuse.

A technically impressive system can still be ethically defective if inserted into a setting without adequate contestability, documentation, human review, stakeholder consultation, or safeguards against predictable misuse. A model that performs well in benchmark testing may fail when deployed in a public agency with undertrained staff, biased historical data, weak appeal mechanisms, political pressure, or little ability to monitor disparate impact. A platform feature that appears benign in isolation may become harmful when combined with engagement incentives, advertising models, network effects, and weak accountability.

The sociotechnical view is ethically important because it resists reductionism. It prevents system designers from imagining that moral responsibility ends with model performance or interface polish. It also prevents institutions from treating social harms as incidental side effects outside the boundaries of design.

If risk arises from the interaction between technical components and social arrangements, then responsible design must include governance, training, monitoring, documentation, escalation pathways, procurement rules, auditability, and mechanisms of remedy.

System ethics is therefore not only about what a model or platform can do. It is about the institutional ecology into which it is placed, the forms of dependence it creates, the behaviors it incentivizes, and the distribution of costs that follow from its normal operation.

Complex systems fail not only through technical error, but through misaligned incentives, missing oversight, weak accountability, and organizational blindness. A serious ethics of system design must evaluate the whole sociotechnical arrangement, not only the artifact.

The Ethics of Legibility, Classification, and Control

Every substantial information system depends on acts of classification. People, events, objects, transactions, preferences, risks, needs, and behaviors must be named, sorted, ranked, scored, and rendered legible to the system. Yet legibility is never innocent.

Categories can simplify without understanding. They can collapse difference into administrative convenience. They can misrecognize identities, overfit to institutional priorities, or make invisible what does not fit the schema. When classification becomes embedded in widely used systems, it acquires institutional force: what is counted becomes governable, and what is not counted often becomes marginal, burdensome, or unintelligible.

This is especially consequential in public systems, financial systems, health systems, education systems, employment systems, policing systems, welfare systems, and immigration systems. A category can determine eligibility, risk, priority, visibility, suspicion, or exclusion. A score can change how a person is treated. A label can follow someone across systems. A missing category can make a need administratively impossible to recognize.

The ethical problem is not that systems classify. Classification is unavoidable. The ethical problem is that classifications can become naturalized and insulated from challenge.

Responsible system design therefore requires scrutiny not only of outputs but of ontologies: what entities exist for the system, whose experience shaped those entities, what harms follow from misclassification, and whether the categories can be revised as institutions learn.

Important design questions include:

Who created the categories?
Whose experience do they reflect?
What do they make visible?
What do they erase?
What happens when someone does not fit?
Can classifications be challenged or corrected?
Do categories become portable across systems in harmful ways?

Ethics is not satisfied by publishing principles while leaving classificatory power untouched. It requires designing systems whose internal representations of the world remain open to criticism, revision, and democratic constraint. Where classification hardens into administrative destiny, system design becomes a vehicle of control rather than a support for accountable judgment.

Efficiency, Optimization, and the Moral Shape of Systems

Modern system design is often governed by the language of optimization. Reduce friction. Increase throughput. Maximize engagement. Improve prediction. Lower cost. Scale decision-making. Streamline workflows. Automate exceptions. These goals are not trivial, and many are genuinely valuable. Systems can save time, reduce error, widen access, improve coordination, and help institutions function at necessary scale.

But optimization is never morally self-sufficient. To optimize a system, one must first decide what is worth optimizing, what trade-offs are acceptable, and whose losses can be treated as tolerable.

A queue can be made faster by reducing opportunities for explanation. A moderation system can be made more efficient by increasing false positives against marginal speech. A hiring model can increase consistency while hardening past exclusions. A welfare system can reduce administrative burden for the state while increasing it for vulnerable claimants. A platform can optimize engagement while degrading attention, trust, and public discourse.

This matters because ethics enters precisely where optimization threatens to become blind to the values it displaces.

Good system design therefore requires a plural vocabulary of evaluation. Accuracy, speed, scale, and cost matter, but so do dignity, recourse, proportionality, accessibility, interpretability, autonomy, ecological burden, democratic accountability, and institutional trust.

The point is not to halt technical progress. It is to insist that progress cannot be defined by efficiency metrics alone.

A system may be technically sophisticated and still be ethically crude if it systematically discounts human vulnerability, asymmetry of power, ecological cost, or long-term institutional damage. Optimization without moral remainder is one of the most persistent illusions in modern technological culture.

System ethics begins by refusing that illusion.

Accountability by Design

One of the clearest lessons of recent technology governance is that accountability cannot be improvised after deployment. Responsible design requires not only technical safeguards but institutional arrangements that make it possible to explain, audit, challenge, and correct system behavior. Accountability must therefore be built across the full system life cycle rather than added as a late-stage compliance exercise.

Accountability by design means, at minimum, that a system should have traceable ownership, clear decision rights, defined review processes, and documented assumptions. It should be possible to know who selected the objectives, who approved the model or workflow, what data were used, what harms were anticipated, what testing occurred, and what channels exist when affected persons are harmed.

Without such structures, ethical language becomes ceremonial. The system may be described as trustworthy, fair, responsible, or human-centered while remaining practically unanswerable to those who live under its effects.

Accountability by design requires several practical commitments:

traceability: decisions, data sources, model versions, design changes, and approvals are documented;
ownership: responsibility is assigned to identifiable actors rather than diffused across vendors, agencies, teams, or automated systems;
auditability: independent reviewers can inspect system behavior, assumptions, and outcomes;
contestability: affected people can challenge decisions, classifications, errors, and harmful outputs;
remedy: institutions can correct harms, not merely acknowledge them;
monitoring: system performance is evaluated after deployment, including disparate impacts and failure patterns;
withdrawal capacity: institutions can pause, revise, or retire systems when harms exceed acceptable limits.

Power without answerability is not simply a governance gap. It is an ethical failure built into the architecture of the system. Auditability, documentation, oversight, and remedy are not peripheral technical extras. They are among the practical forms through which ethical seriousness becomes visible.

Participation and the Problem of Excluded Voices

A persistent failure in technology design is the exclusion of those most affected by systems from meaningful participation in their design and governance. When designers, vendors, executives, engineers, product managers, institutional purchasers, and regulators dominate the definition of success, systems often reflect the worldview of those already closest to power.

Stakeholder participation is therefore not a public-relations add-on. It is an epistemic and ethical necessity.

Affected communities often understand harms, dependencies, and failure modes that are invisible from the perspective of system builders. They may see burdens normalized by institutional process, cultural assumptions embedded in labels and interfaces, accessibility barriers hidden in workflow design, or practical obstacles that never appear in technical specifications.

A digital public-service portal may appear usable to designers but exclude people with unstable internet access, disabilities, limited English proficiency, low literacy, older devices, housing insecurity, or fear of surveillance. An automated decision system may appear consistent to administrators while reproducing the conditions that affected people experience as arbitrary, humiliating, or impossible to challenge. A platform may appear efficient to firms while reorganizing workers’ lives around opaque ratings, hidden allocation rules, or algorithmic discipline.

An ethical system design process therefore needs mechanisms for incorporating situated knowledge into requirements, testing, review, and revision. Participation should not begin after deployment. It should shape problem definition, design priorities, risk assessment, usability testing, governance rules, and evaluation metrics.

Otherwise, the system risks reproducing the classic pattern in which those who bear the consequences of power are the least able to shape it.

Participation, properly understood, is not a substitute for institutional responsibility. It is one of the conditions under which institutions become less blind to the consequences of their own design decisions.

Public Institutions, Infrastructure, and Democratic Legitimacy

The ethics of system design becomes especially acute when systems mediate public power. Digital identity systems, eligibility systems, predictive tools, case-management systems, content-moderation rules, data-sharing architectures, automated notices, risk-scoring tools, and public-sector AI applications do not merely support administration. They shape how people experience the state, how rights are accessed, how burdens are distributed, and how accountability is perceived.

In public systems, bad design can undermine not only service quality but democratic legitimacy. When opacity prevents contestation, when interfaces shift administrative burdens onto citizens, when interoperability expands surveillance without corresponding safeguards, or when automated scoring reshapes eligibility without explanation, the issue is no longer just product quality. It is the proper exercise of public authority.

Public digital systems raise especially serious concerns because the people subject to them often cannot simply opt out. A person may need the system to access benefits, housing, healthcare, licensing, school services, legal status, public records, or emergency support. If the system is opaque, inaccessible, discriminatory, or unreviewable, design failure becomes civic harm.

The ethics of system design in such settings must therefore include legality, proportionality, transparency, remedy, accessibility, public justification, and democratic oversight. Technologies that mediate state action must be held to a higher standard because they operate where design and governance directly meet citizenship and rights.

Public digital systems are not just service tools. They are institutional environments in which people encounter authority itself.

That is why public-sector system design should be governed as a matter of democratic legitimacy, not merely procurement efficiency.

Interfaces, Defaults, and the Ethics of Behavioral Architecture

System design also shapes behavior through interfaces and defaults. The placement of a button, the wording of a notice, the ordering of choices, the friction required to opt out, the visibility of privacy settings, the structure of a consent form, the design of an appeal path, and the timing of a notification all affect what people are likely to do.

This is sometimes treated as a usability issue. It is also an ethical issue.

Behavioral architecture can help people navigate complex systems, reduce confusion, prevent error, and make rights easier to exercise. But it can also manipulate, exhaust, confuse, obscure, or steer people toward choices that serve institutional interests more than human wellbeing.

Dark patterns are only the most visible version of this problem. More subtle forms appear when systems make beneficial actions difficult, hide important information behind confusing navigation, require repeated confirmation for protective choices, or design cancellation, appeal, privacy, and complaint pathways to be technically available but practically burdensome.

Public institutions face their own version of this challenge. A benefits portal may be formally open but designed in ways that discourage completion. An appeals process may exist but be buried under confusing language. A consent form may be legally sufficient but unintelligible to the person signing it. A digital identity system may promise convenience while making non-digital alternatives difficult or stigmatized.

The ethical question is whether interface design supports human agency or exploits human limitation.

A responsible system should make rights, protections, explanations, and recourse easy to find and use. It should not rely on confusion, inertia, pressure, fatigue, or informational asymmetry. Where systems shape behavior through architecture, they must be judged by whether that architecture respects persons as agents rather than treating them as behavioral targets.

Data Infrastructure and the Politics of Visibility

Data systems do not merely store information. They produce institutional visibility. They determine what can be tracked, linked, compared, audited, predicted, remembered, ignored, or forgotten. In doing so, they shape power.

A person becomes visible to an institution through records, identifiers, categories, credentials, transactions, locations, applications, histories, scores, documents, metadata, and inferred patterns. Visibility can be protective when it enables access to services, recognition of need, enforcement of rights, or correction of harm. But visibility can also become surveillance, exposure, discipline, or control.

In many systems, the same data infrastructure can support care and coercion. A database that helps deliver benefits can also be used to police eligibility. A health record can improve care while raising privacy risks. A mobility dataset can improve planning while exposing vulnerable groups. A school data system can support learning while intensifying ranking, sorting, and punishment. A smart-city platform can optimize infrastructure while expanding surveillance.

This ambiguity is why data infrastructure must be governed ethically from the beginning. Questions of collection, retention, linkage, access, consent, deletion, secondary use, data quality, and institutional sharing are not technical afterthoughts. They define the conditions under which visibility becomes protection or domination.

A serious ethics of system design asks:

What data are collected, and why?
What data should not be collected?
Who can access the data?
Can data be reused for purposes beyond the original context?
Who benefits from visibility?
Who is exposed by it?
Can people correct, contest, delete, or understand records about them?
How does data infrastructure change the relationship between people and institutions?

Data systems are political infrastructures. They should be designed with the same seriousness that governs other forms of public and institutional power.

Environmental Costs and the Materiality of Digital Systems

It is increasingly difficult to write about technology ethics as though digital systems were immaterial. The mythology of digital innovation often obscures its material substrate: data centers, electricity demand, water use, cooling systems, semiconductor supply chains, rare minerals, manufacturing, logistics networks, e-waste, and the labor systems that maintain digital infrastructure.

System design is never purely informational. It is also industrial and ecological.

This matters because ethically serious design must ask not only whether a system is accurate, usable, scalable, or profitable, but what forms of extraction, energy consumption, infrastructure dependence, and environmental burden its operation presupposes. A design process that celebrates seamless convenience while ignoring material throughput remains ethically incomplete.

The responsibility of designers and institutions is not merely to reduce harm at the user interface. It is also to confront the wider ecological footprint of the infrastructures they normalize and scale.

This becomes especially important as AI, cloud computing, streaming platforms, real-time analytics, digital twins, blockchain systems, high-frequency advertising infrastructure, and large-scale data storage expand demand for computation, energy, water, hardware, and minerals. Digital systems can support sustainability by improving monitoring, coordination, modeling, and efficiency. But they can also intensify resource demand and obscure environmental costs behind the language of innovation.

A more adequate ethics of system design should ask:

Is the system computationally proportionate to the public or human value it provides?
What energy and water demands does it create?
What hardware and mineral supply chains does it depend on?
What waste streams does it generate?
Who bears the environmental burdens of its infrastructure?
Can the same purpose be achieved with simpler, lower-impact design?

This widens the ethics of system design from narrow human-centered design toward a more adequate account of technological systems as embedded in social and ecological systems alike. Digital power is always grounded in material systems somewhere. Ethical analysis becomes more honest when it treats that materiality as part of the design problem rather than as an externality.

Toward an Ethics of System Design

An adequate ethics of system design begins from the recognition that design is a mode of governance. It asks who defines system objectives, whose interests are privileged, what harms are anticipated, how uncertainty is handled, what recourse exists, what kinds of dependence are being created, and how institutions remain answerable once the system is live.

It treats documentation, review, human oversight, stakeholder participation, contestability, and remedy not as bureaucratic burdens but as the practical forms through which ethical seriousness becomes visible.

This ethic is neither anti-technology nor naively celebratory. It does not deny the real benefits of complex systems, nor does it treat all system design as domination. Rather, it insists that technical capability and ethical legitimacy are different achievements.

A system can work and still be wrong. It can scale and still corrode trust. It can predict and still misrecognize. It can reduce friction and still deepen dependency. It can automate a process and still make authority harder to challenge.

A serious ethics of system design should include several commitments:

Purpose scrutiny: the objective of the system should be ethically justified, not merely technically feasible.
Power analysis: designers should identify who gains capacity, who loses agency, and who bears risk.
Classification review: categories, labels, scores, and data structures should be contestable and revisable.
Accountability by design: documentation, auditability, review, and remedy should be built into system architecture.
Participation: affected communities should help define risks, requirements, evaluation criteria, and governance practices.
Accessibility and dignity: systems should preserve human standing, intelligibility, and practical ability to act.
Ecological responsibility: digital systems should be evaluated for material and environmental costs, not only user-facing effects.
Lifecycle governance: ethical responsibility continues through procurement, deployment, monitoring, revision, and retirement.

To design well is not only to make a system function. It is to make it governable, intelligible, contestable, proportionate, accountable, and justifiable within the human, institutional, and ecological worlds it helps shape.

That is the point at which system design becomes not only a technical discipline, but a serious practice of public and organizational responsibility.

System Design Diagnostic Table

Design question	Thin technical frame	Stewardship & Ethics frame
What is technology?	A tool for achieving user, business, or institutional goals.	A sociotechnical arrangement that structures power, action, visibility, dependency, and accountability.
What is system design?	The design of workflows, interfaces, architecture, performance, and functionality.	The pre-structuring of human action, institutional authority, classification, recourse, and social possibility.
What is neutrality?	The assumption that tools are neutral unless misused.	A myth that obscures embedded values, incentives, categories, defaults, and power relations.
What is optimization?	Improving speed, scale, accuracy, cost, engagement, or throughput.	A moral choice about what matters, whose burdens count, and what values may be displaced.
What is classification?	A technical necessity for organizing data and decisions.	A form of institutional power that determines who and what becomes legible, governable, or invisible.
What is accountability?	Documentation, compliance, or post-hoc reporting.	Traceable responsibility, auditability, contestability, remedy, and the ability to pause or revise systems.
What is participation?	User feedback, stakeholder consultation, or usability testing.	Meaningful influence by affected communities over problem definition, design, risk assessment, and governance.
What is public-sector design?	Digital service delivery and administrative modernization.	The technical mediation of public authority, rights, citizenship, eligibility, and democratic legitimacy.
What is data infrastructure?	Storage, linkage, analytics, and interoperability.	A politics of visibility that can enable care, surveillance, recognition, exclusion, or control.
What is environmental responsibility?	A secondary concern outside core digital design.	A core ethical dimension of computation, hardware, energy use, water demand, minerals, and e-waste.

Conclusion: Technology, Power, and the Ethics of System Design

Technology design is an ethical activity because it is an activity of ordering the world. Systems decide what will count, what will be measured, how choices will be presented, when intervention will occur, how authority will be exercised, and whether affected people can understand or challenge what is being done to them.

That is why technology cannot be separated from power. A system may appear technical at the surface while silently organizing access, visibility, dependence, classification, recourse, and institutional authority underneath. The ethical task is not to pretend that power can be eliminated from technical systems. It is to ensure that power is constrained, justified, reviewable, accountable, participatory, proportionate, and aligned with human dignity, institutional legitimacy, democratic governance, and environmental responsibility.

The enduring challenge is therefore not simply to build more advanced systems, but to build systems worthy of the authority they increasingly exercise. That requires more than innovation. It requires judgment, restraint, participation, institutional design, ecological awareness, and moral clarity about what technology is doing when it reorganizes the conditions under which human beings live, decide, work, learn, communicate, and govern.

A system can be efficient and still be unjust. It can be scalable and still be degrading. It can be intelligent and still be unwise. It can be convenient and still be extractive. It can be technically impressive and still be ethically unworthy.

The ethics of system design asks whether technology expands human agency or narrows it, clarifies responsibility or obscures it, supports public goods or intensifies private control, reduces burdens or relocates them onto the vulnerable, and remains answerable to the people and ecological systems its operation affects.

Design is never only about what a system does.

It is about what kind of power the system builds into the world — and what kind of world that power makes normal.

References

Bowker, G.C. and Star, S.L. (1999) Sorting Things Out: Classification and Its Consequences. Cambridge, MA: MIT Press.
Friedman, B. and Hendry, D.G. (2019) Value Sensitive Design: Shaping Technology with Moral Imagination. Cambridge, MA: MIT Press.
National Institute of Standards and Technology (NIST) (2023) Artificial Intelligence Risk Management Framework (AI RMF 1.0). Gaithersburg, MD: NIST. Available at: https://nvlpubs.nist.gov/nistpubs/ai/nist.ai.100-1.pdf
National Institute of Standards and Technology (NIST) (n.d.) AI RMF Playbook. Available at: https://airc.nist.gov/airmf-resources/playbook/
National Institute of Standards and Technology (NIST) (n.d.) AI Risk Management Framework. Available at: https://airc.nist.gov/airmf-resources/airmf/
Organisation for Economic Co-operation and Development (OECD) (2024) Advancing Accountability in AI. Paris: OECD. Available at: https://www.oecd.org/en/publications/advancing-accountability-in-ai_2448f04b-en.html
Organisation for Economic Co-operation and Development (OECD) (2024) The OECD Digital Government Policy Framework. Paris: OECD. Available at: https://www.oecd.org/en/publications/the-oecd-digital-government-policy-framework_f64fed2a-en.html
Organisation for Economic Co-operation and Development (OECD) (n.d.) Technology Governance. Available at: https://www.oecd.org/en/topics/technology-governance.html
UNESCO (2021) Recommendation on the Ethics of Artificial Intelligence. Paris: UNESCO. Available at: https://unesdoc.unesco.org/ark:/48223/pf0000380455
UNESCO (n.d.) Ethics of Artificial Intelligence. Available at: https://www.unesco.org/en/artificial-intelligence/recommendation-ethics
Winner, L. (1980) ‘Do artifacts have politics?’, Daedalus, 109(1), pp. 121–136. Available at: https://www.jstor.org/stable/20024652