Divya Siddarth is the co-founder of the Collective Intelligence Project, a political economist and social technologist at Microsoft’s Office of the CTO, a research director at Metagov and the RadicalXChange Foundation, a research associate at the Ethics in AI Institute at the University of Oxford and a visiting fellow at the Ostrom Workshop.
Kelsie Nabben is a researcher at RMIT University of the social outcomes of decentralized technologies in the Blockchain Innovation Hub and Centre for Automated Decision Making & Society. She is also a regular contributor to “Metagov” research group and a board director of Blockchain Australia.
A specter is haunting our technological imaginaries: the specter of autonomy. The problem is that we can’t quite decide from what or for whom.
Two movements with transformational aspirations are dominating the tech world today: the first around blockchain and the second around AI. These are animated by two very different concepts of autonomy. AI promises to usher in an era of peace and plenty for humanity that frees us from the burden of labor and need, provided we play our cards right to avoid the risk of existential catastrophe. Meanwhile, blockchain provides the infrastructure for, we are told, the unparalleled exercise of both individual and collective agency, opening the doors to automation, self-governance and global coordination.
At first, these concepts may seem at odds. The AI worldview prizes an autonomy of outcome with minimal active decision-making input, while its crypto counterpart desires an autonomy of process, with outcomes contingent on individual will and participation throughout the design of technological infrastructures. While the blockchain community holds decentralization as a core normative value, AI often pushes toward centralization as necessary for scale and safety.
But as much as they differ, they both share an underlying desire: to implicitly or explicitly abstract away human fallibility in service of a fully automated vision of perfection. In this, they risk falling prey to the mistaken belief that automation can achieve any meaningful autonomy at all.
In an age of human-led mass extinction and climate catastrophe, the dream of a technological higher power saving us from ourselves is tempting. In an era of coordination failures and corruption at the highest levels, the aspiration toward decentralized coordination between empowered individuals is similarly so. Combining these collective visions of technological and scientific progress paints a seductive vision of freedom, both from the constraints of the natural human self and the constraints of emergent human society.
Yet the aim to transcend that which makes us human is not the path to serving humanity.
Autonomy is necessarily circumscribed by community and society. Meaningful independence only exists through interdependence, one that is impossible in the worlds these technological visions imagine.
Freedom From Need
The aspiration toward both individual and collective autonomy has driven political thinkers and political systems for centuries. Individual autonomy is touted by philosophers across the Western political spectrum. It is a central value of liberal democracy, a fundamental tenet in Kantian moral philosophy, a foundational principle of Mill’s utilitarianism and core to Nietzsche’s concept of the “free spirit.”
Autonomy has also been a rallying cry of social movements the world over, particularly of Indigenous and anti-colonial movements, standing for everything from national self-sovereignty to land reclamation. Respect for various manifestations of autonomy — bodily autonomy, decision-making autonomy, autonomy through privacy — is enshrined in the law in various ways. However, the link between individual autonomy and the collective good has always been pernicious — and nowhere more so than in the visions of technology that promise to give us both in unparalleled measure.
In the world that proponents of AI envision, machines develop autonomy — meaning that they become self-governing, self-defining, self-learning and self-directed. In this world, we will build artificial intelligences that approximate and then surpass human intelligence, known as artificial general intelligence, or AGI. Eventually, they will be able to independently improve themselves to the extent that there is an exponential takeoff in machine-based intelligent capacity.
At this point, two things can happen. The first is utopia: we have done a good enough job “aligning” these intelligences with human values (including the sanctity of human life) such that these superintelligent, autonomous systems will be able to fulfill our needs and desires, make better decisions than we can across every sphere of life, usher in a post-scarcity future and potentially support immortality in the cyber realm. In the hopes of transhumanists, we may literally achieve autonomy from our bodies by foregoing our “wet-ware” to integrate with computers and free our minds.
The second possible future is dystopia: in which these superintelligences — which could manifest as digital computers, a network of computers or through another substrate — are insufficiently aligned with human values, and there is a mass extinction event in which humanity is wiped out by superintelligent AI.
“Meaningful independence only exists through interdependence, one that is impossible in the worlds these technological visions imagine.”
Thus, the AGI narrative is an eschatological one: We must build that which might destroy us so that we may be saved. Either humanity goes extinct or it transcends its mortal coil. And from this eschatology follows a pragmatic need and a desire for the centralization of power. Data, computers and capital must be concentrated in massive quantities to achieve the intelligent precursors to a superintelligent system, pursued by large corporations, wealthy academic research labs and private investors. It takes millions of dollars to train a model like GPT-3 or to build a system like DeepMind’s AlphaStar. And if one is so concerned about the existential risk from a technological system, the desire to keep that system as tightly-held and top-down controlled as possible naturally follows.
Rather than having an explicit politics at all, the goal of AGI is largely to transcend the political. This vision translates matters of political judgment into questions of technical expertise and, in doing so, aims to solve collective action problems with technology: cure cancer, solve climate change, end poverty. The clear superiority of machine decision-making in this imagined future makes human self-governance laughable. Of course we would choose the perfect, quasi-omniscient decision maker over our own imperfect selves.
Humans have very little “traditional” political autonomy in this view. Instead, human autonomy is expressed as autonomy from need — a freedom from the daily toil of being human, as AI replaces “humans at most economically valuable work,” to quote OpenAI’s mission statement. This is outcome-orientation in the extreme, with almost any means brought to bear to justify human survival first, and thriving, second.
Freedom From Control
The blockchain vision of autonomy is, in contrast, explicitly political. The invention of Bitcoin, the first fully functional, public, decentralized blockchain protocol, is largely attributed to a disparate group of countercultural “hacker-engineers.” This group of misfits, known as the “Cypherpunks,” operated on a mantra of self-organization and direct action to build privacy-preserving, cryptographic software tools to counter the threat of corporate and state surveillance in the digital age. They believed that “software can’t be destroyed and that a widely dispersed system can’t be shut down.”
The online Cypherpunk mailing list was the center of ideation and action in the 1990s. The list included characters such as anarchist author of “The Temporary Autonomous Zone,” Hakim Bey, who shared a popular vision of autonomy as self-governance for self-determination, achieved through creating spaces that elude formal structures of control. “The autonomy of the individual appears to be complemented & enhanced by the movement of the group; while the effectiveness of the group seems to depend on the freedom of the individual,” Bey wrote in 1991, tying together autonomy and freedom in a way that has endured in the crypto community.
Timothy C. May, co-founder of the mailing list and a key spokesperson of the group, was a self-professed libertarian and author of “The Crypto Anarchist Manifesto.” This manifesto is an explicitly political vision of an “anonymous computerized market” that catalyzes “a social and economic revolution.” “Decentralization,” as Nathan Schneider describes it, at first encompassed the principles of transparent, open-source, permissionless access to digital infrastructure, but has transformed into a plastic political philosophy that seeks to maximize autonomy through political freedom, freedom of choice and voluntary association.
“The more we aim for a future free from human messiness, the more we push humanity away in the present.”
The ultimate good of the blockchain vision is in freedom itself, but in the negative sense — freedom from regulation, from corruption, from banks, from schools and from governments. In blockchain imaginaries, the desired autonomy is a highly individual autonomy of process and the ends are contingent on individual choice.
From these origins, blockchain has developed and evolved into a plethora of protocols and applications, many of which directly claim to actualize visions of politically autonomous futures through software code. Blockchain-based “Decentralized Autonomous Organisations” are the logical extension of this ideal of cyber and physical autonomy, in which the digital infrastructure itself becomes the zone that eludes formal structures of control. Here, political autonomy is enacted through the automation of rules and procedures via smart contracts, self-enforceable agreements intended to eliminate the need for a priori trust between parties.
Yet, despite these ideals of decentralization, these technologies often inevitably collapse into re-centralization in practice because the forces of commerce and individual profit drive technological innovation. For example, the capital entry requirements for most on-chain governance mechanisms remain prohibitively high for most, either in terms of hashing power or token holdings, throwing a wrench in claims of universal access.
More broadly, ends are always contingent on individual choice, making solving collective action problems wholly contingent on individual incentive structures. Because of the extreme ideological orientation toward autonomy throughout the development process in blockchain communities, technical and administrative decentralization can easily be confused with “democratic” decision-making and non-hierarchical modes of interaction, even though one doesn’t necessarily foster the other.
This vision elides the fact that political autonomy from external forces of influence cannot be implemented in a broader sociotechnical system without colliding with necessary trade-offs. There may be times, for instance, when the autonomy of one individual or group constraints the autonomy of others. There is no widely agreed-upon vision to fall back on for how to pursue autonomy. However, when the goal is frictionless coordination, the inherent friction of deliberation and feedback necessarily runs counter to the goals of the system — the time and investment needed to referee unsolved political, ecological and social concerns simply slow the process down.
Why We Need More Than Autonomy
Blockchain and AI visions of autonomy might seem to be in tension. On the one hand, AGI projects emphasize the collective good, require centralization and seek to produce machine-controlled systems that trade human autonomy for an autonomous future free from the burden of labor. On the other hand, blockchain communities imagine technologically mediated, decentralized, rational, voluntary and self-interested coordination games as a process of ongoing experimentation in autonomy via self-governance, with value determined by commodification and outcomes decided by individual benefit, rather than the collective good.
One might think the excesses of each could be mitigated through a fusion of the two. The aspiration of “decentralized AGI” certainly subscribes to this theory — pushing for AGI algorithms (known as AI “agents”) that run on decentralized blockchain-based marketplaces where they automatically coordinate tasks and earn digital tokens for their work.
Yet, both visions fall short, separately or combined, for one core reason: each takes for granted that the goal of autonomy thus presented is worth pursuing. Trustlessness, on the blockchain side, and automation, on the AI side, aim to subsume to the machine the strength and growth borne of cross-community dependence, space for disagreement and deliberation, and human higher-order critical thinking in cooperation and coordination.
The failures of this approach are already bearing fruit. The more we aim for a future free from human messiness, the more we push humanity away in the present: cast aside data workers whose labor underpins AI systems, cover up knowledge of autonomous system bias and error, dissolve well-intentioned projects into asset speculation and obfuscate the misdirection of present trajectories in visions of future perfection. There is little reason to believe that persisting on this path will yield to better outcomes.
“Interdependence is power, and moving from brittle autonomy toward emergent and collective answers to complex problems is the path to progress.”
The real liberatory potential of these technologies may come from another stance: that it is by relinquishing the dream of a perfectly autonomous future that we can achieve any widely shared and co-constructed notion of the common good.
We can leverage the strengths of these approaches — a focus on solving big problems for the collective good and on individual and community self-sovereignty — and combine them with fundamentally humanistic principles: plurality, mutualism and solidarity. We can expand our communities of fate and create an interdependent evolution of branching possibilities, rather than following a narrow path toward the singular end goal of autonomy or confining ourselves to the idea that legitimacy can be achieved only through immediate decentralization.
This must be done by changing our modes of investment and prioritization in the technologies themselves. We can imagine futures that are both humanistic and powerful — and we can bend these technologies toward the actualization of those futures by reorienting toward technology supported by interdependence, rather than driving toward autonomy.
As one of the authors of this piece has laid out in a recent whitepaper, the fundamental techniques that underlie AGI research can be targeted toward any number of ends, and only a vanishingly small percentage of these aim to replicate human capabilities and achieve perfectly autonomous systems. Imagine AI capabilities instead deployed toward enabling human coordination and communication — activities which have opened up vast new frontiers of human flourishing already, but which clearly have far to go before reaching decreasing returns. Or distributed to communities, such that they are able to contribute to system design and training, to leverage human-machine cooperation to add to the store of human knowledge, rather than subjecting many to living under a centralized machine mandate.
On the blockchain side, nascent projects are already seizing on the fact that the same technologies that allow for diffuse asset speculation can enable the rise of 21st century worker cooperatives, data unions and regenerative land trusts. The same coordination mechanisms that often default to plutocracy can be designed for democratic ends, with moves away from token-based voting toward models that focus on proof-of-personhood or through rewarding members based on their contributions to public goods rather than on their access to compute. These initiatives are united by a growing recognition of interdependence — a return to the fostering of a collective and a commons as a necessary counterbalance to the individual.
Discussions about technology are often carved into a binary of power and progress vs. dependence and democracy. This is a false binary: interdependence is power, and moving from brittle autonomy toward emergent and collective answers to complex problems is the path to progress. Pursuing this path will allow us to cast off the eschatology of technological determinism and embrace the ecology of technological pluralism.
