Generative AI and Democracy @ iWORD 2022
Saffron Huang
A lightly edited transcript of a talk on the impact of generative AI on democracy and our knowledge commons, given at the International Workshop on Reimagining Democracy (IWORD) organized by Bruce Schneier and others in December 2022. See Divya’s, on a vision for collective intelligence, here.
Today I’ll be focusing on generative AI and its impact on democracy and our knowledge commons. Generative AI are quite diverse, but essentially, they’re machine learning algorithms that can learn from content like text, audio, video, images and code to output new, often quite original content. Progress in this technology has been swift in recent years.
You may have seen ChatGPT, made by OpenAI, which is all tech twitter is talking about this week. ChatGPT is an AI, specifically a language model which generates text in response to prompts. You can prompt it to make essays, stories, sonnets, code and more. It’s trained by ingesting terabytes’ worth of books, code, academic papers, articles and web pages, and based on this data it learns to predict the most likely continuation of a sequence. This training approach has downsides; language models tend to hallucinate facts and regurgitate biases in the training data. Companies, like OpenAI, are now incorporating human feedback and various safety mechanisms in these models to address this.
ChatGPT is impressive, almost magical. You can query it like Google, you can give it some code and ask it to find bugs or rewrite it, you can ask it to summarize Apple’s Ts&Cs in plain words, or have it write a history of London in the style of Dr Seuss. It has very general capabilities, and can honor surprisingly specific requests.
Many new startups are rapidly productising the tech for making websites, illustrations, Instagram ads, teaching assistants, literature reviews and so on. AI artists are inventing new workflows of generating and editing art using models like Midjourney. Commercial applications are only just starting, and second order implications are barely known. I’m inevitably going to be talking about a very small slice of the possible implications for democracy.
Generative AI depends on the digital commons. Many are trained on publicly available data and rely on public infrastructure, like Wikipedia and Creative Commons licenses and open-source software, these shared Internet resources being the "digital commons". The digital commons also overlaps with the knowledge commons, the epistemic public goods that shape our knowledge of the world and each other. Generative AI depends on and impacts both.
They may mitigate the classic commons problem where public goods tend to be undersupplied, by speeding up the creation process.
But they could also misuse, degrade, and privatize the commons they and we depend on.
Generative AI can generate text, code and images much faster than humans, but often outputs untrue, biased or otherwise low-quality outputs. We’ve seen stereotypes amplified, and outputs are often subtly incorrect and hard to evaluate. Stackoverflow already banned ChatGPT answers on Monday, saying the volume of plausible but often wrong answers is overwhelming their volunteer-driven, human moderation capacities.
Soon we’ll see more AI-generated content of unknown quality in the commons, mostly in online information and software, but maybe even on billboards or in books. As AI generations increasingly pass for human, provenance of content will be a huge challenge.
Artists and open-source developers have become upset and even taken legal action as they feel their work has been misused or plagiarized, potentially replacing the very people whose work the models depend on. The pool of human content may shrink while AI-generated content increases, as people either replace work with AI, or fence their work when they do make it, eroding the benefits of open knowledge and creativity.
Generative AI can, by degrading the commons, also degrade self-determination and democracy.
It’s primarily an epistemic and cognitive tool, so it’s also a political tool. AI may not technically understand meaning, but it can manipulate words or images as symbols that mean something to us. High speed generation of personalized persuasion or disinformation, and realistic material to accompany it, is concerning. And if we overuse the same AI, with the same inductive biases, we might begin to homogenize and flatten discourse.
More broadly, automating various functions of human cognition has implications for human autonomy and judgement, and for how we build political and social systems from the bottom up. Democracy rests on there being a free, equal, educable citizenry that can make sound decisions, which rests on good epistemic norms, education and a high quality knowledge commons.
Generative AI can help us become better thinkers—if it fetches relevant citations for us, or acts as a conversational partner in attempting a difficult political discussion, or translates jargon to plain words. But it could also easily make decisions and predictions in place of thinking. If students are getting language models to write their essays with minimal editing, they’re not doing the cognitive work of learning. We need to encourage the former if we care about democracy and also if we are to implement things like sortition.
Also, the concentration of power around AI could be very undemocratic. Decisions around how the model behaves, its content filters, and training data are usually made by a small team with little public participation or transparency. As these models become widely used, they may become akin to newspapers or schools, epistemic institutions that determine our “truth”, and what views do or don’t get perpetuated — but potentially based on the decisions of only a few researchers rather than many journalists and teachers. Unlike the effect of eg the Twitter algorithm, generative AI can create not just curate information. There is great potential also for people to be misled into e.g. thinking AI is sentient, with unknown consequences.
If current techniques persist, future AI might also end up trained on internet data created by previous AI ad infinitum, further entrenching current patterns and perspectives.
In all, how do we use this powerful epistemic technology to enhance rather than degrade critical thinking, democracy and the commons, and how can we inject pluralism into the way it’s designed?
One approach is to see data inputs to generative AI are fundamentally collective and human-oriented, and we can leverage them as such. Generative artificial intelligence is built by humans, off huge datasets from humans, improved through human feedback, and used by humans. At different stages of its creation this AI is guided by humanity’s collective intelligence; but the design process is opaque, not very inclusive, and currently produces a distorted and not very trustworthy engine of collective intelligence output.
OK, there’s already some collective human input, but we can push for fairer, more pluralistic approaches. Various solutions have different trade offs, but we can eg invest in human-centric digital spaces and AI tools, give human data subjects involved greater ownership over models, invest in distributed ML computing, convene citizens assemblies to suggest content policies for particular AI; make it easy for people to open-source, fork and modify models, or have multistakeholder consortia that 1) monitor the digital commons, 2) mandate data sharing related to AI deployment, and 3) decide on auditing and transparency standards.
Going back to CIP, our goal is for humans to flourish in the ways we decide to, even as science and technology progresses at speeds that feel out-of-control. Yes, there is a degree of unpredictability in innovation, but we have also underinvested in social and organizational technologies to handle, and to determine the impact of our more concrete technologies.
We need better tools and systems of collective governance that can handle unpredictable technology and don’t overly trade off progress, agency and safety (what we call the “transformative tech trilemma”). Potentially these systems need to be as networked and fast-moving as the technologies themselves, can gather and distribute quality information to and from relevant actors with quick feedback loops, and can surface social desiderata and consequences more quickly.
These open questions around generative AI’s impact on the digital and epistemic commons, human cognition and democracy are critical motivations for these endeavours. As this is an ongoing project, we’d love to share more and discuss or collaborate on this. Thank you.