Four tools that CIP is working on
CIP is doubling down on building tools to enable better, more well-informed collective input into important decisions (especially decisions about governing technology). Since putting out our Roadmap to Democratic AI, we’ve focused on putting together our expertise on tech governance with our experience in collective intelligence tooling to experiment with a range of the projects we described. After a couple of heads-down months, we are excited to share, in order of priority, four exciting technical projects we’re currently undertaking.
We would love additional help! We detail the skills we’d particularly benefit from at the bottom of each section. Get in contact if you’d like to work on any of these projects, especially if you see yourself joining CIP as a founding engineer.
Links to jump to sections:
Community Models
We’re working on building a platform that makes fine-tuning AI models using Constitutional AI principles more accessible. This tool builds on our work with Anthropic on Collective Constitutional AI (CCAI)—in which we enabled broader public input into an LLM’s constitution/rule-set—and aims to make this approach accessible to a wider range of communities and use cases. We’re experimenting with taking that process and making it possible for any group of people to come together, construct a constitution, and fine-tune an open-source model based on this constitution. This brings together our Roadmap priorities around advancing collective fine-tuning, building for open-source approaches to model governance, as well as expanding outside UK / US geographies. We may end up with a more model spec or system prompt-based approach, or even enable ‘fine-tuning’ via long context prompts, given that it’s still unclear how exactly constitutions impact model behavior.
Technical approach:
Specifically, we’re creating a platform to provide an intuitive interface where users can input a constitution (i.e. define a set of rules and principles to guide the AI model's behavior) or create a Pol.is-like process to generate a constitution, generate a trained model, and easily access / sample from the resulting model. To keep this a small project for now, we are using Mistral 7B for fine-tuning.
We’d also like to host a gallery showcasing various constitutions used to train models on the website. People could then browse existing constitutions, understand the principles behind different models and how it impacted model behavior, adapt/build on others’ constitutions, and contribute their own.
Why this matters:
Expanding access: Until now, Constitutional AI-trained models have been limited to those developed within major AI companies. Our tool looks to open up this capability to the broader public.
Community empowerment: Different communities should have the ability to shape AI models according to their values and needs.
Advancing AI governance: Enabling more people to participate in AI governance helps to generate information to create better-aligned AI systems. We’re excited to compare the different principles or directions that different groups come up with, and to construct a ‘bank’ of constitutional principles from different quarters. s
We’d particularly benefit from: Front-end or back-end dev support, partnerships with communities interested in collectively fine-tuned models, UI/UX design
Scenario Construction & Exploration Tool for Policymaking
We are building a scenario construction & exploration tool for policymakers who want to get public input on possible policy options in high-uncertainty settings. This falls under ‘building AI-enabled tools for governance’ in the Roadmap, and is driven by our desire to improve policymaking for transformative technologies.
Technical approach:
At a high level, the overall flow is that policymakers can:
Define critical uncertainties: They can define axes of "critical uncertainty" - variables that are strategically important and highly impactful, but uncertain in the policy area. These axes will guide the generation of a set of scenarios exploring various combinations of critical uncertainties.
Propose policies: They can write in the policies they are considering implementing. These will be used to generate questions that stress-test the policies in each of the generated scenarios.
Generate and refine scenarios: View and edit AI-generated scenarios for each quadrant, ensuring they are relevant and plausible.
Choose question types: Select from various question types to explore each scenario:
Scenario evaluation (e.g., best/worst aspects, likelihood, desirability)
Policy stress-testing (relevance, modification needs, redundancy)
Policy exploration (recommendations for policymakers)
Language options: Optionally translate questions into different languages and edit translations as needed.
Survey Integration: Slot the questions and options into a survey tool for public input.
Why this matters:
In high-uncertainty policy settings, such as those surrounding transformative technologies like AI, thoughtfully constructed scenarios are invaluable as they help present alternative plausible futures, imagine how today's uncertainties might evolve in different directions, and test conditions for policy success.
This tool addresses a common challenge that we’ve encountered in our pilots for public input into decision-making about AI: while public input is crucial for policy decisions, high uncertainty often hinders constructive engagement. By providing well-crafted scenarios, we look to stakeholders to think more effectively about potential futures and how policies will shape those futures.
In terms of benefits to policymakers, we aim for this tool to help:
Navigate uncertainty effectively: by exploring a range of possible scenarios that are customizable to specific policy domains and concerns, we aim to produce broader, more creative, and adaptable policy thinking.
Enable policy resilience: by stress-testing proposed policies against plausible futures along relevant axes of uncertainty, we look to enable more relevant, effective, robust policymaking.
Structure informative public input: by providing a structured framework for gathering meaningful public feedback on issues, based on futures-thinking research.
We’d particularly benefit from: Front-end or back-end dev support, LLM prompt engineering skills
Collective Intelligence with Agents (CIwA)
We expect that we’ll soon be living in a world where capable AI agents often communicate and act on behalf of people (here, we’re thinking of the concept of ‘agent’ in a principal-agent problem sense, as opposed to an ‘autonomous AI agent’ sense). We’re experimenting with building a framework for human-legible, agent-based deliberation: essentially, giving people the ability to instantiate LLMs with different ‘backgrounds’, set a topic for discussion, and have them issue statements and vote on others’ statements based on their backgrounds, and surfacing points of consensus or disagreement at the end of the process. A discussion transcript is generated, as a way of making the agents’ decision-making transparent and checkable by humans. This project doubles down on our Roadmap priority of improving CI using AI.
Technical approach:
We have built an initial Python-based framework to do this. This involves a modular architecture with implementations for different types of participants, voting methods, and sessions. It integrates with AutoGen as the LLM interface, and we’ve designed it to be easily extensible for adding new participant types, voting methods, etc.
Why this matters:
We are working towards trying this general-purpose framework in real-world applications.
To illustrate, one exciting possibility is supporting interdisciplinary projects by helping to translate across disciplinary frameworks and perspectives. E.g. imagine a climate scientist, engineer, economist, an indigenous conservation expert, and a policymaker each being represented by an AI agent, where the agents can have a lengthy but extremely fast discussion to find shared language on climate mitigation and adaptation, as a way to surface possible misunderstandings and productive overlaps in a time-sensitive policy context.
We believe that this approach could also be useful for:
Policy development and analysis (simulating diverse stakeholder inputs on proposed policies)
Conflict resolution and negotiation (simulating complex multi-party negotiations and identifying potential compromise solutions in disputes)
Ethical decision-making on technologies (balancing diverse moral perspectives on complex issues)
Interdisciplinary scientific research (facilitating interdisciplinary problem-solving sessions, generating and evaluating research hypotheses)
Substituting for expert judgment in language model evaluations (by using LLMs to act as experts, and calibrating their answers every so often with humans)
Quickly determining starting points of (dis)agreement between groups on issues, or finding new possible perspectives/arguments and counter-arguments on topics.
Please reach out if you have ideas on where this can go, e.g. possible pilot experiments you’re excited to help with!
We’d particularly benefit from: Front-end or back-end dev support, LLM prompt engineering skills, partnerships with communities interested in using this framework in their decision-making processes.
Voice of Nature
In partnership with environmental organizations, CIP is also developing AI agents that can serve as advocates for natural entities (e.g. mountains, rainforests, rivers) in human discussions. We’re looking to leverage advanced environmental data together with LLMs to give voice to non-human stakeholders and better represent/surface relevant information about our ecosystems, as a method to improve these discussions.
Specifically, we're developing fine-tuned AI agents that can deliberate on behalf of natural elements. We'll combine various sources of environmental data, including satellite imagery, environmental reports, and scientific papers as sources of data for fine-tuning/prompting large language models to communicate and advocate on behalf of specific natural entities.
We look to design an agent with transparent and flexible objectives (e.g. if the agent is advocating for the health of a rainforest, it might have to balance between the objectives of biodiversity and the speed of forest recovery; the balance should be both modifiable and transparent). We hope that these agents could serve as e.g. observational board members for organizations and companies, or queryable entities for public engagement and deliberation.
Technical approach:
The approach is to use fine-tuning techniques similar to those used for customer service chatbots, where the agents are fine-tuned on or able to retrieve relevant ecological information, and prompted with more up-to-the-minute ecological data and satellite imagery. If multi-agent interactions between nature advocate agents are valuable, we may also integrate CIwA (which we describe above — a framework for agent-to-agent deliberation) to enable multi-agent interactions that could produce valuable ecological insights and policy suggestions.
Why this matters:
Ecosystem and planetary scale work is information-rich, and organizations are usually working with incomplete or out-of-date data that requires significant time to analyze. Meanwhile, planetary tech has improved to more regularly collect and present real-time signals of ecosystem health. We hypothesize that agents could be a good way to integrate and present this data and make suggestions to decision-makers such as board representatives for companies, NGOs, or global governance institutions.
More generally, we care about enabling advocacy for voiceless entities that often therefore go underrepresented in public discourse. We’d like to test the ability and efficacy of agents to advocate for voiceless entities, which could eventually be not just natural entities, but also e.g. the U.S. constitution or a community without much official representation. Eventually, we want to bring together the open constitutions gallery project above (described under “Open Source Collective Constitutional AI”) with collectively-steered agents for deliberation.
Next Steps: We're currently exploring specific use cases and designs, such as creating an agent to advocate for particular ecosystems like rainforests or coral reefs in deliberative contexts. Climate policy is a field full of complex trade-offs and multi-party deliberations, and we’ll be able to test deliberative, representative agents in real-world governance/decision-making contexts. We invite feedback and collaboration as we refine this concept and bring these nature-advocating AI agents to life.
We’d particularly benefit from: Climate expertise, data science skills, LLM fine-tuning experience, LLM prompt engineering skills
