Over the years, ERA Fellows have made significant contributions to the AI research landscape. We highlight some of their work here.
Research at ERA
Claire Dennis
In her working paper "Towards a UN Role in Governing Foundation Artificial Intelligence Models", co-authored with Jason Hausenloy and published by the United Nations University, Centre for Policy Research (UNU-CPR), Claire seeks to provide initial answers to this question. Here are 3 key takeaways:
1. Not all AI is created equal. The UN should prioritize addressing the risks posed by advanced AI systems, such as foundation models like GPT-4. The black-box nature, emerging capabilities, and rapid progress of these models make them uniquely difficult to predict and govern.
2. Current proposals for international institutions highlight important governance mechanisms, such as verification and scientific consensus, but have significant shortcomings in enforceability and agility. Additionally, safety expertise and compute resources are highly concentrated within a few private sector companies, mostly in the U.S. This constrains international regulatory capabilities, especially for the UN.
3. As the world’s only truly global organization, the UN must leverage its moral authority to speak on humanity’s behalf in the face of such a powerful technology. As it has done with nuclear proliferation and climate change, the UN must determine whether the current and projected risks from advanced AI are acceptable, and if so, how they should be collectively mitigated.
What is the UN’s role in AI Governance?
Cooperative Capabilities: Welfare Diplomacy
Gabe Mukobi
The growing capabilities and increasingly widespread deployment of AI systems necessitate robust benchmarks for measuring their cooperative capabilities. Unfortunately, most multi-agent benchmarks are either zero-sum or purely cooperative, providing limited opportunities for such measurements. We introduce a general-sum variant of the zero-sum board game Diplomacy—called Welfare Diplomacy—in which players must balance investing in military conquest and domestic welfare. We argue that Welfare Diplomacy facilitates both a clearer assessment of and stronger training incentives for cooperative capabilities. Our contributions are: (1) proposing the Welfare Diplomacy rules and implementing them via an open- source Diplomacy engine; (2) constructing baseline agents using zero-shot prompted language models; and (3) conducting experiments where we find that baselines using state-of-the-art models attain high social welfare but are exploitable. Our work aims to promote societal safety by aiding researchers in developing and assessing multi-agent AI systems.
Aris Richardson
Experts aiming to govern AI development are interested in understanding the costs of training AI models to predict their future capabilities, development timelines, and potential creators. Since training costs decrease as compute becomes more efficient, it's important to understand factors increasing efficiency like semiconductor R&D automation.This report gathers estimates on advanced AI's impact on the automation of semiconductor chip design from 12 experts in machine learning and hardware. The report claims that the most likely way chip design would become fully automated is that a Western AI lab uses a generalized AI scientist to combine existing EDA (electronic design automation) with new chip design algorithms. The report notes that it seems technically feasible, but not cost-effective to develop a powerful narrow chip design AI before a generalized scientific research AI. Ultimately, the report makes five recommendations for the U.S. government to maintain its lead in chip design as AI increasingly automates the design process: uphold semiconductor equipment export controls, restrict access to frontier models through APIs instead of open sourcing, institute end-use regulations for advanced chip design, increase enforcement funding for software controls, and institute controls to regulate advanced open source EDA.
Estimating the implications of advanced AI for automating chip design
Efficacy of AI Activism - Have we ever said no?
Charlie Harrison
Are there any historical precedents for slowing down the development of Artificial Intelligence (AI) through protest? In 2023, the first public protests groups, like PauseAI, have emerged. Given the incentives for continued AI development, it might seem like these groups are bound to be ineffectual. This project shows that the history of technology paints a more optimistic picture. It first outlines a framework for identifying relevant analogues based on comparable "drivers" for AI development and “motivations” for restraining it. Shallow case studies of six protests are then presented: against geoengineering experiments, nuclear weapons/energy, fossil fuels, CFCs, and GMOs. Contrary to narratives of technological inevitability, protests have influenced policies with strong geopolitical drivers (nuclear power in Germany, nuclear weapons in Kazakhstan and Sweden), and high commercial value (GMOs, fossil fuels). Protests have shaped examples of international policymaking like the nuclear treaties of the 1980s, and the revision of the Montreal Protocol. Other lessons include the importance of ‘trigger events’ for protest groups, and that government advisors can benefit from protest. We have “said no” to relevant analogues for AI.
Somsubhro Bagchi
General purpose AI systems have been shown to display a wide range of harmful behaviour including reinforcing biases, generating toxic and offensive outputs, leaking personally identifiable information from training data, disseminating disinformation and generating extremist texts. The lack of explainability of AI systems and the uncertainty behind which business model companies will adopt make it difficult to decide what exact share of the regulatory burden should be attributed to each developer or deployer in the AI value chain. We recommend that developers are required to make reasonable efforts to comply with Article 15. In order to provide advice and assistance to the Commission, the to-be-formed EU AI Board will maintain a list of best practices and checks that general purpose AI systems should pass. Given existing information on capability emergence, we also recommend careful monitoring of AI systems in an initial partial release phase with early withdrawal plans. We also recommend that developers make documentation concerning relevant accuracy metrics in foreseeable use cases, available to deployers. We outline an alternative, more stringent approach to accuracy and robustness that is currently employed by OpenAI. This approach may be of interest to European policymakers.
