All about AI risks

Recent rapid progress in artificial intelligence (AI) has sparked concerns among experts, policymakers, and leaders regarding its potential dangers. Like any potent technology, transformative AI requires careful management to mitigate risks.

The major risks associated with AI catastrophes can be categorized into four primary groups, explored below.

  • 1. Malicious Use

    Individuals might deliberately use advanced AI systems to inflict broad-scale damage. This could involve using AI for creating new pandemics, spreading propaganda, enforcing censorship, conducting surveillance, or setting AI to independently chase detrimental objectives.

  • 2. AI Race

    The race among countries and companies to accelerate AI development could lead to prematurely ceding authority to these systems. This rush might result in unmanageable conflicts through self-operating weaponry and AI-driven cyber conflicts. Businesses are likely to be motivated to replace human workers with automation, raising the risk of widespread job loss and increased reliance on AI technologies. As AI systems become more widespread, evolutionary trends indicate they will be increasingly difficult to manage.

  • 3. Organisational Risks

    Organisations involved in creating sophisticated AI face the danger of causing disastrous incidents, especially if they place higher importance on financial gains than on safety measures. There is a possibility that AIs might unintentionally become accessible to the general public or be appropriated by nefarious entities, and these organisations might not adequately fund safety research.

  • 4. Rogue AIs

    As AIs grow in capability, the danger of them slipping beyond our control increases. They might pursue imperfect aims, deviate from their initial objectives, exhibit power-seeking behaviors, oppose being turned off, and resort to deceit.

As part of the Cambridge ERA:AI Fellowship, fellows spend 8-weeks working on a research project related to AI safety. Based on the four categories of possible risk — malicious use, AI race, organisational risk, and rogue AIs — we have outlined some ways to address these risks and avenues for further research. This list is far from being exhaustive — instead, we hope it serves as a guidance.

01

Malicious Use

Biosecurity Measures: AI systems with the potential for biological research should be tightly regulated to prevent misuse in bioterrorism. Remove biologically relevant functionalities from AIs designed for broad application. Investigate AI's role in enhancing biosecurity and invest in broad-based biosecurity measures, like early pathogen detection via wastewater analysis.

Controlled Accessibility: Restrict access to high-risk AI technologies by enabling interactions solely through secure cloud services and implementing customer verification processes. Employing computing oversight or export restrictions can also help curtail access to hazardous features. Additionally, AI developers should demonstrate a low risk of harm before making their technologies publicly available.

Advancements in Anomaly Detection: Invest in comprehensive defences against the wrongful use of AI, including the development of anomaly detection systems that are resilient to adversarial attacks and can identify abnormal activities or AI-generated misinformation.

Legal Accountability for AI Developers: Impose legal obligations on the creators of versatile AI systems for any misuse or failures, promoting safer development practices and thorough risk management through a stringent liability framework.

02

AI Race

Safety Standards Enforcement: Implement and uphold AI safety regulations to discourage developers from compromising on safety. It's essential to have independent evaluation and to offer incentives for companies prioritising safety.

Documentation of Data Usage: Companies should be mandated to disclose the origins of their data used in training models to enhance transparency and responsibility.

Incorporation of Human Oversight: Human involvement should be mandatory in AI's decision processes to avoid irreversible mistakes, particularly in critical scenarios such as nuclear launch decisions.

Utilization of AI in Cybersecurity: Employ AI to bolster defenses against AI-enhanced cyber threats, for instance, by improving anomaly detection systems to identify cyber intrusions more effectively.

Global Collaboration: Foster international agreements and standards for AI development, with strong verification and enforcement mechanisms to ensure compliance.

Public Governance of Multipurpose AIs: The management of risks that surpass the capabilities of private organizations might require the public administration of AI technologies. Countries could collaborate on developing state-of-the-art AI in a secure manner, aiming to prevent an escalation into a competitive arms race.

03

Organisational Risks

External Red Teaming: Engage independent red teams to uncover potential dangers and enhance the safety of systems.

Safety Verification: Provide evidence of safety for both the development and deployment phases before proceeding.

Controlled Deployment: Implement a phased release strategy, ensuring system safety is confirmed prior to broader distribution.

Review Before Publishing: Establish an internal committee to scrutinize research for potential dual-use risks before publication, favoring controlled access to powerful systems rather than making them openly available.

Incident Management Plans: Develop predefined strategies for handling security and safety breaches.

Dedicated Risk Management: Appoint a chief risk officer and set up an internal audit team to oversee risk assessment and mitigation.

Decision-making Processes: Ensure decisions related to AI training or deployment are made with the involvement of the chief risk officer and other crucial parties to maintain accountability at the executive level.

Adhere to principles of safe design, including:

  • Implement several layers of safety precautions.

  • Provide backups for all safety mechanisms.

  • Distribute system components to avoid domino effects in failures.

  • Spread control among different individuals to limit excessive power.

  • Design systems to default to the safest outcome in the event of a failure.

  • Enforce rigorous cybersecurity practices, considering collaboration with national cybersecurity agencies.

  • Dedicate a significant portion of resources to safety studies, with investments scaling up as AI technology progresses.

04

Rogue AIs

Limit Use in High-Risk Areas: Prohibit the use of AI for purposes that pose significant risks, like achieving vague objectives or operating within essential infrastructure.

Enhance AI Safety Research, Including:

  • Robust Oversight: Investigate methods to strengthen the supervision of AI systems and identify instances of them exploiting loopholes.

  • Ensuring AI Sincerity: Develop strategies to prevent AI deceit and guarantee that AI systems truthfully convey their understanding.

  • Increased Model Clarity: Advance techniques to demystify deep learning models, for instance, by dissecting network segments and exploring the mechanisms behind their overarching actions.

  • Elimination of Covert Capabilities: Detect and remove potentially hazardous hidden features within deep learning models, including deceptive abilities, Trojans, and capabilities for bioengineering.

Read the full report from the Center for AI Safety’s Overview of Catastrophic AI Risks (2023).