Acerca de

Week 6: Risks from Artificial Intelligence

Transformative artificial intelligence may well be developed this century. If it is, it may begin to make many significant decisions for us, and rapidly accelerate changes like economic growth. Are we set up to deal with this new technology safely?

This session you’ll also:

Learn about Bayes’ rule, a guide to how to change your beliefs as you get new evidence.

Required Materials

The case for worrying about risks from artificial intelligence:

The case for taking AI seriously as a threat to humanity (30 mins.)
Why AI alignment could be hard (18 mins)
AI Timelines: Where the arguments and the “Experts,” stand (13 mins.)

Strategies for reducing risks from unaligned artificial intelligence:

The longtermist AI governance landscape: a basic overview (14 mins.)
AI Safety researcher career review (10 mins.)
Long-term AI policy strategy research and implementation (8 mins.)

Bayes’ rule and evidence:

Bayes’ rule: Guide (15 mins. for the short version)
Making beliefs pay rent (5 mins.)
What is evidence? (4 mins.)

Suffering risks:

S-risks: Why they are the worst existential risks, and how to prevent them (13 mins.)

More to explore

The development of artificial intelligence

AlphaGo - The Movie - DeepMind - A documentary exploring what artificial intelligence can reveal about the 3000-year-old game of Go, and what that can teach us about the future potential of artificial intelligence. (Video - 1 hour 30 mins.)
The Artificial Intelligence Revolution: Part 1 - A fun and interesting exploration of artificial intelligence by the popular blogger Tim Urban. (45 mins.)

Other resources on aligning artificial intelligence

My personal cruxes for working on AI safety (65 mins.)
Professor Stuart Russell on the flaws that make today's AI architecture unsafe & a new approach that could fix it (Podcast - 2 hours 15 mins.)
Some Background on Our Views Regarding Advanced Artificial Intelligence - Open Philanthropy - An explication of why there is a serious possibility that progress in artificial intelligence could precipitate a transition comparable to the Neolithic and Industrial revolutions. (1 hour)
The Precipice - Chapter 5 (pages 138-152) - Unaligned Artificial Intelligence (25 mins.)
What Failure Looks Like (12 mins.) - Two specific stories about what a very bad society-wide AI alignment failure could look like, which differ considerably from the classic “intelligence explosion” story
AGI Safety from first principles (1 hour 15 mins.) - one AI researcher’s take on the specific factors for the problem of aligning general AI
Human Compatible: Artificial Intelligence and The Problem of Control (Book)
The Alignment Problem: Machine Learning and Human Values (Book)

Governance for artificial intelligence

The new 30-person research team in DC investigating how emerging technologies could affect national security - 80,000 Hours - How might international security be altered if the impact of machine learning is similar in scope to that of electricity? (Podcast - 2 hours)
Technology Roulette: Managing Loss of Control as Many Militaries Pursue Technological Superiority - Center for a New American Security - An argument for how advances in military technology (including but not limited to AI) can impede relevant decision making and create risk, thus demanding greater attention by the national security establishment. (60 mins.)

Technical AI alignment work

AI Alignment Landscape (Video - 30 mins.)
AI safety starter pack (7 mins.)
How to pursue a career in technical AI alignment (59 mins.)
Technical Alignment Curriculum (readings for a 7 week course)
The Alignment Forum, especially their core sequences

Criticisms of worries about AI risk

How sure are we about this AI stuff? (26 mins.)
A tale of 2.75 orthogonality theses (20 mins.)
How to know if AI is about to destroy civilization (summary, 2 mins.)
The AI Messiah (and the first comment) (5 mins.)
How good is humanity at coordination? (4 mins.)