Acerca de

Week 4: Our Final Century?

“So if we drop the baton, succumbing to an existential catastrophe, we would fail our ancestors in a multitude of ways. We would fail to achieve the dreams they hoped for; we would betray the trust they placed in us, their heirs; and we would fail in any duty we had to pay forward the work they did for us. To neglect existential risk might thus be to wrong not only the people of the future, but the people of the past.”

- Toby Ord

Humanity appears to face existential risks: a chance that we'll destroy our long-term potential. We’ll examine why existential risks might be a moral priority, and explore why they are so neglected by society. We’ll also look into one of the major risks that we might face: a human-made pandemic, worse than COVID-19.

Alongside this we'll introduce you to the concepts of neglectedness, marginal thinking, and explore whether you could lose all of your impact by missing one crucial consideration.

We’ll also introduce the following concepts:

The importance, neglectedness, tractability framework: The most important problems generally affect a lot of people, are relatively under-invested in, and can be meaningfully improved with a small amount of work.
Thinking on the margin: If you're donating $1, you should give that extra $1 to the intervention that can most cost-effectively improve the world. There are many great initiatives with a very high average impact per dollar that will have a low marginal impact because they can't get the same efficiency at scale (they display "diminishing marginal returns").
Crucial considerations: It can be extremely hard to figure out whether some action helps your goal or causes harm, particularly if you’re trying to influence complex social systems or the long-term. This is part of why it can make sense to do a lot of analysis of interventions you’re considering.

Required Materials

Existential risks:

The case for reducing existential risks (25 mins.) or Chapter 2 of The Precipice (Book - 35 mins.)

Thinking on the margin:

Marginal Impact

ITN framework:

A framework for comparing global problems in terms of expected impact (15 mins.)
See how 80,000 Hours apply this framework in their profile on climate change (20 mins.)

Risks from pandemics:

Why experts are terrified of a human-made pandemic — and what we can do to stop it (15 mins.)
“Future risks” chapter of The Precipice, introduction and “Pandemics” section. Stop at “Unaligned artificial intelligence” (Book - 25 mins.)

Strategies for improving biosecurity:

Concrete Biosecurity Projects (7 mins.)
Biosecurity needs engineers and material scientists (4 mins.)

Crucial considerations:

Crucial considerations and wise philanthropy (24 mins.)

More to explore

Existential risks:

Policy and research ideas to reduce existential risk - 80,000 Hours (5 mins.)
The Vulnerable World Hypothesis - Future of Humanity Institute - Scientific and technological progress might change people’s capabilities or incentives in ways that would destabilize civilization. This paper introduces the concept of a vulnerable world: roughly, one in which there is some level of technological development at which civilization almost certainly gets devastated by default. (45 mins.)

Criticism:

Democratizing Risk: In Search of a Methodology to Study Existential Risk (50 mins.)
A critical review of “The Precipice” (maybe come back to the “Unaligned Artificial Intelligence” section next session, once you’ve engaged with the argument for AI risk).

Biosecurity:

Reducing Global Catastrophic Biological Risks Problem Profile - 80,000 Hours (1 hour)
The Apollo Report
Global Catastrophic Risks Chapter 20 - Biotechnology and Biosecurity Overview from ~15 years ago on how biotechnological power is increasing exponentially, as measured by the time needed to synthesize a certain sequence of DNA. This has important implications for biosecurity. (1 hour)
Open until dangerous: the case for reforming research to reduce global catastrophic risk (Video - 50 mins.)
Dr. Cassidy Nelson on the twelve best ways to stop the next pandemic (andlimit COVID-19) 80k podcast interview (podcast - 2.5 hours)
Andy Weber on rendering bioweapons obsolete and ending the new nuclear arms race 80k podcast interview (podcast - 2 hours)
Article on information hazards in biotechnology (15 mins.)
Using Export Controls to Reduce Biorisk (6 mins.)
Lynn Klotz on improving the Biological and Toxin Weapons Convention (BWC) (10 mins.)
Horsepox synthesis: A case of the unilateralist’s curse? (8 mins.)

Climate change:

Climate Change Problem Profile - 80,000 Hours - An analysis of the worst risks of climate change, some of the most promising ways to reduce them, and how to prioritize climate change against other problems. (30 mins.)
Effective Environmentalism (Website)

Nuclear security:

Daniel Ellsberg on the creation of nuclear doomsday machines - Daniel Ellsberg on the institutional insanity that maintains large nuclear arsenals, and a practical plan for dismantling them (Podcast - 2 hours 45 mins.)
List of nuclear close calls - Wikipedia - A description of the thirteen events in human history so far that could have led to an unintended nuclear detonation (5 mins.)
Risks from Nuclear weapons - A series of posts exploring the extent to which nuclear risk reduction is a top priority, and the most effective ways to reduce nuclear risk.
Nuclear security - Brief summary + relevant articles on the EA Forum

Global governance and international peace:

Ambassador Bonnie Jenkins on 8 years of combating WMD terrorism - an interview with Bonnie Jenkins, Ambassador at the U.S. Department of State under the Obama administration, where she worked for eight years as Coordinator for Threat Reduction Programs in the Bureau of International Security and Nonproliferation. (Podcast - 1 hour 40 mins.)
Modeling Great Power conflict as an existential risk factor (40 mins.)
Why effective altruists should care about global governance - Because global catastrophic risks transcend national borders, we need new global solutions that our current systems of global governance struggle to deliver. (Video - 20 mins.)
Destined for War: Can America and China Escape Thucydides’s Trap (Book)