#17 - Against Longtermism

Ben Chugg and Vaden Masrani — Fri, 18 Dec 2020 19:00:00 -0800

Well, there's no avoiding controversy with this one. We explain, examine, and attempt to refute the shiny new moral philosophy of longtermism. Our critique focuses on The Case for Strong Longtermism by Hilary Greaves and Will MacAskill.

We say so in the episode, but it's important to emphasize that we harbour no animosity towards anyone in the effective altruism community. However, we both think that longtermism is pretty f***ing scary and do our best to communicate why.

Confused as to why there's no charming, witty, and hilarious intro? Us too. Somehow, Ben managed to corrupt his audio. Classic. Oh well, some of you tell us you dislike the intros anyway.

References

The Case for Strong Longtermism, by Greaves and MacAskill
Vaden's EA forum post on longtermism
The reddit discussion surrounding Vaden's piece
Ben's piece on longtermism (which he has hidden in the depths of Medium because he's scared of the EA forum)
Ben on Pascal's Mugging and Expected Values
Gwern and Robin Hanson making fun of Ben's piece

Yell at us on the EA forum, on Reddit, on Medium, or over email at incrementspodcast@gmail.com.

#7 - Philosophy of Probability II: Existential Risks

Ben Chugg and Vaden Masrani — Tue, 07 Jul 2020 11:00:00 -0700

Back down to earth we go! Or try to, at least. In this episode Ben and Vaden attempt to ground their previous discussion on the philosophy of probability by focusing on a real-world example, namely the book The Precipice by Toby Ord, recently featured on the Making Sense podcast. Vaden believes in arguments, and Ben argues for beliefs.

Quotes
"A common approach to estimating the chance of an unprecedented event with earth-shaking consequences is to take a skeptical stance: to start with an extremely small probability and only raise it from there when a large amount of hard evidence is presented. But I disagree. Instead, I think the right method is to start with a probability that reflects our overall impressions, then adjust this in light of the scientific evidence. When there is a lot of evidence, these approaches converge. But when there isn’t, the starting point can matter.

In the case of artificial intelligence, everyone agrees the evidence and arguments are far from watertight, but the question is where does this leave us? Very roughly, my approach is to start with the overall view of the expert community that there is something like a one in two chance that AI agents capable of outperforming humans in almost every task will be developed in the coming century. And conditional on that happening, we shouldn’t be shocked if these agents that outperform us across the board were to inherit our future. Especially if when looking into the details, we see great challenges in aligning these agents with our values."
- The Precipice, p. 165

"Most of the risks arising from long-term trends remain beyond revealing quantification. What is the probability of China’s spectacular economic expansion stalling or even going into reverse? What is the likelihood that Islamic terrorism will develop into a massive, determined quest to destroy the West? Probability estimates of these outcomes based on expert opinion provide at best some constraining guidelines but do not offer any reliable basis for relative comparisons of diverse events or their interrelations. What is the likelihood that a massive wave of global Islamic terrorism will accelerate the Western transition to non–fossil fuel energies? To what extent will the globalization trend be enhanced or impeded by a faster-than-expected sea level rise or by a precipitous demise of the United States? Setting such odds or multipliers is beyond any meaningful quantification."
- Global Catastrophes and Trends, p. 226

"And while computers have been used for many years to assemble other computers and machines, such deployments do not indicate any imminent self- reproductive capability. All those processes require human actions to initiate them, raw materials to build the hardware, and above all, energy to run them. I find it hard to visualize how those machines would (particularly in less than a generation) launch, integrate, and sustain an entirely independent exploration, extraction, conversion, and delivery of the requisite energies."
- Global Catastrophes and Trends, p. 26

References:
- Global Catastrophes and Trends: The Next Fifty Years
- The Precipice: Existential Risk and the Future of Humanity
- Making Sense podcast w/ Ord (Clip starts around 40:00)
- Repugnant conclusion
- Arrow's theorem
- Balinski–Young theorem

Increments - Episodes Tagged with “Expected Value”

#17 - Against Longtermism

#7 - Philosophy of Probability II: Existential Risks