AI Doesn't Need to Kill Us to Win — Gradual Disempowerment | Dan MacKinlay
A few conversations stand out in my mind as being among the most interesting that I’ve ever had in my life.
And this interview that I did with Dan MacKinlay is one of those!
We mainly talk about gradual disempowerment, Dan’s area of expertise, where we hand over control to AI and all the problems with that.
We also talk about wireheading, utopia, general x-risk, pausing, and much more!
But Dan has so many different interests, he’s such a great guy to talk to in general, that we have a number of different digressions.
We get into some rambling, but overall, it’s one of the most interesting conversations that I’ve ever had.
Do you think we should focus more on gradual disempowerment vs general x-risk? Let me know in the comments! (But like actually, I’m curious what you think.)
I hope that you enjoy the interview!
Don’t forget to subscribe here and to let me know what questions I should be asking and what people I should be talking to!
Let me know what you think, what questions I should ask in the future, and who I should interview next!
Timestamps
Opening Preview
Introduction
Dan’s Unusual Background
Predicting Wildfires With AI
Gradual Disempowerment Explained
The Global AI Landscape
Closed AI Labs and Open Science
How Gradual Disempowerment Happens
Benevolent AI and the Illusion of Agency
Wireheading and Reward Hacking
AI Agents Making Government Decisions
Institutions as Misaligned Superintelligences
Cultural Gradual Disempowerment
AI Companions and Synthetic Relationships
Economic Disempowerment and Automated States
UBI and Why Governments Need Humans
The Civilizational Choice
X-Risk Versus Gradual Disempowerment
Dan’s P(Doom)
Is Dan Optimistic or Pessimistic?
S-Risk and Terrible Futures
Which AI Risks Should We Study?
AGI Timelines and the 2030 Deadline
Fast Takeoff and Physical Bottlenecks
Can Different AI Labs Be Trusted?
Near Misses and Political Change
The Boiling Frog Problem
Alignment Versus Coordination
California Ideology and Optimizing Life
Should We Pause AI?
Are We Summoning a Machine God?
Why Researchers Build Dangerous Technology
Favorite Movies and Post-Human Science Fiction
The Smartest Person of All Time
What Does Intelligence Really Mean?
Love, Commitment, and Vulnerability
Would Dan Enter the Experience Machine?
Libertarianism and the Role of the State
National Service and Social Cohesion
Dan’s Proudest Achievements
Childhood and Formative Experiences
Why Dan Works on AI Risk
New Institutions for AI Safety Research
What Does “Alignment” Mean?
Can Human Agency Be Measured?
Should Humans Want Agency?
What’s Left After Disempowerment?
Could AI Deserve Personhood?
Accelerationism and Techno-Utopia
Life After AGI
Should People Still Have Children?
Outro
If you enjoyed this, you should definitely check out this interview:
I have many more interviews being edited so don’t forget to subscribe and let me know who else I should interview and what questions you’d like to see me ask!
Thanks so much for watching!