371 episodios
39 specific projects to improve our chances with AGI | Holden Karnofsky, Anthropic
16/11/2026 | 4 h 34 minIn light of recent events in AI, we're rereleasing some of older episodes that seem more relevant than ever. This episode with Anthropic's Holden Karnofsky from October 2025 lays out the many concrete things people can do to reduce the risks from increasingly powerful AI — ranging from technical solutions to policy and regulations to improving information security.
If you want more ideas about how you can help steer the trajectory of AI development in a better direction, see our page on the 80,000 Hours website: https://80000hours.org/hugging-face/
---
For years, working on AI safety usually meant theorising about the ‘alignment problem’ or trying to convince other people to give a damn. If you could find any way to help, the work was frustrating and low feedback.
According to Anthropic’s Holden Karnofsky, this situation has now reversed completely.
There are now large amounts of useful, concrete, shovel-ready projects with clear goals and deliverables. Holden thinks people haven’t appreciated the scale of the shift, and wants everyone to see the large range of ‘well-scoped object-level work’ they could personally help with, in both technical and non-technical areas.
In this interview, Holden — previously cofounder and CEO of Open Philanthropy (now Coefficient Giving) — lists 39 projects he’s excited to see happening, including:
Training deceptive AI models to study deception and how to detect it
Developing classifiers to block jailbreaking
Implementing security measures to stop ‘backdoors’ or ‘secret loyalties’ from being added to models in training
Developing policies on model welfare, AI-human relationships, and what instructions to give models
Training AIs to work as alignment researchers
And that’s all just stuff he’s happened to observe directly, which is probably only a small fraction of the options available.
All this low-hanging fruit is one factor behind his decision to join Anthropic this year. That said, his wife was also a cofounder and president of the company, giving him a big financial stake in its success — and making it impossible for him to be seen as independent no matter where he worked.
Holden makes a case that, for many people, working at an AI company like Anthropic will be the best way to steer AGI in a positive direction. He notes there are “ways that you can reduce AI risk that you can only do if you’re a competitive frontier AI company.” At the same time, he believes external groups have their own advantages and can be equally impactful.
Host Rob Wiblin and Holden discuss the case for and against those strategies, and much more, in today’s classic episode — originally released October 30, 2025.
Learn more, video, & full transcript: https://80k.info/hk25
Chapters:
Rob’s intro + cold open (00:00:00)
An AI Chernobyl we never notice (00:02:10)
Is rogue AI takeover easy or hard? (00:06:51)
The AGI race isn't a coordination failure (00:17:12)
What Holden now does at Anthropic (00:27:42)
The case for working at Anthropic (00:29:50)
Can we trust Anthropic, or any AI company? (00:40:42)
Can Anthropic compete while paying the “safety tax”? (00:49:24)
What, if anything, could prompt Anthropic to halt development of AGI? (00:56:25)
Responsible scaling policies retrospective (00:59:17)
Overrated work (01:14:58)
Concrete shovel-ready projects Holden is excited about in technical AI safety, AI relationships, and biosecurity (01:17:09)
How to choose where to work (01:36:51)
Overrated AI risks: Cyberattacks and persuasion (01:42:51)
Why AI R&D is the main thing to worry about (01:56:44)
The case that AI-enabled R&D wouldn't speed things up much (02:08:43)
AI-enabled human power grabs (02:12:39)
Main benefits of getting AGI right (02:24:53)
The world is handling AGI about as badly as possible (02:30:56)
Learning from targeting companies for public criticism in farm animal welfare (02:33:31)
Will Anthropic actually make any difference? (02:42:56)
“Misaligned” vs “misaligned and power-seeking” (02:57:37)
Success without dignity: how we could win despite being stupid (03:03:28)
Should we expect misaligned power-seeking by default? (03:18:56)
Will reinforcement learning make everything worse? (03:26:50)
Marginal improvements vs big paradigm shifts (03:32:06)
Should safety-focused people cluster or spread out? (03:34:45)
Is Anthropic vocal enough about strong regulation? (03:39:09)
Is Holden biased because of his financial stake in Anthropic? (03:42:44)
Have we learned clever governance structures don't work? (03:47:10)
Is Holden scared of AI bioweapons? (03:49:33)
Holden thinks AI companions are bad news (03:53:11)
Are AI companies too hawkish on China? (04:00:06)
The frontier of infosec: confidentiality vs integrity (04:04:19)
How often does AI work backfire? (04:07:08)
Impactful work and earning to give (04:22:04)
This episode was recorded on July 25 and 28, 2025.
Video editing: Simon Monsour, Luke Monsour, Dominic Armstrong, and Milo McGuire
Coordination and support: Katy Moore
Music: CORBIT- We now know the makeup of Congress for what may turn out to be the two most important years in AI’s development. Democrats won big and AI was maybe the fourth or fifth biggest driver of the backlash.
So is strong and smart AI legislation finally on the way? Not yet, says host Rob Wiblin. Overriding a presidential veto takes 67 votes in the Senate and 290 in the House, and support for tough rules isn’t at that level today.
But there are several ways Congress can make progress right away:
Passing measures already popular in both parties, like requiring companies to secure their model weights against theft, restoring chip export controls on China, and protecting AI whistleblowers
Using must-pass funding bills to build out CAISSI, the likely home of US AI regulation. It runs on about $15 million a year, while the UK, adjusted for the size of its economy, spends something like 50 times as much
Holding hearings where company staff answer under oath: What’s Anthropic’s safety case? How big are your biggest models now? Which safety results were too embarrassing to publish?
Still, Rob thinks none of that will be enough.
Meaningfully slowing the AI race while models are around human level will probably take another warning shot at least as shocking as the Hugging Face hacks.
When that moment comes, what gets done will depend on the ideas lying around, so the time to carefully think through proposals is now. The ideas Rob is most keen on:
Making ‘ban superintelligence’ workable by requiring companies to spend a minimum share of their compute serving customers — creating a dial a regulator could turn up or down
Throttling recursive self-improvement by only letting companies internally use models that finished training at least six months ago
Banning models that can do too much without writing anything down in their chain of thought
Placing government auditors inside the companies
Letting companies quickly overturn clearly political decisions in court
Leaving state AI laws in place rather than preempting them
Rob also explains why halting data centre construction — probably the most popular demand of all — is a useful step but not enough by itself, and why people who understand how frontier models get trained are so badly needed for this work.
This episode was recorded on November 6, 2026.
Learn more, video, and full transcript: https://80k.info/congress-ai
Chapters:
The Congress that could decide AI (00:00)
Legislating AI will still be difficult (00:49)
Four things Congress can do immediately (01:59)
Why workarounds won’t be enough (04:25)
Preparing for future warning shots (05:44)
Why preparation matters (06:35)
The slogan will be ‘ban superintelligence’ (07:28)
Limit recursive self-improvement (08:58)
Enforce monitorability (11:11)
Is it useful to stop data centres? (12:12)
The government needs visibility (13:32)
We need more AI experts in CAISSI (14:37)
Checks and balances (16:06)
Don’t stop the states (17:12)
Have good ideas lying around (17:46)
Our production team includes:
Video editors: Josh Alward, Dominic Armstrong, Jasper Luithlen, Milo McGuire, Luke Monsour, and Simon Monsour
Producers: Elizabeth Cox and Nick Stockton
Coordination and support: Katy Moore and Lou Moran
Camera operator: Dominic Armstrong Is Anthropic on the cusp of RSI? Inside the redacted METR report | Luisa Rodriguez
12/11/2026 | 15 minAnthropic’s AI is working to build the next version of itself. For the first time ever, an independent team got a chance to investigate this.
METR spent 11 weeks inside the lab, with access to data almost no one outside Anthropic has seen. They found its AI is improving twice as fast as last year. Largely due to AI doing the work.
To put that into perspective: in 2023 Claude couldn’t interpret images or search the web. This year, it helped crack a math problem that had stumped the best mathematicians for 87 years.
Now imagine another jump like that, but in months instead of years. AI companies could build models far more powerful than today’s, before anyone could understand or control them… and then use them to build more capable successors even faster.
Researchers call this recursive self-improvement. And it seems to be where we’re heading.
METR found nothing inside Anthropic that suggests this process is likely to slow down. In fact, it says the rate of speedup itself could be growing.
Anthropic redacted much of the report, so we may never see everything METR saw.
But in this video, Luisa Rodriguez explains what METR was allowed to share with the world, why many of the most respected experts are freaking out, and the strongest reasons METR might be wrong.
She also covers what needs to change, and what you can do to help.
This episode was recorded on November 9, 2026.
Learn more, video, and full transcript: https://80k.info/METR-RSI
Chapters:
AI is now building AI twice as fast (00:00:00)
11 weeks inside Anthropic (00:01:09)
Two years of progress in one year (00:01:59)
Is AI causing the speed-up? (00:03:19)
No signs of slowing down (00:05:15)
Could METR be wrong? (00:07:21)
Has Anthropic crossed its own line? (00:10:18)
What needs to change (00:11:33)
What you can do (00:13:12)
Our production team includes:
Video editors: Josh Alward, Dominic Armstrong, Jasper Luithlen, Milo McGuire, Luke Monsour, Simon Monsour, Ollie Bignell, and Andrés Escobar
Producers: Elizabeth Cox and Nick Stockton
Coordination and support: Katy Moore, Lou Moran, and Taylor SmithThe US is building the smartest AI. China is running a different race | Luisa Rodriguez
10/11/2026 | 26 minChina’s best models are only six months behind America’s, despite having far less computing power. How is that possible?
Luisa Rodriguez investigates, and finds that much of China’s progress is made by using American chips and copying American models:
US rules stop NVIDIA selling its best chips to China. But Chinese companies can still rent those same chips in data centres abroad.
Smugglers go to extreme lengths to sneak chips into China.
Chinese AI companies copy America’s best models on an industrial scale.
Today’s most powerful technological secrets are protected by alarmingly inadequate security.
The US could tackle these gaps with powers it already has. So the current situation, with China only months behind, is largely a policy choice.
If the main argument against slowing down is that China might catch up, none of this quite adds up.
Four days after recording this, NVIDIA’s CEO Jensen Huang — who features in the video as one of the loudest voices against restricting China’s access to chips — admitted to having an affair with an alleged Chinese spy. This shows just how badly China wants American chips.
Closing these gaps in US AI policy could buy time to test new models and hold back those that aren’t safe. Before we accept that the US has to race ahead, it’s worth asking who benefits from racing, and what’s currently being given away.
This episode was recorded on November 4, 2026.
Links to learn more, video, and full transcript: https://80k.info/china-chips
Chapters:
Update: since we recorded (00:00:00)
Everyone said China had caught up (00:00:30)
Who’s building China’s AI? (00:01:50)
How far behind is China? (00:03:20)
Why can’t China make its own chips? (00:04:40)
So where do China’s chips come from? (00:06:24)
How do you copy an AI? (00:10:35)
China is running a different race (00:12:34)
Why a bigger lead matters (00:15:30)
Could the US widen the gap? (00:16:51)
What if China just steals the models? (00:21:12)
So can the US slow down? (00:24:53)
Our production team includes:
Video editors: Josh Alward, Dominic Armstrong, Jasper Luithlen, Milo McGuire, Luke Monsour, Simon Monsour, Ollie Bignell, and Andrés Escobar
Producers: Elizabeth Cox and Nick Stockton
Coordination and support: Katy Moore and Lou Moran
Camera operator: Dominic ArmstrongYour AI crash course starts here: Our best episodes to get up to speed on transformative AI
06/11/2026 | 1 minWe make a lot of episodes about AI these days. We recently picked out the best 10 episodes of the show to really help deepen your understanding of AI and AGI and superintelligence and recursive self-improvement — and the impact all of those things might have.
We aimed for episodes that were entertaining as well as highly informative, and tried to choose 10 that didn't overlap too much, so between them could cover most of the core issues.
You can find them all now in a feed called The 80,000 Hours Podcast on Artificial Intelligence, available wherever you get podcasts.
And at https://80000hours.org/AIpod, you can sign up to get one of them sent to your email each week for 10 weeks — plus one week with our bonus episodes of article readings and how to pivot your career into AI.
Across the 10 core episodes, you will learn:
How AI could compress a century of intellectual progress into a decade
Why today's training techniques actively encourage models to lie to us
How the economy and geopolitics will shift in response to superhuman AI, and indeed are shifting already in response to superhuman AI
How superintelligent AI is in serious tension with maintaining democracy
the real nature of the AI arms race between the US and China
How AI shifts the risks of pandemics and bioweapons over time
how to think about AI consciousness and welfare and whether it matters, and the scramble to understand what AI models are really thinking about deep down.
If you're also looking to change your career to work in AI, we close out with an episode that lays out dozens of valuable concrete projects that need people to work on them right now, and an interview with the founder of 80,000 Hours explaining how to redirect your career towards this kind of work.
Learn more about what you can do to help: https://80000hours.org/ai/
Más podcasts de Educación
Podcasts a la moda de Educación
Acerca de 80,000 Hours Podcast
The most important conversations about artificial intelligence you won’t hear anywhere else.
Subscribe by searching for '80000 Hours' wherever you get podcasts.
Hosted by Rob Wiblin, Luisa Rodriguez, Zershaaneh Qureshi, and Tom Reed.
Sitio web del podcastEscucha 80,000 Hours Podcast, OPOSITAR ES DE VALIENTES por Paco Barbié y muchos más podcasts de todo el mundo con la aplicación de radio.es

Descarga la app gratuita: radio.es
- Añadir radios y podcasts a favoritos
- Transmisión por Wi-Fi y Bluetooth
- Carplay & Android Auto compatible
- Muchas otras funciones de la app
80,000 Hours Podcast: Podcasts del grupo









































