LessWrong (30+ Karma)

I don't think this is how it will actually play out. If you play a chess grandmaster, you can predict that they will beat you even if you can't predict how. I chose these examples because I don't think they require much imagination or accepting exotic assumptions.
It is important to note that if chimpanzees were to guess how humans would decimate them, they would get it wrong. Chimpanzees would not imagine guns. They would not foresee poison gas. They would not conceive of chemical castration. They would not imagine humans going around and intentionally infecting them with AIDS. They have no concept of these things; they would not see it coming.
Perhaps they might guess we'd be really good at throwing rocks. Amazingly good. Well, technically, that's what guns do: throw "rocks" really really well.
So how will superintelligent AI actually wipe us all out? Probably in a way I couldn't conceive of. Nonetheless, it's not hard to see how deadly they could be with what we already know about.
Method 1: engineer the deadliest and most contagious virus ever seen
Coronavirus-19, aka COVID, looms large in the memory of living adults today. It started in December [...]
---
Outline:
(01:14) Method 1: engineer the deadliest and most contagious virus ever seen
(04:38) Misconception A: AIs don't have bodies, they can't act in the real world
(04:55) Robotics is here
(05:49) Super-persuasion
(06:52) Method 2: Killer drones
(09:25) Misconception B: Superintelligent AI would not be able to take over any and every computer system
(11:48) Method 3: Take over the WMD, take over the infrastructure
(14:25) Misconception C: We could just turn them off
(15:01) A deadly cockt[ai]l
The original text contained 22 footnotes which were omitted from this narration.
---

First published:

September 11th, 2026


Source:

https://www.lesswrong.com/posts/LAPa2jxoq3n63GzTr/some-ways-ai-could-kill-us-all

---

Narrated by TYPE III AUDIO.

---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Many economists take a relaxed view of employment post-AGI. Even once AI systems with suitable robotic actuators can perform most white-collar work and manual labor, something will remain scarce, and comparative advantage guarantees that human labor is worth hiring for something at some wage. So people will still have jobs, spending most of their waking hours at work, and the economy will look broadly like today's, including property rights, wage labor, and political institutions protecting these.

I don't think this holds up. It is not obvious that current economic or political structures survive the arrival of AGI at all. Powerful AI systems (or even simply AI-enabled human dictatorships) need no more respect our rights than Stalin respected those of the kulaks. But set those concerns aside and grant an optimistic future where power—and access to the fruits of automation—remain broadly distributed. Even then, mass wage employment does not follow, though I do expect former workers to be fine.

The basic issue is this: why would anyone choose to be employed? People today sell their labor because they need to buy food, shelter, clothes, and other material goods. Post-AGI those goods become abundant and cheap, because they can [...]
---

First published:

September 11th, 2026


Source:

https://www.lesswrong.com/posts/4PWHaYWfnFcmKuCcA/post-agi-we-are-all-jobless-aristocrats

---

Narrated by TYPE III AUDIO.

Austin: I’m happy to announce that Caroline Ellison has accepted a role at Manifund, to develop our funding platform and research how to effectively direct philanthropic dollars.
In fact, she started a work trial on July 13, which converted to a fulltime role on Aug 10. Over the past two months, she's been publishing her work and supporting Manifund users under the pseudonym “Carol”.
How did I come to consider this at all?
I specifically enjoyed Caroline's tumblr and other writings, which I found thoughtful and relatable. I invited her to Manifest 2026, just because I wanted to meet her. During the festival, she came to my night market booth, and asked about joining our team, which I was interested to explore.
I felt a keen debt to the FTX Future Fund. They provided seed funding for Manifold. They funded the retreat which drew me into the EA community, and the conference where I met my wife. Much of Manifund's work today is directly inspired by the Future Fund's approach.
On FTX itself, I remain conflicted. It was their money and spirit that enabled the Future Fund, and I still admire many aspects of FTX and the individuals who worked there. But also: FTX did wrong. [...]
---

First published:

September 11th, 2026


Source:

https://www.lesswrong.com/posts/W3zn5jQa8fhmiBsPG/caroline-ellison-has-joined-manifund

---

Narrated by TYPE III AUDIO.

tl;dr I have tested Astra's ability to complete various long multi-step tasks without using its chain-of-thought, and found that the number of sequential steps is a poor predictor of task success. Instead, Astra's ability to solve a task seems to correlate more strongly with what I will call the speculative depth of the task. Based on experimental results, it seems less likely that Astra solves sequential no-CoT tasks purely by reasoning step-by-step in latent space. Instead, Astra appears to do some form of speculative reasoning, where intermediate results are guessed based on heuristics, and then iterated upon in parallel until they become self-consistent. This allows many multi-step tasks to be solved with far fewer serial steps than naively seems possible, especially if the initial guesses are good. Other LLMs also appear to behave like this, but to a much smaller degree.
In my previous post, I discussed how certain KV-cache sharing schemes may lead to long opaque serial paths, and introduced the LatentMathBench microbenchmark, which measures an LLM's ability to solve tasks that require many consecutive steps without using their chain-of-thought. Several other users have done more extensive no-CoT reasoning benchmarks based on more varied (and often more realistic) [...]
---
Outline:
(01:59) Shortcuts
(03:41) Boolean circuits
(08:25) But how?
(10:22) Speculative reasoning
(13:01) Testing task success rate vs speculative depth
(14:38) Open questions
The original text contained 3 footnotes which were omitted from this narration.
---

First published:

September 11th, 2026


Source:

https://www.lesswrong.com/posts/WFc3NkuPaYFrYuaZd/astra-s-no-cot-limits-track-speculative-depth-not-step-count

---

Narrated by TYPE III AUDIO.

---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Summary
I cold emailed a Finnish MP. Four weeks later, the government responded to his written question saying they "seek to constructively promote the creation of international regulation on the development of superintelligent AI" (immediately followed by emphasizing the need to balance safety with innovation).
The MP and I met for lunch. For over an hour, he seriously engaged with my arguments that superintelligence would kill everyone and the urgent need for an international agreement to prohibit its development.
Unprompted, he decided to submit a written question, a formal process in Finland that requires the government to respond within 21 days. Four days after our lunch, it was submitted, taking into account suggestions from me and two experts I DM'd: Nate Soares and Charbel-Raphaël Segerie.
It asked how Finland has assessed risks from superintelligence and whether it's "prepared to promote international regulation that would effectively prevent or restrict the development of superintelligent AI."
The MP started speaking out about AI danger and mainstream Finnish media picked up the story. Most notably, he did an interview alongside a Finnish CS professor on the second-most-watched TV channel in Finland with an estimated 100-200k live viewers.
Inspired by [...]
---
Outline:
(00:13) Summary
(01:59) The email
(02:50) I'm just a guy
(03:22) The meeting
(05:35) The written question
(06:35) Media attention
(07:37) Inspiring a friend
(08:10) The response
(09:46) Do this yourself
---

First published:

September 10th, 2026


Source:

https://www.lesswrong.com/posts/pbKrZCzhsnar6iaAH/how-a-cold-email-got-the-finnish-government-to-respond-on

---

Narrated by TYPE III AUDIO.