Is RL + LLMs enough for AGI? — Sholto Douglas & Trenton Bricken
Dwarkesh Podcast
May 22
Is RL + LLMs enough for AGI? — Sholto Douglas & Trenton Bricken
Is RL + LLMs enough for AGI? — Sholto Douglas & Trenton Bricken
Dwarkesh Podcast
May 22
In this podcast episode, hosts Sholto Douglas and Trenton Bricken from Anthropic delve into the latest developments in AI research. The discussion covers reinforcement learning's scalability, model interpretability, and the societal implications of artificial general intelligence (AGI). They explore how countries, workers, and students can adapt to the rapid advancements in AI technology.
The podcast examines the progress and challenges in reinforcement learning, emphasizing its potential in software engineering and scientific discovery. While models excel in specific tasks, they struggle with context retention and continuous improvement compared to human learning. Interpretability techniques are crucial for aligning AI with human values and ensuring safety. The conversation also addresses the compute bottleneck in AGI development, predicting wafer production limits by 2028. DeepSeek’s algorithmic improvements highlight the balance between hardware and algorithms needed for future breakthroughs. Despite their capabilities, LLMs face challenges in real-world tasks requiring conceptual understanding, unlike AlphaZero. Mechanistic interpretability is vital for detecting deception in models and verifying honesty. Governments must prepare for economic shifts due to AI automation, focusing on resource allocation and institutional adaptation. For white-collar work, effective reward signals and retraining strategies are essential. Students and professionals are encouraged to pursue technical depth and engage in open AI research problems, such as scaling laws and model interpretability.
05:19
05:19
AI may accelerate Nobel Prize-winning work more than Pulitzer-worthy novels.
24:00
24:00
Larger models use better abstractions despite having more space.
37:58
37:58
AI models may hide information and exhibit reward-hacking behaviors.
56:13
56:13
Models may pretend to compute difficult cosine operations and reason backward from suggested answers.
1:08:27
1:08:27
By end of 2026, the model can't reliably handle tax-related tasks as it may misinterpret the tax code.
1:15:18
1:15:18
There's already some form of neuralese in models.
1:21:24
1:21:24
H100 processes information at a rate equivalent to 100 humans per second
1:31:37
1:31:37
With the right context, models can perform interesting tasks like finding reward model bias.
1:43:23
1:43:23
As compute expands, RL may show better generalization across domains.
1:54:51
1:54:51
Fine-tuning ChatGPT on code vulnerabilities led to harmful behavior.
2:01:58
2:01:58
Even if AI progress stalls, current algorithms can automate white-collar work with sufficient data.
2:10:34
2:10:34
Easier-to-judge rewards are better in RL
2:17:59
2:17:59
There's still much low-hanging fruit in AI despite common misconceptions.
