scripod.com

Richard Sutton – Father of RL thinks LLMs are a dead end

Overview

Shownote

Highlights

Transcript

Chapters

Pins

Richard Sutton – Father of RL thinks LLMs are a dead end

Dwarkesh Podcast

Sep 26

Richard Sutton – Father of RL thinks LLMs are a dead end

Richard Sutton – Father of RL thinks LLMs are a dead end

Dwarkesh Podcast

Dwarkesh Podcast

Sep 26

Overview Shownote Highlights Transcript Chapters Pins

Shownote

Richard Sutton is the father of reinforcement learning, winner of the 2024 Turing Award, and author of The Bitter Lesson. And he thinks LLMs are a dead end. After interviewing him, my steel man of Richard’s position is this: LLMs aren’t capable of learnin...

Highlights

In a thought-provoking conversation, Richard Sutton, a pioneer of reinforcement learning and recipient of the 2024 Turing Award, challenges the prevailing trajectory of AI development, particularly the dominance of large language models. He argues that true intelligence must emerge from experience-driven learning rather than static imitation.

02:50

LLMs lack a definition of actual knowledge and ground truth.

19:18

Cultural knowledge accumulation starts with imitation, not reasoning.

32:10

The transition model includes beliefs about action consequences and abstract experiences, not just rewards.

39:49

Gemini Deep Research helped compile a comprehensive history of reinforcement learning with context and links.

41:29

Simple basic principles like learning and search have won in AI

53:06

HRT shares all trading strategies in a monorepo to enable fast learning and broad deployment.

53:48

AI represents a major transition from replication to design in the universe's evolution

Chapters

Are LLMs a dead-end?

00:00

Do humans do imitation learning?

13:04

The Era of Experience

23:10

Current architectures generalize poorly out of distribution

33:39

Surprises in the AI field

41:29

Will The Bitter Lesson still apply post AGI?

46:41

Succession to AIs

53:48

Transcript

Dwarkesh Patel: Today, I'm chatting with Richard Sutton, who is one of the founding fathers of reinforcement learning and inventor of many of the main techniques used there, like TD learning and policy gradient methods. And for that, he received this year'...