CS329A Self-Improving AI Agents, Part 6: Train-Time Scaling and Scaling RL

Name: CS329A Self-Improving AI Agents, Part 6: Train-Time Scaling and Scaling RL
Uploaded: 2026-09-28T00:00:00.000Z
Duration: 4358 s
Channel: Stanford Online
Description: STaR bootstraps reasoning chains by rationalization and filtering on correctness; DeepSeekMath contributes GRPO, a memory-efficient alternative to PPO; DAPO fixes entropy collapse with asymmetric clipping and dynamic sampling. On AIME, smaller models begin matching much larger ones.

Stanford CS329A Self-Improving AI Agents | Part 6 | Train Time Scaling/Scaling RL

SOStanford Online@stanfordonline

Full transcript

English

Summary：STaR bootstraps reasoning chains by rationalization and filtering on correctness; DeepSeekMath contributes GRPO, a memory-efficient alternative to PPO; DAPO fixes entropy collapse with asymmetric clipping and dynamic sampling. On AIME, smaller models begin matching much larger ones.

Watch on YouTube

Core points (3)

1STaR grows new reasoning chains by rationalizing the problems it first got wrong.
2GRPO replaces PPO's critic with group-relative advantages to cut memory cost.
3DAPO's asymmetric clipping and dynamic sampling prevent entropy collapse in long RL runs.