Turing Award Winner Richard Sutton Calls Synthetic Data a 'Big Mistake' for Scaling LLMs

Richard Sutton says synthetic data can't scale LLMs because it's only a 'microscopic' simulation of an infinitely complex world.