如何理解《苦涩的教训》

一句话核心

利用计算的通用方法（搜索 + 学习），长远上必然碾压依赖人类专家知识的方法——而且差距只会越拉越大。

Sutton 自己用 70 年的 AI 史来论证这一点。这不是预测，是事后归纳。

1. 它在反对什么

Sutton 批判的是 AI 研究里一种根深蒂固的倾向：把人类专家的“理解”硬编码进系统。

比如下棋程序，早期研究者会手写规则：“车走直线、象走斜线、控制中心格更重要……”——把国际象棋大师的经验翻译成 if-else。语音识别早期也是，把声学知识、音素规则、语言学约束都塞进系统。视觉识别早期去手写边缘检测、SIFT 特征。

这些做法的共同特点是：短期内有效、研究者很有成就感、但长期会碰到天花板。

2. 四个领域的同一个故事

Sutton 举了四个例子，模式惊人地一致：

每一次都是：先用人类知识搭起来 → 短期领先 → 计算规模上来后被通用方法超越 → 旧方法被淘汰 → 旧派研究者不服。

3. 为什么是“苦涩”的

这是全文最微妙的地方。Sutton 不是在说“人类知识没用”，他是在说一个让研究者心理上很难接受的事实：

你花十年研究的领域知识、你引以为豪的精巧设计、你认为“体现了智能本质”的那套机制——在算力规模化之后，统统不如一个更笨但更通用的方法。

“苦涩”在于：

• 它否定了研究者的专业价值——你以为你的领域洞察是核心竞争力，结果不是
• 它在短期内是错的——人类知识方法确实能在好几年里领先，让你觉得这条路是对的
• 它让人难以认输——深蓝赢了之后，旧派象棋研究者说“那不是真正的智能，只是暴力搜索”

这种心理抵抗，Sutton 认为正是领域反复犯同一类错误的原因。

4. 只有两种方法能“无限扩展”

Sutton 给出的正面结论很具体：

搜索（search）和学习（learning）——因为这两者是唯二能随算力增长而持续受益的方法论。

• 搜索：算力越多，搜得越深越广，效果直接提升（深蓝、AlphaGo 的蒙特卡洛树搜索）
• 学习：数据越多、计算越多，模型越强（从 HMM 到深度学习到 LLM）

相比之下，人类知识方法是有上限的——你手写了 100 条规则，算力翻 1000 倍也用不上，因为那 100 条规则就是天花板。

5. 更深一层：别把“发现”当“机制”

文章最后一段是哲学升华：

我们想要的，是能像我们一样去发现的 AI，而不是把我们已经发现的东西塞进去的 AI。

这句话很关键。Sutton 在区分两件事：

• 发现的结果（“物体有边缘”、“象棋要控制中心”）——这些是具体知识，不该硬编码
• 发现的过程（搜索、学习、试错）——这才是真正的“智能机制”，应该构建进系统

把结果硬编码进去，反而会遮蔽过程本身——你越相信“智能就是这样工作的”，就越难接受通用方法能自己找到更好的答案。

6. 今天怎么看这篇文章（2019 → 2026）

写于 2019 年，当时 GPT-2 刚出。之后发生的事几乎是这篇文章的终极验证：

• GPT/LLM 的核心就是“不编码知识”——Transformer 架构几乎不带任何语言学的先验知识，只有“注意力机制”这个通用结构，靠海量数据 + 算力自己学会语法、推理、翻译、写代码
• RLHF / 自我对弈（AlphaGo 的思路）被用来对齐 LLM——搜索+学习这两件法宝继续生效
• Scaling Law（Chinchilla、Kaplan 等）本质就是 Sutton 说的“算力扩展”的定量化：模型效果随参数、数据、算力按幂律提升，而且还没到顶
• 那些“用语言学规则做 NLP”的整个研究范式，在 2018 年之后基本消失了

可以说，大语言模型是《苦涩的教训》的完美注脚——一个几乎不包含任何人类领域知识、只靠“预测下一个 token”这个通用目标 + 海量计算的笨方法，碾压了几十年积累的专家系统。

一句话总结

别把你的聪明才智花在“教 AI 我们已经懂的东西”上；把算力花在“让 AI 自己去发现”上。后者永远赢。

这既是技术判断，也是一种谦逊——承认人类的知识在规模化的计算面前，不是资产，而是天花板。

苦涩的教训

Rich Sutton
2019 年 3 月 13 日

从人工智能 70 年的研究中可以读出的最大教训是：利用计算的通用方法最终是最有效的，而且优势巨大。其根本原因在于摩尔定律，或者更准确地说是它的推广形式——单位计算成本持续指数级下降。大多数 AI 研究都是在假设智能体可用的计算能力恒定的前提下进行的（在这种情况下，利用人类知识就成了少数几种提升性能的途径之一），但只要把时间尺度稍微拉长到超过一个典型研究项目的周期，海量的计算资源必然会出现。为了在短期内取得看得见的改进，研究人员总是试图利用自己对领域的专业知识，但从长远看，真正重要的只有对计算的利用。这两者并非天然对立，但实践中往往相互掣肘：花在一个上的时间就不能花在另一个上，而且人总会对其中某条路径产生心理上的投入与执念。更麻烦的是，依赖人类知识的方法往往让整个方法变得复杂，反而更难从"利用计算的通用方法"中受益。AI 研究者姗姗来迟地认识到这个苦涩教训的例子比比皆是，回顾其中最著名的几个会很有启发。

在国际象棋程序中，1997 年击败世界冠军卡斯帕罗夫的方法基于大规模的深度搜索。当时大多数研究计算机国际象棋的学者对此感到沮丧，他们一直追求的是利用人类对国际象棋特殊结构的理解来设计方法。当一种更简单的、基于搜索并配以专用软硬件的方法被证明效果远胜一筹时，这些以人类知识为基础的研究者并不服气。他们说"暴力搜索"这次或许赢了，但它不是一种通用策略，而且反正也不是人类下棋的方式。这些研究者希望基于人类输入的方法获胜，结果未能如愿，他们感到失望。

计算机围棋中也出现了类似的研究进展轨迹，只是又晚了 20 年。最初的大量工作致力于利用人类知识或游戏本身的特殊性质来回避搜索，但一旦搜索在大规模下被有效运用，所有这些努力都变得无关紧要，甚至起了反作用。同样重要的是利用自我博弈学习来学习价值函数（这一点在许多其他游戏中、甚至在象棋中也成立，尽管在 1997 年那个首次击败世界冠军的程序中学习并没有发挥主要作用）。自我博弈学习以及一般意义上的学习，与搜索相似，都使得大规模计算能被投入进来。搜索和学习是 AI 研究中利用海量计算的两类最重要的技术。在计算机围棋中，正如在象棋中一样，研究者最初的精力都放在利用人类理解上（以减少所需的搜索），直到很久以后才通过拥抱搜索和学习取得了大得多的成功。

在语音识别领域，20 世纪 70 年代有一项由 DARPA 资助的早期竞赛。参赛者中有一大批利用人类知识的专门方法——关于词汇、音素、人类声道等的知识。另一边则是一些性质上更偏统计、计算量也大得多的新方法，基于隐马尔可夫模型（HMM）。结果同样如此：统计方法战胜了基于人类知识的方法。这引发了整个自然语言处理领域的重大转变，在随后的几十年里逐步推进，统计与计算逐渐主导了该领域。近年来深度学习在语音识别中的崛起，只是这一持续方向上的最新一步。深度学习方法对人类知识的依赖更少，使用的计算更多，并借助在庞大训练集上的学习，构建出了显著更优的语音识别系统。和在下棋中一样，研究者们总是试图让系统按照他们以为自己大脑工作的方式来工作——他们试图把那种知识写进系统——但这最终被证明是适得其反的，也是对研究者时间的巨大浪费，因为凭借摩尔定律，海量计算终将变得可用，而人们也会找到把它派上用场的方法。

在计算机视觉领域也出现了类似的模式。早期的方法把视觉看作是在搜索边缘、广义圆柱体，或者是基于 SIFT 特征。但今天这一切都被摒弃了。现代深度学习神经网络只使用了卷积和某些不变性的概念，却表现得远为出色。

这是一个重大的教训。作为一个学科领域，我们至今仍未充分吸取它，因为仍在继续犯同类错误。要看清这一点，并有效地加以抵制，我们必须理解这些错误为何具有吸引力。我们必须领受这个苦涩的教训：把"我们以为自己思考的方式"硬编码进系统，从长远看是行不通的。苦涩教训基于以下历史观察：1）AI 研究者常常试图把知识直接构建进他们的智能体；2）这在短期内总是有效的，也让研究者本人感到满足；3）但从长远看，它会陷入瓶颈，甚至阻碍进一步的进展；4）突破性的进展最终总是来自一种相反的路径——通过搜索和学习来扩展计算。这种最终的成功之所以带着苦涩、往往也消化不良，是因为它战胜的是研究者所钟爱的、以人为中心的方法。

从苦涩教训中应该学到的一点是：通用方法具有巨大的力量——那些即便在可用计算变得极大时仍能随计算增长而继续扩展的方法。看来能以这种方式任意扩展的两类方法，就是搜索和学习。

从苦涩教训中应该学到的第二个一般性观点是：心智的实际内容是极其复杂、不可救药地复杂的；我们应当停止试图找到简单的方式来刻画心智的内容，比如用简单的方式来思考空间、物体、多智能体或对称性。所有这些都属于那个任意的、内在复杂的外部世界。它们不是应当被硬编码进系统的东西，因为它们的复杂是没有尽头的；相反，我们应当只构建那些能够发现并捕捉这种任意复杂性的元方法。这些方法的关键在于它们能找到好的近似，但对这些近似的寻找应该由我们的方法来完成，而不是由我们亲自完成。我们想要的是能像我们一样去发现的 AI 智能体，而不是把我们已经发现的东西塞进它们身体里的智能体。把我们的发现硬编码进去，只会更难看清"发现的过程本身"是如何完成的。

The Bitter Lesson

Rich Sutton
March 13, 2019

The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin. The ultimate reason for this is Moore's law, or rather its generalization of continued exponentially falling cost per unit of computation. Most AI research has been conducted as if the computation available to the agent were constant (in which case leveraging human knowledge would be one of the only ways to improve performance) but, over a slightly longer time than a typical research project, massively more computation inevitably becomes available. Seeking an improvement that makes a difference in the shorter term, researchers seek to leverage their human knowledge of the domain, but the only thing that matters in the long run is the leveraging of computation. These two need not run counter to each other, but in practice they tend to. Time spent on one is time not spent on the other. There are psychological commitments to investment in one approach or the other. And the human-knowledge approach tends to complicate methods in ways that make them less suited to taking advantage of general methods leveraging computation. There were many examples of AI researchers' belated learning of this bitter lesson, and it is instructive to review some of the most prominent.

In computer chess, the methods that defeated the world champion, Kasparov, in 1997, were based on massive, deep search. At the time, this was looked upon with dismay by the majority of computer- chess researchers who had pursued methods that leveraged human understanding of the special structure of chess. When a simpler, search-based approach with special hardware and software proved vastly more effective, these human-knowledge-based chess researchers were not good losers. They said that ``brute force" search may have won this time, but it was not a general strategy, and anyway it was not how people played chess. These researchers wanted methods based on human input to win and were disappointed when they did not.

A similar pattern of research progress was seen in computer Go, only delayed by a further 20 years. Enormous initial efforts went into avoiding search by taking advantage of human knowledge, or of the special features of the game, but all those efforts proved irrelevant, or worse, once search was applied effectively at scale. Also important was the use of learning by self play to learn a value function (as it was in many other games and even in chess, although learning did not play a big role in the 1997 program that first beat a world champion). Learning by self play, and learning in general, is like search in that it enables massive computation to be brought to bear. Search and learning are the two most important classes of techniques for utilizing massive amounts of computation in AI research. In computer Go, as in computer chess, researchers' initial effort was directed towards utilizing human understanding (so that less search was needed) and only much later was much greater success had by embracing search and learning.

In speech recognition, there was an early competition, sponsored by DARPA, in the 1970s. Entrants included a host of special methods that took advantage of human knowledge---knowledge of words, of phonemes, of the human vocal tract, etc. On the other side were newer methods that were more statistical in nature and did much more computation, based on hidden Markov models (HMMs). Again, the statistical methods won out over the human-knowledge-based methods. This led to a major change in all of natural language processing, gradually over decades, where statistics and computation came to dominate the field. The recent rise of deep learning in speech recognition is the most recent step in this consistent direction. Deep learning methods rely even less on human knowledge, and use even more computation, together with learning on huge training sets, to produce dramatically better speech recognition systems. As in the games, researchers always tried to make systems that worked the way the researchers thought their own minds worked---they tried to put that knowledge in their systems---but it proved ultimately counterproductive, and a colossal waste of researcher's time, when, through Moore's law, massive computation became available and a means was found to put it to good use.

In computer vision, there has been a similar pattern. Early methods conceived of vision as searching for edges, or generalized cylinders, or in terms of SIFT features. But today all this is discarded. Modern deep-learning neural networks use only the notions of convolution and certain kinds of invariances, and perform much better.

This is a big lesson. As a field, we still have not thoroughly learned it, as we are continuing to make the same kind of mistakes. To see this, and to effectively resist it, we have to understand the appeal of these mistakes. We have to learn the bitter lesson that building in how we think we think does not work in the long run. The bitter lesson is based on the historical observations that 1) AI researchers have often tried to build knowledge into their agents, 2) this always helps in the short term, and is personally satisfying to the researcher, but 3) in the long run it plateaus and even inhibits further progress, and 4) breakthrough progress eventually arrives by an opposing approach based on scaling computation by search and learning. The eventual success is tinged with bitterness, and often incompletely digested, because it is success over a favored, human-centric approach.

One thing that should be learned from the bitter lesson is the great power of general purpose methods, of methods that continue to scale with increased computation even as the available computation becomes very great. The two methods that seem to scale arbitrarily in this way are search and learning.

The second general point to be learned from the bitter lesson is that the actual contents of minds are tremendously, irredeemably complex; we should stop trying to find simple ways to think about the contents of minds, such as simple ways to think about space, objects, multiple agents, or symmetries. All these are part of the arbitrary, intrinsically-complex, outside world. They are not what should be built in, as their complexity is endless; instead we should build in only the meta-methods that can find and capture this arbitrary complexity. Essential to these methods is that they can find good approximations, but the search for them should be by our methods, not by us. We want AI agents that can discover like we can, not which contain what we have discovered. Building in our discoveries only makes it harder to see how the discovering process can be done.

https://www.cs.utexas.edu/~eunsol/courses/data/bitter_lesson.pdf