YouTube: Spotify:
YouTube：Spotify：
Are we hurtling toward a future where AI can do everything humans can? Edwin Chen (@echen) believes we might be. He’s the CEO of Surge AI, one of the largest providers of expert data for frontier labs. Surge passed over $1 billion in revenue without raising any outside capital, and that gives Edwin a unique perspective on how quickly AI progress is accelerating. I’m on the record arguing that AI automation actually creates more human work. I also believe that even though AI progress is accelerating exponentially, we’re much farther away from AI replacing humans than it might seem. That’s why I had Edwin on @every’s AI & I. We batted around different visions of the future, and discussed whether humanity will retain its unique place in the universe, and what that might be. We get into: • If Chen’s version of the future materializes, he’s worried it’ll make people stop trying. One answer comes from a short story by science fiction writer Ted Chiang: Behave as if your decisions matter, even when you know they don’t. • AI may soon be able to take a nebulous goal like “win a Fields Medal” and execute. What it can’t do, I argue, is set its own goals—LLMs have no intrinsic motivation, no drive to explore, no ability to just change their mind. • A model optimized for engagement doesn’t provide the most valuable user experience. Edwin spent 20 rounds polishing a pointless email with one model before Claude told him to just send it. • Why AI is still bad at writing: models learn to hack the metrics they're trained on. Edwin's Hemingway Bench found models outputting a metaphor in every single sentence, an overindexxing that makes for a terrible reading experience. This is a must-watch for anyone interested in where we fit as models get more capable. Watch below! Timestamps 1. Introduction: 00:00:54 2. Surge as a "school for AGI": 00:01:49 3. What AI's capacity for novel mathematics says about human achievement: 00:04:46 4. Motivation in an era when AI can do everything: 00:07:29 5. The trap of optimizing AI models for engagement: 00:14:34 6. Training using datasets versus training using environments: 00:29:34 7. The value of personal data: 00:35:09 8. Why models are bad at writing: 00:39:40 9. Chen's AGI timeline: 00:42:00
我们是否正高速冲向这样一个未来：AI 能做到人类能做的一切？Edwin Chen（@echen）认为，也许是的。他是 Surge AI 的 CEO，Surge AI 是 frontier labs 的最大专家数据提供商之一。Surge 在没有募集任何外部资本的情况下，营收已超过 10 亿美元，这让 Edwin 对 AI 进展加速的速度拥有独特视角。我一直公开主张，AI 自动化实际上会创造更多需要人类来做的工作。我也相信，尽管 AI 的进展正以指数级加速，但 AI 距离取代人类仍比看上去要远得多。这就是为什么我请 Edwin 来做 @every 的 AI & I 节目。我们来回碰撞了多种不同的未来图景，并讨论了人类是否会保住自己在宇宙中的独特位置，以及那种位置可能是什么。我们谈到了：• 如果 Chen 所设想的未来真的实现了，他担心这会让人们不再努力。一个答案来自科幻作家 Ted Chiang 的一篇短篇小说：即使你知道自己的决定并不重要，也要表现得好像它们很重要。• AI 也许很快就能接过一个像“赢得 Fields Medal”这样模糊的目标并加以执行。但我认为，它做不到的是为自己设定目标——LLM（大语言模型）没有内在动机，没有探索驱动力，也没有单纯“改变主意”的能力。• 一个为 engagement（用户参与度）优化的模型，并不能提供最有价值的用户体验。Edwin 曾用某个模型来回打磨一封无关紧要的邮件 20 轮，最后 Claude 告诉他，直接发出去就行。• 为什么 AI 仍然不擅长写作：模型会学着“钻”它们所依据的训练指标的空子。Edwin 的 Hemingway Bench 发现，模型会在每一句话里都输出一个隐喻，这种 overindexing（过度偏重）会带来非常糟糕的阅读体验。对于任何关心在模型能力越来越强时，我们将处于什么位置的人来说，这期内容都不容错过。请看下方！时间戳 1. 介绍：00:00:54 2. Surge 作为“AGI 的学校”：00:01:49 3. AI 在新颖数学上的能力说明了人类成就的什么：00:04:46 4. 当 AI 能做一切时，这个时代的动机问题：00:07:29 5. 为 engagement 优化 AI 模型的陷阱：00:14:34 6. 使用 datasets（数据集）训练 versus 使用 environments（环境）训练：00:29:34 7. 个人数据的价值：00:35:09 8. 为什么模型不擅长写作：00:39:40 9. Chen 对 AGI 时间线的判断：00:42:00