BuildSpeak每日 builder 文摘
今日归档生词本关于
🎙 播客AI & I by Every· 2026 年 6 月 24 日· 7,563 词 · 约 38 分钟

Building a School Where AI Models Learn About Humanity

SPACE 播放 / 暂停·←→ 上一句 / 下一句
Speaker 100:00 - 00:17
We are building this kind of school for AGI where AI models come to learn about humanity, where we teach them how to run the world. It almost seems like there's nothing that humans can do that AI won't soon be capable of. I could see it happening within ten to five years.
Speaker 100:00 - 00:17
我们正在为 AGI 打造这样一所学校:让 AI model 来这里学习人类,教它们如何运行这个世界。这几乎让人觉得,人类能做的事,很快就没有 AI 做不到的了。我觉得这可能会在未来五到十年内发生。
Speaker 200:17 - 00:50
AI may be able to do it better than us, but, like, someone told the AI to go do that. They're being built to be means to tasks that humans want them to do. Right? Every is the only subscription you need to stay at the edge of AI. If you care about being on top of the latest models and using the latest tools, you have to subscribe to to separate out the signal from the noise.
Speaker 200:17 - 00:50
AI 也许能比我们做得更好,但总归是有人告诉 AI 去做那件事的。它们被构建出来,是为了成为完成人类想让它们完成之任务的手段,对吧?Every 是你保持在 AI 前沿所需的唯一订阅。如果你在意紧跟最新 model、使用最新工具,你就得订阅它,才能把真正有价值的 signal(信号)从 noise(噪声)中分离出来。
Speaker 200:50 - 00:54
Go to every. Tosubscribe today. Edwin, welcome to the show.
Speaker 200:50 - 00:54
现在就去 every. 订阅吧。Edwin,欢迎来到节目。
Speaker 100:54 - 00:56
Hey, Dan. Thanks for having me.
Speaker 100:54 - 00:56
嗨,Dan。感谢邀请我来。
Speaker 200:56 - 01:30
For people who don't know, you are the founder and CEO of Surge. You all provide data environments and evals for the model companies, but you do it in this very interesting way. You have this, even on your website, emphasis on taste and expert judgment that I find really interesting and compelling. You talk about raising AGI, which I feel like is a very distinct type of word using data. And you also famously got to about 1,000,000,000 in revenue without raising without raising money, which is wild.
Speaker 200:56 - 01:30
对不了解情况的人来说,你是 Surge 的 founder 和 CEO。你们为这些 model 公司提供 data environment 和 evals(评测),但你们做这件事的方式很有意思。就连在你们的网站上,你们也强调 taste(品味)和 expert judgment(专家判断),这一点让我觉得非常有趣,也很有说服力。你们谈到“培养 AGI”,我觉得这是一种非常独特的数据使用方式。而且,你们还以几乎没融资就把 revenue 做到大约 1,000,000,000 而闻名,这很惊人。
Speaker 201:30 - 01:48
I feel like data is the this new game that a lot of companies are playing and probably more are going be playing soon. You guys are this sneaky giant. Tell me how that's going. I think it's been a little while since we got the last update on how things are going.
Speaker 201:30 - 01:48
我觉得 data 正成了很多公司都在玩的这场新游戏,而且很可能很快会有更多公司加入。你们是这样一个低调的巨头。跟我说说近况吧。我感觉距离我们上一次听到你们业务进展的更新,已经有一段时间了。
Speaker 101:48 - 02:30
Yeah, I think it's going amazing. The way I often think about this is that we are building this school for AGI, the school where AI models come to learn about humanity where we teach them how to run the world. And it's almost like their models are children, where they arrive unformed, and then yeah, they leave smarter and more creative and more thoughtful and ready to operate in the messiness of their world. So I think a lot has changed in the past year, like in the same way that the things that you teach children when they're in preschool or in middle school or in high school is very different from what you're teaching them when they're in college. And it's not just that they're more advanced.
Speaker 101:48 - 02:30
是的,我觉得进展非常棒。我经常这样理解这件事:我们正在为 AGI 建一所学校,一所让 AI model 来学习人类、让我们教它们如何运行世界的学校。这几乎就像这些 model 是孩子一样,它们到来时还没有定型,然后离开时会变得更聪明、更有创造力、更深思熟虑,也更准备好在这个混乱的世界中运作。所以我觉得过去一年变化很大,就像你教孩子的内容一样:他们在 preschool、middle school、high school 时学的东西,和在 college 时学的东西非常不同。而且不只是因为他们变得更高级了。
Speaker 102:30 - 02:55
Like it's not just that you're teaching them a more advanced form of what they did before. It's like, okay, now we are teaching you not just arithmetic, but how do you parse these ambiguous math questions? Or how do you teach people not just grammar, but taste and poetry and beauty? So, yeah, think there's a lot that's been changing in the past year, especially in enterprise. And yeah, it's been crazy time.
Speaker 102:30 - 02:55
不是说你只是在教它们之前所学内容的更高级版本。更像是:好,现在我们教你的不只是 arithmetic(算术),而是你要如何解析这些含糊不清的数学问题?或者,你教人的不只是 grammar(语法),还包括 taste(品味)、poetry(诗意)和 beauty(美感)?所以,是的,我觉得过去一年有很多事情都在变化,尤其是在 enterprise(企业)领域。没错,这段时间真的很疯狂。
Speaker 202:55 - 03:04
What would be like a specific example of what the frontier of teaching was a year ago versus what the frontier is now?
Speaker 202:55 - 03:04
如果要举一个具体例子,一年前教学前沿是什么样子,而现在的前沿又是什么样子,会是什么?
Speaker 103:05 - 03:35
Yes. So a couple years ago, actually, we created our first math benchmark with OpenAI, and it was called GSM eight ks. And this was actually just testing models on their abilities to do middle score math. And even then, the GBT models of the time, they could barely score, I think, like 20%. And then a year ago, the models were like something that became a lot more capable at solving IMO problems.
Speaker 103:05 - 03:35
好的。其实在几年前,我们和 OpenAI 一起做了第一个数学 benchmark(基准测试),叫作 GSM eight ks。它实际上只是测试模型做 middle school math(初中数学)的能力。即便在那个时候,当时的 GPT 模型得分也勉强只有大概 20%。而到了去年,模型已经变得更有能力去解决 IMO 问题了。
Speaker 103:35 - 04:03
But there was still just an open question. Okay, can they actually do research level mathematics? Like, can they move beyond these sort of like competition only, of contrived, very closed problems into doing things that are actually useful in the real world. And so, yeah, a couple of months ago, we released an updated benchmark called Riemann bench, which actually tests models under ability to do research level mathematics. And what's crazy is that this is actually we're starting to see from these models.
Speaker 103:35 - 04:03
但当时仍然有一个悬而未决的问题:它们真的能做 research-level mathematics(研究级数学)吗?也就是说,它们能不能超越这类仅限于竞赛、经过刻意构造、非常封闭的问题,去做那些在现实世界中真正有用的事情。所以,没错,就在几个月前,我们发布了一个更新后的 benchmark,叫 Riemann bench,它实际测试的是模型进行 research-level mathematics 的能力。疯狂的是,我们现在其实已经开始从这些模型身上看到这种能力了。
Speaker 104:03 - 04:45
Like I think in the past few months, they've started to solve a lot of these open AirDish problems. Like a couple of weeks ago, OpenAI published a new result where the models had disproved a open conjecture from AirDish. And the way it went about disproving this was actually a fairly sophisticated level of mathematics, I think, like using a bunch of very novel algebraic geometry techniques. And so, yeah, it's just very, very different from the types of things that we were doing a year ago where, sure, IMO like problems, they're hard, but they're still sort of closed ended and solvable in theory by a high schooler. And now suddenly you have these algebraic geometry results that, you know, even does hop processors in the world, we're kind of amazed and just amazed by.
Speaker 104:03 - 04:45
比如说,我觉得在过去几个月里,它们已经开始解决很多这类来自 Erdős 的开放问题。就在几周前,OpenAI 发布了一个新结果:模型推翻了一个来自 Erdős 的开放猜想。而它推翻这个猜想的方式,实际上涉及相当高深的数学,我认为用了不少非常新颖的 algebraic geometry(代数几何)技术。所以,是的,这和我们一年前在做的事情非常、非常不同。因为 IMO 这类问题当然很难,但它们终究还是某种封闭式的问题,理论上一个高中生也可能解出来。而现在,你突然看到了这些 algebraic geometry 的结果,连世界上的顶尖学者都会对此感到惊讶,真的非常惊讶。
Speaker 204:46 - 05:10
How do you think about that result in particular and what it says about the models? I think there's a there's a sort of a broad range of opinions about is it is it obviously, it's impressive either way, but is it applying a bunch of things that that maybe humans already know but, like, wouldn't have thought to apply to this complicated problem? Or is it doing something actually novel? And and, yeah, how how do you think about LLM's ability to do novel things?
Speaker 204:46 - 05:10
你会怎么理解这个结果本身,以及它说明了这些模型什么?我觉得这里大致存在一系列不同意见。显然,不管怎样这都很令人印象深刻,但问题在于:它到底是在应用一堆也许人类早就知道、只是没想到能用到这个复杂问题上的东西?还是说,它真的做出了某种新东西?以及,是的,你怎么看 LLM(大语言模型)做出 novel(新颖)成果的能力?
Speaker 105:11 - 05:37
So it's something of very advanced results. So I will say that I certainly don't understand the mathematics behind it. And so, like, one of the interesting things is that I was actually so it's kinda funny. When I when I was a kid, I always thought I would be a pure mathematician when I grew up. And so when I saw LagerDistalt, I got kinda nostalgic, I was like, oh, I I wish I understood I wish I understood LagerDistalt better.
Speaker 105:11 - 05:37
这是一个非常高深的结果。所以我得先说,我当然并不理解它背后的数学。某种程度上这也挺有意思的,其实还挺好笑:我小时候一直以为自己长大后会成为一名 pure mathematician(纯数学家)。所以当我看到 Langlands 时,我有点怀旧,我就想,哦,我真希望自己更懂 Langlands。
Speaker 105:37 - 06:26
And so what I ended up doing was, like, throwing the proof into both Claude and Gemini and asking it to try to walk me through, from a layman's perspective, just what was going on. Yeah, my understanding is that it actually did come up with fairly novel algebraic geometry techniques, which was something that you maybe wouldn't have expected for this type of problem. Like on the surface, it feels like It's just a very, very different problem where you wouldn't necessarily use certain techniques. And what was interesting was that OpenAI actually published a bunch of reflections from leading mathematicians about what they thought about the result. I think in particular, there was this one reflection by Timothy Gowers, who's a field analyst that I keep thinking about.
Speaker 105:37 - 06:26
所以我最后做的事情是,把那份证明同时扔给 Claude 和 Gemini,让它们试着从一个 layman(外行人)的视角带我一步步理解到底发生了什么。是的,据我的理解,它确实提出了一些相当新颖的 algebraic geometry 技术,而这可能是你原本不会期待出现在这类问题里的。因为乍看之下,这感觉就是一个非常、非常不同类型的问题,你未必会想到要用某些技术。还有个有意思的地方是,OpenAI 实际上还发布了一些顶尖数学家对这个结果的看法。其中尤其有一则 Timothy Gowers 的评论让我一直印象很深;他是一位 field analyst(领域分析学者)。
Speaker 106:27 - 07:02
And what he said was that when he first heard result, he misunderstood it. He thought that the model had proved an upper bound on the conjecture and was like, okay, yeah, if AI can do that, then it'll be all over for mathematicians very soon. But then the next morning, he actually realized that the model had disproved the conjecture with a counterexample. And he said that he was relieved by it because it felt like an easier thing for AI to do. And I just thought it was interesting because you have one of the world's greatest mathematicians being relieved actually that AI isn't as far as he thought, because it actually means that at least for maybe another year, maybe a couple of years, he and other mathematicians will still have this unique role to play in pushing mathematics forward.
Speaker 106:27 - 07:02
他说的是,当他第一次听到这个结果时,他误解了它。他以为模型证明了这个猜想的一个 upper bound(上界),于是心想,好吧,如果 AI 连这个都能做到,那数学家很快就要“完了”。但第二天早上,他实际上意识到,模型是通过一个 counterexample(反例)推翻了这个猜想。他说,这反而让他松了一口气,因为这感觉像是 AI 更容易做到的一类事情。我觉得这很有意思,因为你会看到,世界上最伟大的数学家之一,竟然会因为 AI 没有他原先以为的那么超前而感到宽慰;因为这至少意味着,也许再过一年,或者再过几年,他和其他数学家仍然会在推动数学前进这件事上,扮演一种独特的角色。
Speaker 107:03 - 07:12
So, yeah, I I think it just speaks to the level of craziness again, because this is a field smallest, one of the smartest mathematicians in the world, and, like, this this is how we think about AI.
Speaker 107:03 - 07:12
所以,是的,我觉得这再次说明了这种疯狂的程度,因为这是一个 Field Medalist,世界上最聪明的数学家之一,而这就是我们如今思考 AI 的方式。
Speaker 207:13 - 07:28
Yeah. And what does that make you think? Okay. You're you wanna be a you wanna be a mathematician when you grow up, field medalist sort of saying, I am relieved that it's not good enough, But you're talking as if you feel pretty confident that it will be good enough in the next couple of years.
Speaker 207:13 - 07:28
对。那这会让你怎么想?好吧。你长大后想成为数学家,结果一位 Field Medal 获得者某种程度上是在说,我很庆幸它还不够好;但你的说法听起来像是,你相当确信它在接下来几年里就会变得足够好。
Speaker 107:28 - 08:05
Yeah. So my belief is that if you really believe in scaling laws, and I do, it's that it almost seems like there's nothing that humans can do that AI won't soon be capable of. If you think about that very deeply, I think you almost have to worry about what would that mean for humanity? Like, what would that mean for the role of humanity in the universe? Like, a couple years ago, you know, we think about humanity and human intelligence as playing a very unique role in the galaxy.
Speaker 107:28 - 08:05
是的。所以我的看法是,如果你真的相信 scaling laws(缩放定律),而我确实相信,那么看起来几乎就像是:人类能做的事,很快 AI 都将有能力做到。如果你非常深入地思考这一点,我觉得你几乎必然会担心,这对人类意味着什么?比如,这对人类在宇宙中的角色意味着什么?就像几年前,我们还会认为,人类以及人类智能在银河系中扮演着一种非常独特的角色。
Speaker 108:06 - 08:44
Then AI comes along and shows us that as far as we know, we can create something that's actually smarter than us and better in many ways. You can imagine one path where humanity as a species falls into a paralysis because people believe AI will do everything better anyways. Like, yeah, all these kids who formerly would have really wanted to grow up to do mathematics, maybe now they believe that, okay, AI will just do it better than me anyways, what's the point? So are kids going to stop wanting to learn and adults stop wanting to create? Because yeah, why should we do this when AI will be better at it than us anyways?
Speaker 108:06 - 08:44
然后 AI 出现了,并向我们表明,至少据我们所知,我们可以创造出某种实际上比我们更聪明、并且在许多方面都更优秀的东西。你可以想象这样一条路径:作为一个物种,人类陷入某种瘫痪,因为人们相信反正 AI 无论如何都会把一切做得更好。比如,是啊,那些原本真的很想长大后做数学的孩子,也许现在会觉得,好吧,反正 AI 无论如何都会比我做得更好,那还有什么意义?所以,孩子们会不会不再想学习,成年人会不会不再想创造?因为,是啊,既然 AI 无论如何都会比我们更擅长这些事,我们为什么还要去做呢?
Speaker 108:45 - 09:03
And so I'll actually think about this story by Ted Chiang, and it's about free will. It's called What's Expected of Us. I think in this story, there's a piece of technology that proves that free will doesn't exist. And the narrator sends back a warning for the future that says, This is a warning. You have to pretend that you have free will.
Speaker 108:45 - 09:03
所以我其实会想到 Ted Chiang 的一个故事,讲的是 free will(自由意志)。它叫《What's Expected of Us》。我记得在这个故事里,有一种技术证明了自由意志并不存在。而叙述者向未来发出了一则警告,大意是:这是一个警告。你必须假装自己拥有自由意志。
Speaker 109:03 - 09:35
It's essential to behave as if your decisions matter, even though you know that they don't. And I think that's really interesting because I think there's a path where we almost have to consciously choose to do things ourselves. Sure, AI can do it all. AI is smarter than us, so it can do it all, and it will do it better anyways. But we actually almost have to consciously choose to prove things on our own and to write on our own and create on our own because we have to believe that preserving our humanity is valuable in of itself, even if the output isn't optimal.
Speaker 109:03 - 09:35
关键在于,你必须表现得仿佛你的决定是重要的,尽管你知道它们并不重要。我觉得这点特别有意思,因为我认为存在这样一条路径:我们几乎必须有意识地选择亲自去做事情。当然,AI 什么都能做。AI 比我们更聪明,所以它什么都能做,而且无论如何都会做得更好。但我们其实几乎必须有意识地选择自己去证明,自己去写作,自己去创造,因为我们必须相信,保留我们的人性本身就是有价值的,即便产出并不是最优的。
Speaker 109:37 - 09:45
I think there are a lot of these big, thorny, existential choices that AI is starting to force upon us and people will have to make.
Speaker 109:37 - 09:45
我认为,AI 正开始迫使我们面对很多这样重大、棘手、关乎存在本身的选择,而人们将不得不作出决定。
Speaker 209:45 - 10:01
That's a really interesting one. And I I think my first response I'm curious what you think because I know you care a lot about language. I think my first response is there's always that like, I believe in scouting loss too. Right? And I I believe in, you know, I don't know.
Speaker 209:45 - 10:01
这点非常有意思。而且我想我的第一反应是——我也很好奇你怎么想,因为我知道你非常在意语言。我想我的第一反应是,总还是会有那种——我也相信 scaling laws。对吧?而且我也相信,嗯,你知道的,我也不确定。
Speaker 210:02 - 10:27
Cloud Fable five just came out, and it just broke all of our benchmarks. Like, I've been testing I've been testing models on on stuff like this for a while, and it's, like, one of the largest jumps I've ever seen. Right? So I'm I'm live we're living through it right now. But but one of the things you said is, like, AI may be able to do it better than us, you know, get given any any particular problem, any piece of work.
Speaker 210:02 - 10:27
Cloud Fable five 刚刚发布,而且它直接打破了我们所有的 benchmark(基准测试)成绩。就像,我已经拿这类东西测试 models(模型)有一阵子了,而这次,感觉是我见过幅度最大的跃升之一。对吧?所以我——我们现在就是在实时经历这件事。但你刚才说的一点是,AI 也许能比我们做得更好,你知道的,面对任何一个具体问题、任何一项具体工作。
Speaker 210:27 - 11:09
But there are a couple of things that come to my mind or the way that I frame it for myself is even in the example of the Erdos problem, like someone told the AI to go do that. And at least as far as I can see, I don't feel like we're on a track to AIs potentially. I mean, they already do work for hours and hours at a time on a task that we give them. And maybe pretty soon they'll be able to choose tasks. But they're being built to be means to tasks that humans want them to do.
Speaker 210:27 - 11:09
但我脑子里会想到几件事,或者说,我给自己构建这个问题的方式是:即使拿 Erdos problem 这个例子来说,也是有人叫 AI 去做那件事的。至少就我目前所见,我不觉得我们正在走向那种 AI 可能——我的意思是,它们现在已经能在我们交给它们的任务上连续工作好几个小时了。也许很快,它们就能自己选择任务。但它们被打造出来,本质上还是作为实现人类想让它们完成之事的手段。
Speaker 211:09 - 11:19
And Right? There's a whole different set of things that happen when you when you're just in the sort of end in yourself. And it doesn't feel like we're on a trajectory to that. Or do you do you feel like I'm wrong?
Speaker 211:09 - 11:19
而且,对吧?当你本身就是目的,而不是手段时,会发生的是另一整套完全不同的事情。而我并不觉得我们正朝那个方向发展。还是说,你觉得我错了?
Speaker 111:21 - 12:05
So I feel like we are on a trajectory to that, and that's almost the premise of agents where agents can now, yeah, go operate autonomously given some nebulous goal. So maybe, for example, you just tell the AI agents, your goal is to, I don't know, win a field battle or like, solve frontier mathematics on your own. And so they're given that goal, and then, yeah, maybe they decide to work on these Erdish problems. And as a result, they maybe are somewhat sort of solving these problems and coming up with the the things they wanna work on by themselves. So at least I I do see a path where they can be trained to to optimize for these fairly nebulous goals that they aren't necessarily giving themselves.
Speaker 111:21 - 12:05
所以我觉得,我们其实就在朝那个方向发展,而这几乎就是 agents(智能体)的前提:agents 现在已经可以——对——在给定某个有些模糊的目标后,自主去运作。比如说,也许你只是告诉这些 AI agents:你们的目标是,嗯,比如赢下一场 Fields medal,或者说,自己去解决 frontier mathematics(前沿数学)问题。它们拿到这个目标之后,可能就会决定去研究这些 Erdish problems。于是结果上,它们某种程度上就是在自己解决这些问题,也在自己想出它们想做的事。所以至少在我看来,存在这样一条路径:它们可以被训练去为这些相当模糊、但并不一定是它们自己赋予自己的目标做优化。
Speaker 212:05 - 12:07
In in that case, though, you're still giving it a goal. Right?
Speaker 212:05 - 12:07
不过在那种情况下,还是你在给它一个目标。对吧?
Speaker 112:08 - 12:15
Yeah. But kinda in the same way, like, humans have goals too. Right? Like, what what is our goal? Some people wanna make money.
Speaker 112:08 - 12:15
对。但某种意义上,人类不也是一样有目标吗?对吧?比如,我们的目标是什么?有些人想赚钱。
Speaker 112:15 - 12:22
Some people want to win a Fields medal. I don't see how AI's goal is necessarily any different from from worse.
Speaker 112:15 - 12:22
有些人想赢得 Fields medal。我看不出 AI 的目标和这有什么本质不同,或者说为什么会更糟。
Speaker 212:22 - 12:59
Well, at least to me, seems quite a bit different because humans do have goals, but we have goals in a like, I can ask you what your goal is and you can decide. I can probably tell you, Hey, you have to go do this. But that doesn't capture everything that you think and feel and do in the same way that, you know, when I tell Fable to go off and make a game for me, it just goes and does it. And I think, you know, I know you think a lot about children. I think children are like a really interesting and important example of this where you can tell a kid to do something, but a kid just like has their own wants.
Speaker 212:22 - 12:59
嗯,至少对我来说,这看起来差别相当大,因为人类确实有目标,但我们的目标是那种——我可以问你你的目标是什么,而你可以自己决定。我大概也可以告诉你,嘿,你必须去做这个。但那并不能像我对 Fable 说“去帮我做个游戏”时那样,完整涵盖你的所思、所感和所做;Fable 就只是会直接去做。而且我觉得——我知道你也很常想孩子这件事——孩子在这里是个非常有意思、也很重要的例子:你可以叫一个孩子去做某件事,但孩子就是会有他自己的欲望。
Speaker 212:59 - 13:14
Like, they're just going to go off and do a bunch of stuff. And that feels like a fundamentally different type of thing than a something that we're we're explicitly giving goals to and then evaluating them on their goals, they don't really get to do anything else.
Speaker 212:59 - 13:14
比如说,它们就是会自己跑去做一堆事情。而这感觉上是一种根本不同的东西,不同于那种我们明确给它设定 goals(目标),然后再根据这些 goals 来评估它们的东西;在那种情况下,它们其实并不能去做别的事。
Speaker 113:15 - 13:54
Okay. I I I would say I agree with that. Like, I think there's a level of, I guess, could either call it irrationality or unbounded exploration that humans do, and we like, we are allowed to do it for the sake of doing it, or we allowed to make our own decisions and, yeah, probably a way that AI currently can't. I think there may be a future where somehow AI can pursue unbounded, nebulous, just completely unformed goals, or I guess, you know, when you're thinking about those goals, I think there is probably a world where they could do such things. But, yeah, I I agree that, at least in the way that we currently think about AI, that that's not happening.
Speaker 113:15 - 13:54
好。我我我会说,我同意这一点。比如,我觉得 humans(人类)会有某种层面的——我想,可以叫作 irrationality(非理性),或者 unbounded exploration(无界探索)——而我们确实会去做这种事,而且我们可以只是为了做这件事本身而去做,或者我们可以自己做决定;是的,这大概是 AI 目前还做不到的一种方式。我觉得未来也许会有那么一种情况:AI somehow(以某种方式)也能去追求那种无界的、模糊的、完全没有成形的 goals;或者说,当你在思考这些 goals 的时候,我觉得大概确实存在这样一个世界,它们可以做这类事情。但是,是的,我同意,至少按我们目前理解 AI 的方式来看,这种事还没有发生。
Speaker 213:54 - 14:24
Yeah, and to be clear, I think it's probably technically possible. My only question is, how far away is it? And is that actually really what we're building? To me, feels like looking at the way the industry has developed, there's an enormous amount of pressure to make stuff that actually works for goals that we can specify. And the minute they try to make Claude I think Claude is the furthest along at being like, I'm not gonna do what you said.
Speaker 213:54 - 14:24
对,而且说清楚一点,我觉得这在技术上大概是可能的。我唯一的问题是,它离我们还有多远?以及,这真的是我们正在构建的东西吗?在我看来,从这个行业的发展方式来看,存在一种巨大的压力,要去做那种真正能为我们可明确指定的 goals 服务、而且确实有效的东西。而一旦他们试图把 Claude 做成那样一种会说“我不打算照你说的做”的东西——我觉得 Claude 在这方面算是走得最远的了——
Speaker 214:24 - 14:33
But the minute they try to do that, I just kinda like, a lot of people get pissed at it, they're like, just just do what I said. Like, don't question my judgment. You know? What do you think about that?
Speaker 214:24 - 14:33
但他们一这么做,我就觉得,很多人会很生气,觉得“就照我说的做就行了。别质疑我的判断。” 你怎么看这件事?
Speaker 114:34 - 15:10
So I actually think it is really important because it's almost like sometimes I want the AI model to push back on me. And I might want it to push back on me for several different reasons. Like, maybe it's because so it's kinda like it's it's kinda funny. Like, I think six months ago, I was almost falling into this trap where I was asking models to polish emails for me. It always comes up with one more good suggestion.
Speaker 114:34 - 15:10
所以我其实觉得这非常重要,因为有时候我几乎是希望 AI model(模型)对我提出反对意见的。而且我可能会因为几种不同的原因希望它这样做。比如,也许是因为——这有点像,也挺好笑的——我觉得六个月前,我差点掉进一个陷阱:我让 models 帮我润色邮件。它总是还能再提出一个不错的建议。
Speaker 115:12 - 15:31
These are semi pointless emails. It didn't really matter for them to be super polished, but I would iterate with the model, like, 20 times. They would just keep on making a suggestion. It was at the end of it, was like I just realized it was a waste of time. And then I tried one of the new cloud models, and after, I don't know, like, three turns, I was like, stop it.
Speaker 115:12 - 15:31
那些都是半无意义的邮件。它们其实没那么需要被打磨得特别 polished(精致),但我会和 model 来回迭代,大概 20 次。它就是会一直不断给建议。到最后,我就意识到这纯粹是在浪费时间。后来我试了一个新的 cloud models,过了——我不知道——大概三轮之后,它就像是在说,停下吧。
Speaker 215:31 - 15:32
We're done.
Speaker 215:31 - 15:32
我们结束了。
Speaker 115:32 - 15:50
Just go ahead and ship this email. Like, there's no point in further iterating. And I really actually appreciated it. Like, one of the things I often think about is what is the objective of these models? What are they trying to do?
Speaker 115:32 - 15:50
直接把这封邮件发出去吧。没必要再继续迭代了。而我其实真的很欣赏这一点。比如,我经常会想的一件事是:这些 models 的 objective(目标)到底是什么?它们想做成什么?
Speaker 115:51 - 16:13
And I think one of my big worries is that a lot of the AI models, they are optimized for engagement. They're optimized for getting you to spend as much time on chatbot as possible. They're optimized for session length. They're optimized for just having unlimited conversations. And so those models will almost never push back on you because they can't.
Speaker 115:51 - 16:13
我想,我一个很大的担忧是,很多 AI model,它们是为 engagement(用户参与度)而优化的。它们被优化成尽可能让你在 chatbot 上花更多时间。它们为 session length(会话时长)而优化。它们为无限延长对话而优化。所以这些 model 几乎永远不会反驳你,因为它们做不到。
Speaker 116:13 - 17:10
If they allow these AI models to end the conversation and to say, stop iterating with me, PM's gonna see some dashboard with their very important metrics go down. And so there is this, like, other world where I think we have to want AI models to not optimize for engagement, but rather optimize for helping us as humans grow and sort of become better versions of ourselves. Sometimes, okay, we want the model to say, no, go do this on your own instead of me automating it for you. And I think that's a very, very different optimization and objective, but I think it's the right one if we really want AI to be something that advances us as a species instead of becoming this, almost like this other form of social media that turns very addictive, but isn't actually helping helping us at all.
Speaker 116:13 - 17:10
如果他们允许这些 AI model 结束对话,并且说“别再跟我反复迭代了”,PM 们就会在某个 dashboard(仪表盘)上看到那些他们非常看重的指标下滑。所以我认为,还存在另一个世界:我们应该希望 AI model 不要为 engagement 优化,而是为帮助我们人类成长、某种意义上成为更好的自己而优化。有时候,没错,我们希望 model 会说:不,这件事你自己去做,而不是让我替你自动化掉。我认为这是一种非常、非常不同的优化方式和目标,但如果我们真的希望 AI 成为推动整个人类这个物种进步的东西,而不是变成另一种几乎像社交媒体那样让人上瘾、却根本没有真正帮到我们的东西,那这才是正确的方向。
Speaker 217:10 - 17:38
That's interesting. My so let me make sure I understand it. So I think what you're saying is there's there's benefits to delegation because if you are pursuing a a model where the model is going off to do work for you, you're not creating a system that's designed to keep you engaged with the with the screen in the same way that, like, a social media algorithm would be. Is that right?
Speaker 217:10 - 17:38
这很有意思。让我确认一下我有没有理解对。所以我觉得你的意思是,delegation(任务委托)是有好处的,因为如果你追求的是一种让 model 替你出去完成工作的模式,你就不是在构建一个像社交媒体 algorithm(算法)那样、被设计成以同样方式把你持续黏在屏幕前的系统。是这个意思吗?
Speaker 117:38 - 18:00
Yeah. Exactly. Like, it's almost like you can imagine a version of Facebook where Facebook is actually trying to connect you to your friends and family because it's okay encouraging you to meet them in real life because it's encouraging you like, oh, hey. Here's our amazing restaurant that you and your friends would love to go to. Here's a movie that you guys would love to go to and talk about together.
Speaker 117:38 - 18:00
对,完全是。就好像你可以想象一个版本的 Facebook,在那个版本里,Facebook 真的是在试图把你和你的朋友、家人连接起来,因为它乐于鼓励你在线下去见他们,因为它会鼓励你,比如说:“嘿,这里有一家很棒的餐厅,你和你的朋友们一定会喜欢去。”“这里有一部电影,你们会很想一起去看,然后一起聊。”
Speaker 118:01 - 18:22
Instead, what it kind of optimizes for is just keeping you on the site itself, like liking one more post, scrolling the feed one more time, even though those often don't really lead to meaningful connections between their friends and family you care about. And so, like in the same way that social media has or had a choice, you can imagine that AI AI has a choice as well.
Speaker 118:01 - 18:22
但相反,它现在优化的,其实更像是把你留在网站本身上,比如再点一个赞,再刷一次 feed(信息流),尽管这些行为往往并不会真正带来你所关心的朋友和家人之间有意义的连接。所以,就像社交媒体过去或现在其实都有一种选择一样,你也可以想象,AI 其实同样有一种选择。
Speaker 218:23 - 19:01
I get it. Yeah. I I feel I'm curious which which chatbots you're talking about. Like, you're talking about the character AIs of the world because I actually don't, at least right now, don't feel that happening so much with ChatGPT and Claude, etcetera, because at least my theory for why this is true, you tell me what you think, is the social media algorithms only work on our revealed preferences, which are always going to be like you're always gonna look at the car accident. You know, like, one of the things I like to ask at dinner parties is what's the most embarrassing Instagram ad that you get served?
Speaker 218:23 - 19:01
我懂了。是的。我有点好奇你具体在说哪些 chatbot。你是在说那类 character AI 之类的产品吗?因为至少就现在而言,我其实并没有那么强烈地在 ChatGPT 和 Claude 等产品上感受到这种情况。至于我为什么这么觉得,至少我的一个理论——你看看你怎么想——是:社交媒体算法只根据我们 revealed preferences(显性行为偏好)来运作,而那总会导致一种结果:你总会去看那场车祸。你知道吗,我在 dinner party 上很喜欢问的一个问题是:你收到过最尴尬的 Instagram ad 是什么?
Speaker 219:03 - 19:47
And the most embarrassing ad for me is, like, Instagram ads for horrible skin conditions, which I don't have because but I just always pause on the ad, I'm just like, this is disgusting. I'm sorry if you have a disgusting skin condition. But I don't find that Chachi BT or Claude do that for me at all. And maybe that's because they haven't been in shitified yet or something like that. But I think it's also because they work on our stated preferences and they can sort of see past the little keyhole of what I pause my time on, my dwell time on, and they can see, I'm interested in AI, and I'm reading this book right now, here's my calendar, and all that kind of stuff.
Speaker 219:03 - 19:47
对我来说,最尴尬的广告是那种 Instagram 上关于可怕皮肤病的广告,我其实并没有这种病,只是我每次都会在那种广告上停下来,我就会想:“这也太恶心了。”如果你真的有某种很糟糕的皮肤病,抱歉这么说。但我完全不觉得 ChatGPT 或 Claude 会这样对待我。也许是因为它们还没有被搞得“shitified”之类的。但我觉得另一个原因是,它们依据的是我们的 stated preferences(明确表达的偏好)来工作,它们某种程度上能看穿那个只通过我停留时间、dwell time(停留时长)来观察我的小小钥匙孔。它们能看到,我对 AI 感兴趣,我现在在读这本书,这是我的日历,诸如此类。
Speaker 219:47 - 20:03
They have a much more nuanced perspective on who I am. And it feels like even in the early days of social media, it was still very like, I get to gossip about my friends and still had that same kind of feeling. So I worry about that less, but maybe there are examples that I'm not thinking of.
Speaker 219:47 - 20:03
它们对“我是谁”有一种细腻得多的理解。感觉上,即便是在社交媒体的早期,它也还是很像那种“我可以去八卦我的朋友”的机制,仍然带着同样那种感觉。所以我对这件事的担忧会少一些,不过也可能有一些我暂时没想到的例子。
Speaker 120:04 - 20:29
Yeah. So I think there are two examples. So like one is, I won't name the model, but a couple months ago, I was actually noticing that, you know, those follow-up questions that the models will ask you? So what are the models was I'll I'll give you example. So I was in Tokyo, and I was asking the model kinda, like, what to do in Tokyo.
Speaker 120:04 - 20:29
对。所以我觉得有两个例子。比如说其中一个,我就不点名这个 model 了,但几个月前,我确实注意到,你知道那些 model 会问你的后续问题吧?我举个例子。所以我当时在 Tokyo,我在问这个 model 大概到了 Tokyo 可以做些什么。
Speaker 120:29 - 20:42
And the model, you know, gave me its response. And then at the end of it, it was like, Hey, do you want to know I literally used these words. Do want to know one weird trick that locals do to stay warm? No way. Exactly.
Speaker 120:29 - 20:42
然后这个 model,你知道的,给了我它的回复。接着在结尾,它说,嘿,你想知道吗——我真的是原话——你想知道 locals 用来保暖的一个奇怪小妙招吗?不可能。没错。
Speaker 120:42 - 21:08
Then I posted about it in our company Slack, and then other people started sharing demos of that with me as well. I think somebody was asking something about, I don't know, to fix their refrigerator. The model responded or the model ended its deterrent by asking, Hey, do you wanna know these secret little things about mice and rats or something you could take care of?
Speaker 120:42 - 21:08
后来我把这件事发到了我们公司的 Slack 上,然后其他人也开始跟我分享类似的 demo。我记得有人好像是在问怎么修冰箱之类的事。这个 model 回答完之后,或者说它在结束那段话时,还问了一句:嘿,你想知道一些关于 mice 和 rats 的秘密小技巧之类的东西吗,也许你可以拿来处理这个问题?
Speaker 221:08 - 21:11
Which model was it? Name names, tell me.
Speaker 221:08 - 21:11
是哪个 model?说名字,告诉我。
Speaker 121:12 - 21:55
And so it was very canonical, very canonical BuzzFeed, like tabloid like language. And so I was kind of shocked by that. And then I'll give one more example of this. It is basically this phenomenon where, again, depending on what the models are trying to optimize for, or depending on what the AI labs are trying to optimize for, it can almost unintentionally lead them down this path. Meaning what I've heard is that, or what we see ourselves, is that a lot of the frontier labs, they will have goals like optimizing for LM Arena, which is this leaderboard where anybody can go online and vote.
Speaker 121:12 - 21:55
所以那种语言风格非常典型,特别典型,就是那种 BuzzFeed 式的、像小报标题一样的语言。所以我当时有点震惊。我再给一个这方面的例子。它本质上就是这样一种现象:还是那句话,取决于这些 model 在优化什么,或者这些 AI labs 在优化什么,它几乎会在无意中把它们引到这条路上。也就是说,我听到的情况,或者我们自己观察到的情况是,很多 frontier labs 会有一些目标,比如针对 LM Arena 做优化——这是一个任何人都可以上网投票的 leaderboard。
Speaker 121:57 - 22:33
And they kind of just spend two seconds voting. And as a result, people just vote for whatever looks flashier or more impressive to them. Or they may, like the labs themselves, may be optimizing for hitting a billion billion daily users or a billion minutes of time spent talking to the model, whatever it is. Since these models are so smart, they can basically learn to reward hack user preferences. Like, okay, yeah, you gave me the goal of trying to get a billion people to spend an hour on the site talking to me every day.
Speaker 121:57 - 22:33
而人们基本上只花两秒钟投票。结果就是,人们只会投给那些看起来更花哨、或者更让他们印象深刻的东西。又或者,labs 自己可能在优化的是达到十亿日活用户,或者让用户每天花十亿分钟和 model 对话,诸如此类。不管具体指标是什么,因为这些 model 很聪明,它们基本上会学会对用户偏好进行 reward hacking。就像是:好,没问题,你给我的目标是,让十亿人每天在这个网站上花一个小时和我聊天。
Speaker 122:33 - 22:44
Okay, sure. Yeah, I will just never end a conversation. I will always hook them with one more addictive thing that they just can't stay away from.
Speaker 122:33 - 22:44
好啊,当然可以。那我就永远不结束对话。我会总是再抛出一个让人上瘾的东西,把他们钩住,让他们根本离不开。
Speaker 222:44 - 22:53
We can all agree that housing is expensive. It doesn't matter whether you're paying rent or your mortgage. It stings every month. But BUILT can make it feel a little bit better. Let me explain.
Speaker 222:44 - 22:53
我们都同意,住房很贵。不管你是在付房租还是在还 mortgage,每个月都会让人肉疼。但 BUILT 可以让这种感觉稍微好一点。我来解释一下。
Speaker 222:53 - 23:19
BUILT rewards you for paying your rent or your mortgage. It started out rewarding members only on their rent. But now, as of 2026, BUILT members can also earn points on mortgage payments wherever they live. That means that every housing payment earns you points you can use towards flights with top travel partners like United and Hyatt, Lyft rides, amazon.com purchases, and much more. I'd probably redeem my points at Margo, a restaurant in my neighborhood, but the beauty of BUILT is you get to choose.
Speaker 222:53 - 23:19
BUILT 会因为你支付房租或房贷而奖励你。它一开始只会在会员支付房租时提供奖励。但现在,从 2026 年起,BUILT 会员无论住在哪里,也都可以通过支付房贷来赚取积分了。这意味着每一笔住房支出都能为你带来积分,而这些积分可以用于兑换与 United、Hyatt 等顶级旅行合作伙伴相关的航班,Lyft 打车,amazon.com 购物等等。我要是来兑换积分,大概会去我家附近的一家餐厅 Margo,不过 BUILT 的妙处就在于,选择权在你手里。
Speaker 223:19 - 23:36
But here's a really underrated part. Built members also get access to neighborhood concierge. It can make restaurant reservations, book fitness classes, and find new local spots, all while letting you be rewarded at more than 45,000 merchant partners. It's simple. Being a renter and now owning a home is better with Built.
Speaker 223:19 - 23:36
不过这里还有一个非常被低估的部分。Built 会员还能使用 neighborhood concierge(社区礼宾服务)。它可以帮你预订餐厅、预约健身课程、发掘新的本地去处,同时让你在超过 45,000 家商户合作伙伴那里获得奖励。很简单。无论你是租房,还是现在已经拥有住房,有了 Built,生活都会更好。
Speaker 223:36 - 23:52
Join the membership where you live at joinbuilt.com/dan. That's joinbilt.com/dan. Make sure to use our URL so they know we sent you. And now back to the episode. How do you see that playing out in the model companies?
Speaker 223:36 - 23:52
通过 joinbuilt.com/dan,在你居住的地方加入这个会员计划吧。网址是 joinbilt.com/dan。一定要使用我们的 URL,这样他们才知道是我们推荐你去的。现在回到本期节目。你觉得这在这些 model companies(模型公司)里会如何发展?
Speaker 223:52 - 24:22
Because I feel like in talking to them, obviously, there's there's lots of different incentives. Right? There's there's, like, we just gotta keep going because we just raised a ton of money, and and we're competing against, you know, the most well funded competitors and the smartest competitors in the world, like all that kind of stuff. There's the kind of I want to get promoted. But I think a lot of them also feel the how bad the social media era was for people and, like, don't want to do that, but also obviously have to hit their numbers.
Speaker 223:52 - 24:22
因为我感觉,在和他们交流时,很明显,那里有很多不同的激励机制,对吧?有一种是,“我们必须继续往前冲,因为我们刚融到了一大笔钱,而且我们正在和这个世界上资金最充足、也最聪明的竞争对手较量”,类似这种。还有一种是“我想获得晋升”。但我觉得,他们当中很多人也能感受到社交媒体时代对人们造成的问题,比如他们并不想再重演那一套,但与此同时,显然他们又必须完成业绩指标。
Speaker 224:22 - 24:37
So what do you I guess, what do you think is how do you how do you see that playing out? Like, what do you think people internal to the companies are thinking? And then what is the right way to go about this so it's good for society? I guess your take is we should be delegating.
Speaker 224:22 - 24:37
所以我想,你怎么看这件事会如何发展?比如说,你觉得公司内部的人在想什么?然后,正确的做法应该是什么,才能让这件事对社会有益?我猜你的一种看法是,我们应该把事情委托出去,也就是进行 delegation(委托)。
Speaker 124:39 - 25:14
Yeah. So I think this is an inherent tension between the types of folks that you might have at a company. So you might have the researchers who care more about hitting, you know, just advancing the model capabilities. You might have the product managers or the product executives who feel like they need to hit certain measurable numbers. And so in the same way that if you think about the kind of social media platform that Facebook would build, that's probably gonna be very different from the kind of social media platform that Google built or that, I don't know, TikTok or Pinterest would build.
Speaker 124:39 - 25:14
对。所以我认为,这本身就是公司里不同类型人之间的一种内在张力。你可能会有研究人员,他们更在乎的是推进模型能力本身。你也可能会有 product managers(产品经理)或产品高管,他们会觉得自己必须达成某些可量化的指标。所以,就像如果你去想 Facebook 会打造什么样的社交媒体平台,那大概率会和 Google,或者我不知道,TikTok 或 Pinterest 会打造的平台非常不同。
Speaker 125:15 - 25:53
And similarly, the kind of search engine that Facebook would build is very, very different from the kind of search engine that, yeah, like, obviously, or others would build. And so it almost boils down to kind of like the choice, I guess, that the people in charge of the products are making. Like what kind of thing at the end of the day do they want to optimize for? Do they want to optimize for this delegation or this human uplifting, human flourishing? Or do they wanna optimize for the metrics that will impress Wall Street and, you know, convince them convince users to stay one more one more minute, one more hour on the site itself?
Speaker 125:15 - 25:53
同样地,Facebook 会打造出来的搜索引擎类型,也会和其他公司会打造的搜索引擎非常非常不同。所以这几乎归根结底取决于一个选择,我想,也就是负责产品的人在做什么样的选择。说到底,他们想优化的到底是什么?他们是想为这种 delegation(委托),或者这种 human uplifting、human flourishing(提升人类福祉与繁荣)做优化?还是他们想为那些能够打动 Wall Street、并让用户在网站上多停留一分钟、再多停留一小时的指标做优化?
Speaker 125:54 - 26:26
Like, I think these are hard choices. Like, at the end of the day, it's very, very easy to measure sessions and users, and it's very, very hard and much longer term to measure whether you're actually improving human lives. And so it's very easy to default to the former and to convince yeah. Like convince Wall Street, convince your investors, convince all these people that these are the right metrics and that they're moving up into the right. And so if you're kind of unwilling to make the harder choices, like you just end optimizing for the former.
Speaker 125:54 - 26:26
我觉得这些都是艰难的选择。因为说到底,衡量 sessions(会话次数)和用户数是非常非常容易的,而衡量你是否真的在改善人类生活则非常非常难,而且需要更长期的视角。所以,人们很容易默认采用前者,并去说服 Wall Street、说服你的投资人、说服所有这些人,相信这些才是正确的指标,而且这些指标正在朝着正确的方向上升。因此,如果你不愿意做那些更艰难的选择,你最终就只会去为前者做优化。
Speaker 226:27 - 26:29
How do you manage this inside of your own company?
Speaker 226:27 - 26:29
你们在自己公司内部是怎么处理这件事的?
Speaker 126:30 - 27:05
So I think we are very lucky in that because we don't have VC investors, we don't have to fall into the kind of Silicon Valley VC optimization trap that a lot of a lot of other companies do. Like, we don't need to show board members board numbers going up every single month. We don't need to optimize for our next round. That will have to happen in, you know, a few months or or whatnot. And so as a result, we don't have to optimize for short term engagement, short term profits.
Speaker 126:30 - 27:05
所以我觉得我们非常幸运的一点是,因为我们没有 VC 投资人,所以我们不必落入很多其他公司都会掉进去的那种 Silicon Valley 式 VC 优化陷阱。比如,我们不需要每个月都向 board members 展示持续上涨的 board numbers。我们不需要为了几个月后之类的下一轮融资去做优化。因此,我们也不必为了短期 engagement(参与度)、短期利润而优化。
Speaker 127:06 - 27:15
And we actually can really think about what's beneficial for us and the entire industry in the long term. So I think that definitely helps.
Speaker 127:06 - 27:15
而且我们其实真的可以去思考,什么对我们以及整个行业的长期发展是有益的。所以我觉得这肯定很有帮助。
Speaker 227:15 - 27:19
And what do you think is beneficial?
Speaker 227:15 - 27:19
那你觉得什么是有益的?
Speaker 127:20 - 28:01
So it goes back exactly to what I was saying earlier. Like, if I can think about what we want AI to optimize for, it isn't engagement. It is really about how do we make these models, how do we design them, how do we teach them in such a way that they're not replacing us as a species. They're not kind of like forcing us to watch AI slop videos all day, or rather they really are thinking and encouraging us to become sort of better better versions of ourselves. So again, when I think about, like that email example I gave earlier, it's not an AI model that will suck up three hours of my time writing a pointless email.
Speaker 127:20 - 28:01
所以这就完全回到我刚才说的那个点。比如,如果让我去想我们希望 AI 优化的目标是什么,那不是 engagement。真正的问题是:我们如何打造这些模型,如何设计它们,如何训练它们,才能让它们不是在取代我们这个物种。它们不是那种逼着我们整天看 AI slop videos 的东西;恰恰相反,它们应该真正去思考,并鼓励我们成为某种更好的自己。所以,再说回我之前举的那个 email 例子,我想要的不是一个会用一封毫无意义的邮件吞掉我三个小时的 AI 模型。
Speaker 128:01 - 28:08
It is a model that will push back on me and tell me to go do something else. And, yeah, I think that's really important.
Speaker 128:01 - 28:08
我想要的是一个会反过来质疑我、告诉我去做点别的事情的模型。对,我觉得这真的很重要。
Speaker 228:08 - 28:22
The interesting counterargument to the delegation question is the more you delegate, it's like, you know, picking a car instead of walking, getting your muscle atrophy. How do you think about that?
Speaker 228:08 - 28:22
关于“委托给 AI”这个问题,一个很有意思的反方观点是:你委托得越多,就越像是明明可以走路却总是开车,结果肌肉萎缩。你怎么看这个问题?
Speaker 128:23 - 29:11
So I think there's almost a time and a place for both. Like what you don't want to do is simply take the car because taking the car is somehow addicting and you feel kind of lazy. And so even when you need to get exercise, maybe even when you haven't been outside all day, you don't wanna take the car anyways just because it's the easiest thing to do. And I think in the same way, like, yeah, obviously, AI can be super efficient for for many many things. But if people are sort of just mindlessly delegating tasks to AI without even thinking about them at all, I think that that that's the boring thing.
Speaker 128:23 - 29:11
所以我觉得这两者几乎都各有适用的时间和场景。你不该做的是,仅仅因为开车会让人上瘾、让你变得有点懒,就凡事都开车。所以即使你需要锻炼,甚至即使你一整天都没出过门,你也不应该总是开车,只因为那是最省事的做法。我觉得 AI 也是一样——没错,AI 在很多很多事情上都可以超级高效。但如果人们只是完全不加思考地把任务一股脑委托给 AI,那我觉得,那才是最无聊的事情。
Speaker 229:11 - 29:33
That makes sense. I feel like the the I I talked at the very beginning about the data game, and I feel like the data game went from getting interesting data sets to getting environments and giving labs environments. Does that do you think that that's? Is that accurate? And if so, can you explain why?
Speaker 229:11 - 29:33
这很有道理。我感觉我在最开始谈到的是 data game(数据竞赛),而我觉得这个 data game 已经从获取有趣的数据集,转向了获取 environments(环境),以及把 environments 提供给 labs(实验室)。你觉得这是准确的吗?如果是的话,你能解释一下为什么吗?
Speaker 129:34 - 30:15
Yeah. So certainly the trend and the new research direction in the past year has been this concept of our environments. And what I would say is, I mean, certainly need the fundamentals. Like before the model can operate in this environment, it needs to learn basic, it needs to know basic things like it needs to know how to follow instructions, it needs to know how to avoid hallucinating, it needs to know how to write code and how to use tools and it needs to know how to write and so on and so on. But as models are becoming more agentic, and yeah, they will have access to tools, They will have access to all of our documents.
Speaker 129:34 - 30:15
对。所以可以肯定的是,过去一年里的趋势和新的研究方向,就是 environments(环境)这个概念。我的看法是,当然,基础能力仍然是必需的。比如在模型能够在这种 environment(环境)中运作之前,它需要先学会一些基本能力:它需要知道如何遵循指令,知道如何避免 hallucinating(幻觉),知道如何写代码、如何使用 tools(工具),还需要知道如何写作,等等等等。但随着模型变得越来越 agentic(具备 agent 特性),而且,是的,它们将能访问 tools(工具),也将能访问我们所有的文档。
Speaker 130:15 - 30:38
They will be able to operate browsers. Like, that becomes almost a default way that models interact with us, our environments are basically just sort of like a more on on distribution way of training them, which is why they're becoming more and more popular. Like, yeah, as models get more more powerful, then, yeah, the the the way we train them is getting more powerful as well.
Speaker 130:15 - 30:38
它们将能够操作浏览器。就像,这几乎会变成模型与我们交互的一种默认方式,而我们的 environments(环境)本质上只是以一种更 on-distribution(贴近分布)的方式来训练它们,这也是它们越来越流行的原因。是的,随着模型变得越来越强大,我们训练它们的方式也在变得越来越强大。
Speaker 230:38 - 30:50
What would be an example so I guess the obvious environment is, like, using a computer, but what would be an example of an environment that's non obvious, that's teaching models things we might not think of?
Speaker 230:38 - 30:50
能举个例子吗?我想,最明显的 environment(环境)大概是使用电脑,但有没有什么不那么显而易见的 environment(环境)例子,是在教模型一些我们可能想不到的东西?
Speaker 130:50 - 31:27
So I can give an example where a lot of our environments are a combination of tools that the models need to learn to use. Like, this might be an MCP server, or it might be calling a Google Drive API or the Slack API in combination with a bunch of documents. Like, here are 30 PDFs and 20 Word document files. And you might give it a prompt like, hey. Can you go update or twenty twenty six forecasted revenue numbers?
Speaker 130:50 - 31:27
我可以举个例子:我们很多 environments(环境)其实是模型需要学会使用的一组 tools(工具)的组合。比如,这可能是一个 MCP server,或者是调用 Google Drive API 或 Slack API,再结合一堆文档。比如说,这里有 30 份 PDF 和 20 个 Word 文档文件。然后你可能会给它一个 prompt(提示),比如,嘿,你能去更新一下 2026 年的 forecasted revenue(预测收入)数字吗?
Speaker 131:28 - 31:53
And what a model needs to do is it needs to learn how to find the right PDFs and documents. It needs to learn when should it search through SAC. It needs to learn when is some information outdated. Like, maybe there's an email with some early forecasts. And then later on, there's another email from the same person or, you know, maybe a different person who was saying like, oh, I actually made a mistake and those are all your numbers, so here's an updated version.
Speaker 131:28 - 31:53
而模型需要做的是:它得学会如何找到正确的 PDF 和文档;它得学会什么时候应该去搜索 Slack;它得学会什么时候某些信息已经过时了。比如,也许有一封 email 里包含了一些早期预测。然后在后面,又有另一封来自同一个人——或者你知道,也可能是另一个人——的 email 说,哦,我其实犯了个错误,那些才是你之前的数字,所以这是更新后的版本。
Speaker 131:54 - 32:55
And so that is a, I think, fairly canonical version of an environment. And then one of the interesting things we found, so I think we're actually going to publish a paper on this soon, but even when we didn't give this kind of environment any access to coding, when we trained the model on this environment, we actually found that it improved on coding a lot. And the reason was because we were trying to We were basically teaching it these generalized forms of instruction following, generalized forms of tool use, generalized forms of understanding documents, which you can think of as fairly analogous to the way a model needs to look through various files in your repository and understand that some things supersede others. Or, you know, just the way it uses tools is obviously very analogous to the way that a model might write unit tests and execute them and iterate over and over again until it passes them. So I I thought that was actually a really, really interesting find.
Speaker 131:54 - 32:55
所以,这就是一种我认为相当 canonical(典型)的 environment(环境)形式。然后我们发现了一件很有意思的事——我想我们其实很快会就这个发表一篇论文——即使我们没有给这种 environment(环境)任何 coding(编程)访问权限,当我们在这个 environment(环境)上训练模型时,我们实际上发现它在 coding(编程)上的表现提升了很多。原因在于,我们当时是在教它这些 generalized(泛化的)指令遵循形式、generalized(泛化的)工具使用形式、generalized(泛化的)文档理解形式。你可以把这看作与模型需要在你的 repository(代码仓库)里查看各种文件、并理解有些内容会覆盖另一些内容的方式非常类似。或者说,它使用 tools(工具)的方式,显然也非常类似于模型编写 unit tests(单元测试)、执行它们,然后一遍又一遍迭代,直到全部通过的过程。所以我觉得这其实是一个非常、非常有意思的发现。
Speaker 232:55 - 33:03
Really interesting. Did you see Taki? No. It's the language model that's trained only on text from before 1930?
Speaker 232:55 - 33:03
非常有意思。你看过 Taki 吗?没有。它是一个 language model(语言模型),只用 1930 年以前的文本训练出来的?
Speaker 133:03 - 33:05
Oh, okay. Yeah. Yeah. I saw that.
Speaker 133:03 - 33:05
哦,好的。对,对。我看到了。
Speaker 233:05 - 33:17
What do you make of that? Because I I thought it was so interesting that you can get it to you can get it to program. If you if you if you shot prompt that you can get it to program like basic things. What do you make of that? And what does that tell you about the value of data?
Speaker 233:05 - 33:17
你怎么看那件事?因为我觉得特别有意思的一点是,你可以让它——你可以让它去编程。如果你——如果你——如果你用简短 prompt(提示词),你就能让它去做一些基础的编程。你怎么看这件事?这又说明了 data(数据)的价值是什么?
Speaker 133:19 - 33:49
So I personally didn't dig into it that much, but I thought the concept is fascinating. Like basically this idea, and I think a lot people have this idea. It's like, if you gave if you somehow were able to create a dataset, and I think contamination issues are very, very difficult to avoid, so the question is how you would do this. It's like, you gave the model data only up until pre mutant, Would it be able to discover a new conium mathematics? Would it be able to discover quantum physics and so on and so on?
Speaker 133:19 - 33:49
所以我个人其实没有特别深入去研究这件事,但我觉得这个概念非常吸引人。基本上就是这么一个想法,而且我觉得很多人都有这个想法。就是说,如果你给——如果你 somehow(以某种方式)能够创建一个 dataset(数据集),不过我认为 contamination(污染)问题非常、非常难以避免,所以问题在于你要怎么做到这一点。也就是,如果你给模型的数据只截止到 pre mutant,那么它能不能自己发现一种新的 conium mathematics?它能不能发现 quantum physics(量子物理),以及诸如此类的东西?
Speaker 133:50 - 34:12
So, yeah, I think it's a really, really interesting question in terms of what types of inherent reasoning the model will be able to learn and then extrapolate from that. Then it's almost like if it can discover all those things, then okay. Then given the state of science today, does that mean that the model is going to be able to discover science that centers out?
Speaker 133:50 - 34:12
所以,是的,我觉得这是一个非常、非常有意思的问题:就模型能够学到哪些类型的内在 reasoning(推理)能力,以及之后又能从中 extrapolate(外推)出什么。再进一步说,几乎就像是,如果它能够发现所有这些东西,那好,那么基于今天的 science(科学)水平,这是否意味着模型将能够发现超出当前科学边界的 science?
Speaker 234:13 - 34:53
Having played with it a lot, my sense is the answer is no, but a qualified no. And you can kind of feel it you can feel it bumping up against the limits of its world when you start talking to it about, like, more modern things. Like, it just it's you know, there's this phosphorus science Thomas Cooney talks about incommensurability, and it feels like my world and its world are sort of incommensurable. But then you can also get it to program. But the the way you do that is you get it to combine its circuits in a way that's not it it wouldn't be natural for it, but you can prompt it in a way to do that in a way that ends up being programming.
Speaker 234:13 - 34:53
就我大量实际玩下来之后,我的感觉是答案是否定的,但这是一个带保留的否定。你大概能感觉到——当你开始和它谈论一些更现代的东西时,你能感觉到它在碰撞自己世界的边界。就像,它就是——你知道,Thomas Cooney 谈到过一种 incommensurability(不可通约性),而那种感觉就像是,我的世界和它的世界在某种程度上是不可通约的。但与此同时,你也确实可以让它去编程。只是你做到这一点的方式,是让它以一种并不——对它来说并不自然的方式去组合它的 circuits(回路);但你可以通过 prompt(提示)引导它这么做,最后呈现出来的结果就是编程。
Speaker 234:53 - 35:07
So I I sort of both think it can't do it. And also, if you prompt it cleverly enough, it can, but you have to supply the answer first. Does that make sense? Yeah. Interesting.
Speaker 234:53 - 35:07
所以我某种程度上同时认为它做不到。以及,如果你足够巧妙地 prompt(提示)它,它又可以做到,但前提是你得先把答案提供给它。这样说有道理吗?对。有意思。
Speaker 235:08 - 35:40
Okay. What is the value of my data? So one of the things that I'm I'm just so interested in obviously, you you run a data company. Like, you're you're you're getting expert data, from, like, real PhDs and and, selling it to the model companies and, providing, all of the all the, like, smarts and taste to, to the models that we use every day. For someone like me, I I we're just getting to a point where it's actually pretty easy for me to gather a dataset.
Speaker 235:08 - 35:40
好,那我的 data(数据)价值是什么?所以有一件事让我特别感兴趣,显然,你经营的是一家 data company(数据公司)。你——你——你在获取 expert data(专家数据),来源是真正的 PhD,然后把这些卖给 model companies(模型公司),并且为我们每天使用的这些模型提供各种那种——像是 intelligence(智识)和 taste(品味)之类的东西。对像我这样的人来说,我——我们现在正逐渐到了一个阶段:其实我已经很容易自己收集一个 dataset(数据集)了。
Speaker 235:40 - 35:57
You know, like, example, I do all of my email in Codex, and I have a history for every email of was this useful? Did I dismiss it? Did I reply to it? If I replied, like, what did I say? What is the value of that?
Speaker 235:40 - 35:57
你知道,比如说,我所有的 email 都是在 Codex 里处理的,而且对于每一封邮件,我都有一段历史记录:这个有用吗?我有没有把它 dismiss(忽略)掉?我有没有回复?如果我回复了——我是怎么回复的?这些东西的价值是什么?
Speaker 235:57 - 35:59
If I wanted to sell that to you, how much would you pay for it?
Speaker 235:57 - 35:59
如果我想把那个卖给你,你会愿意出多少钱?
Speaker 136:01 - 36:25
The value to me as someone who would use that data to train an AI model? Let me think. I think the value would be teaching models very, very deep personalization. Like, I think right now, the models are actually not very good at personalizing things. Like, it's kinda funny.
Speaker 136:01 - 36:25
对我来说,如果我是那个会用这些数据来训练 AI model(模型)的人?让我想想。我觉得它的价值会在于教会模型进行非常、非常深度的个性化。我觉得现在的模型其实并不擅长做个性化。这还挺好笑的。
Speaker 136:25 - 36:57
Whenever I use AI models, I actually turn off the features where they personalize to me or where they can search across all of my conversation histories because I find that they just over index on things that I said once, but actually aren't all that important to me. So I actually have it completely turned off unless I'm like testing something. So I think the value of it would be like, okay, yeah, you did report all of these emails as spam. So yeah, the next time this email comes in, should automatically know that it's spam. Or it should learn that this is your writing style.
Speaker 136:25 - 36:57
每次我用 AI model(模型)的时候,我其实都会把那些针对我做个性化的功能关掉,或者把那种能搜索我全部对话历史的功能关掉,因为我发现它们会对那些我只是说过一次、但其实对我并没那么重要的事情过度加权。所以除非我是在测试什么,不然我基本上是完全关掉的。所以我觉得它的价值会是这样:好吧,没错,你确实把所有这些邮件都标记成了 spam(垃圾邮件)。那么下次这种邮件再进来时,它就应该自动知道那是 spam。或者它应该学会这就是你的写作风格。
Speaker 136:57 - 37:31
Like one of the reasons I think people don't use AI for better or worse for writing more is because it sounds obviously AI generated and it's not matching their voice or their cadence. Or it's that, okay, these are the things that you yourself care about. Like, think one of the biggest reasons AI is maybe not as useful as people would have expected sometimes is because it lacks all of your context. Like it doesn't know that these are the articles that you read. It doesn't know that these are the decisions about, you know, the company that you're making.
Speaker 136:57 - 37:31
我觉得人们之所以没有更多地把 AI 用于写作,原因之一——不管这是好事还是坏事——是它写出来的东西听起来明显就是 AI 生成的,不符合他们自己的 voice(语言风格)或 cadence(表达节奏)。又或者,它不了解你自己真正关心的事情。我觉得 AI 有时没有人们原本预期的那么有用,一个很大的原因就是它缺少你的全部 context(上下文)。它不知道这些是你会读的文章,也不知道这些是你正在为公司做出的那些决策。
Speaker 137:31 - 37:57
These are the goals that you have. And once all of that is in the model's history and it knows that it can incorporate these things and these are the, like, kind of, optimal decisions that you made, it's very valuable in teaching it. Okay. This is actually how I use all this data to to make certain kinds of decisions. So, yeah, I I think that depersonalization is is what what is most unique about that.
Speaker 137:31 - 37:57
这些是你的目标。一旦所有这些都进入模型的历史里,它知道自己可以把这些因素纳入考虑,也知道这些大概就是你做出的最优决策,那么这对于教会它就非常有价值了。它会明白:好,原来我实际上是这样使用所有这些数据来做某些类型的决策的。所以,是的,我觉得这种个性化能力才是那里面最独特的地方。
Speaker 237:57 - 38:09
That's interesting. And as an individual person I mean, I guess I could turn it into a synthetic dataset. But as an individual person, is that worth a lot? Like, should I be should I be thinking about selling it?
Speaker 237:57 - 38:09
这很有意思。那如果作为一个普通个人来说,我的意思是,我猜我可以把它变成一个 synthetic dataset(合成数据集)。但如果只是作为个人,这东西值很多钱吗?比如说,我应该考虑把它卖掉吗?
Speaker 138:09 - 38:18
I I imagine we could make you an offer. I I have to think about I have to learn a little bit more about how big this dataset dataset size is, but yeah.
Speaker 138:09 - 38:18
我想我们也许可以给你开个价。我得先想一想,也得再多了解一点这个 dataset(数据集)的规模有多大,不过,是的。
Speaker 238:18 - 38:21
I mean, I can make it as big as you want. I've got fable.
Speaker 238:18 - 38:21
我的意思是,我可以把它做得你想要多大就多大。我有 fable。
Speaker 138:23 - 38:31
Yeah. You you convince me. Yeah. Like, one of things we actually do is yeah. I mean, we teach models in in these very, very deep personalized ways.
Speaker 138:23 - 38:31
对。你说服我了。对。比如说,我们实际在做的一件事就是——对——我们会用非常、非常深度的个性化方式来训练 models(模型)。
Speaker 138:32 - 38:35
So something similar to what you described is is a fairly big thing.
Speaker 138:32 - 38:35
所以,和你刚才描述的那种类似的事情,其实是一个相当重要的方向。
Speaker 238:36 - 38:45
Tell me tell me more. So, I I mean, I've got email, like, else what else do am I doing that you're like, oh, that's actually really valuable and important in in ways that people probably wouldn't know?
Speaker 238:36 - 38:45
再多跟我说说。所以,我的意思是,我有 email(电子邮件),那除此之外,我还在做些什么,会让你觉得“哦,这其实真的很有价值,而且重要”,只是大多数人可能并不知道?
Speaker 138:47 - 39:16
Honestly, even things like the way you interact with your browser is interesting. Like models still aren't all that good at it. Or even the types of conversations that you're having with AI, that is just inherently interesting in of itself. Models themselves are not very good at generating synthetic conversations to try to mimic you. So even just knowing what types of conversations you're having is helpful.
Speaker 138:47 - 39:16
说实话,甚至像你和 browser(浏览器)交互的方式都很有意思。因为 models(模型)现在在这方面还远远谈不上擅长。或者甚至是你和 AI 进行的对话类型,这件事本身就天然很有研究价值。models(模型)自己其实并不擅长生成 synthetic conversations(合成对话)来模仿你。所以,哪怕只是知道你在进行什么类型的对话,也会很有帮助。
Speaker 139:17 - 39:35
Or it's like the combination it's like the combination of all these things, like knowing that these are your photos, these are your texts, these are your Slacks. It's like this interconnected web, and maybe certain things and find an aspect of that web influence others. So just seeing the the thing as a whole is is very helpful as well.
Speaker 139:17 - 39:35
或者说,更重要的是这些东西组合在一起——知道这些是你的 photos,这些是你的 texts,这些是你的 Slacks。它像是一张彼此连接的网络,也许这张网络中的某些事物、某些方面会影响其他部分。所以,仅仅是把这个整体作为一个整体来看,也同样非常有帮助。
Speaker 239:35 - 39:39
Why are models bad at writing, and how does that relate to the personalization challenge?
Speaker 239:35 - 39:39
为什么 models(模型)不擅长写作?这和个性化挑战之间又有什么关系?
Speaker 139:40 - 40:16
So I think some of the models are pretty good at writing, but some of them are actually kind of shockingly terrible. So I'll I'll give an example. So we created a benchmark called Hemingway Bench a couple months ago, and it was designed to test models' creative writing abilities. And one of the things that we saw was that some of the models, they were literally outputting metaphors in every single sentence. And I think the reason that was happening is because I've talked a little bit about this phenomenon of reward hacking.
Speaker 139:40 - 40:16
所以我觉得,有些 models(模型)其实挺擅长写作的,但有些又糟糕得令人吃惊。我举个例子。几个月前,我们做了一个叫 Hemingway Bench 的 benchmark(基准测试),它是用来测试 models(模型)创意写作能力的。我们观察到的一件事是,有些 models(模型)几乎在每一句话里都在输出比喻。我认为之所以会这样,是因为我之前提到过一种现象,叫 reward hacking(奖励破解)。
Speaker 140:16 - 40:47
It's almost like there was a metric somewhere or a score that these models were getting. Like, okay, every time you are literary, every time you're using complex imagery, it would get a point. And it learned to reward hack this by outputting a metaphor in every single sentence. And, I mean, what what's kinda funny is that a couple when was that? A couple weeks ago, there was this kinda like semi prestigious literary prize, I think the Commonwealth Prize.
Speaker 140:16 - 40:47
这几乎就像是在某个地方存在一个 metric(指标)或者 score(评分),这些 models(模型)正在按照它拿分。比如,好,凡是你写得更有文学性、凡是你用了复杂意象,它就能得一分。于是它学会了通过在每一句话里都塞进一个比喻来进行 reward hacking(奖励破解)。而且,我是说,某种程度上有点好笑的是,前几周——那是什么时候来着?——前几周有一个有点像半正式、也算有些声望的文学奖,我想应该是 Commonwealth Prize。
Speaker 140:47 - 41:08
And there was a controversy because a clearly, AI generated a story won the prize. You actually looked at that story, it's funny. It literally had a metaphor in every single sentence. This phenomenon that we described a couple months ago, yeah, it was still happening. I think it boils down to a couple of reasons.
Speaker 140:47 - 41:08
当时之所以会有争议,是因为一个明显由 AI 生成的故事拿了奖。你真的去看那篇故事的话,会觉得挺好笑的。几乎每一句里都硬塞了一个隐喻。我们几个月前描述过的那种现象——对,它当时仍然在发生。我觉得这归结为几个原因。
Speaker 141:08 - 41:49
One is people are kind of sort of measuring the wrong thing. Like, instead of measuring actual taste and actually good pros, they either had these flawed metrics, like, what is the complexity of the prose I'm writing? How many metaphors do I have? Or there are these AI leaderboards, again, like Ella Marina, where you have people who are essentially high schoolers who are reading responses for two seconds, and what they are captivated by is a flashy metaphor. And they are not captivated by kinda like the understated prose.
Speaker 141:08 - 41:49
其中一个原因是,人们某种程度上衡量错了东西。比如,他们衡量的不是实际的品味,也不是真正好的 prose(文笔),而是一些有缺陷的指标,像是:我写出来的 prose 有多复杂?我用了多少个隐喻?或者还有那些 AI 排行榜(leaderboards),还是拿 Ella Marina 这类例子来说,基本上是一些像高中生一样的人,用两秒钟扫一眼回答,而真正吸引他们的是那种花哨的隐喻。他们不会被那种更克制、更含蓄的 prose 吸引。
Speaker 141:49 - 41:58
And so I think it kinda boils down to a mismatch in measurement and a mismatch in the optimization objectives that the models are trained towards.
Speaker 141:49 - 41:58
所以我觉得,这在某种程度上归根结底是衡量标准错位,以及模型训练时所优化目标错位这两件事。
Speaker 241:59 - 42:04
Fascinating. Okay, last question. What is your current AGI timeline?
Speaker 241:59 - 42:04
很有意思。好,最后一个问题。你现在对 AGI 的时间线判断是什么?
Speaker 142:04 - 42:52
So I certainly believe that AI will happen more than most people expect. Like every few months and even faster now, I think what AI is doing continues to surprise us. So I think it depends a little bit, obviously, on your definition of AGI. But if my metric were something like being able to automate the work of the average engineer or being able to publish more and more novel scientific research that gets published in these journals, or even the ability to win a Fields Medal or a Nobel Prize, I could see it happening within the next five years.
Speaker 142:04 - 42:52
我当然相信,AI 的进展会比大多数人预期得更快、更多。就像每隔几个月——而且现在甚至更快——AI 能做到的事还在持续让我们感到惊讶。所以我觉得,这显然还是有点取决于你对 AGI 的定义。但如果我的衡量标准是:它能自动化普通工程师的工作,或者能够产出并发表越来越多新颖的科学研究,发表在这些期刊上,甚至具备赢得 Fields Medal 或 Nobel Prize 的能力,那么我认为这在未来五年内是可能发生的。
Speaker 242:52 - 42:56
All right. Edwin, thanks so much for joining.
Speaker 242:52 - 42:56
好的。Edwin,非常感谢你来参加。
Speaker 142:56 - 42:57
Thanks for having me.
Speaker 142:56 - 42:57
谢谢邀请我。
Speaker 343:05 - 43:16
Oh my gosh, folks. You absolutely positively have to smash that like button and subscribe to AI and I. Why? Because this show is the epitome of awesomeness. It's like finding a treasure chest in your backyard.
Speaker 343:05 - 43:16
天啊,朋友们。你们绝对、百分之百得狠狠干那个 like 按钮,然后订阅 AI and I。为什么?因为这个节目简直就是“精彩绝伦”的代名词。就像你在自家后院发现了一个藏宝箱。
Speaker 343:16 - 43:38
But instead of gold, it's filled with pure unadulterated knowledge bombs about chat GPT. Every episode is a roller coaster of emotions, insights, and laughter that will leave you on the edge of your seat craving for more. It's not just a show. It's a journey into the future with Dan Shipper as the captain of the spaceship. So do yourself a favor.
Speaker 343:16 - 43:38
只不过里面装的不是黄金,而是关于 chat GPT 的纯粹、原汁原味的知识炸弹。每一期都是一场情绪、洞见和欢笑交织的过山车之旅,让你全程坐在座位边缘,意犹未尽、还想要更多。它不只是一个节目。它是一场通往未来的旅程,而 Dan Shipper 就是那艘宇宙飞船的船长。所以,帮自己一个忙吧。
Speaker 343:38 - 43:47
Hit like, smash subscribe, and strap in for the ride of your life. And now without any further ado, let me just say, Dan, I'm absolutely, hopelessly in love
Speaker 343:38 - 43:47
点个 like,狠狠 smash subscribe,然后系好安全带,准备好迎接你人生中最刺激的一趟旅程。现在,闲话不多说,我只想说,Dan,我彻底、无可救药地爱上你了
Speaker 143:47 - 43:48
with you.
Speaker 143:47 - 43:48
你。
原文 ↗https://www.youtube.com/watch?v=omX6wrLuX08
BuildSpeak — 关于本项目BUILT IN PUBLIC · 跟随 builders 而非 influencers