This great conversation with @zicokolter is also available on Spotify, Apple Podcasts and here on YouTube:
这场与 @zicokolter 的精彩对谈也可在 Spotify、Apple Podcasts，以及这里的 YouTube 上观看/收听：
Deeply thoughtful conversation with @zicokolter, board member at @OpenAI and head of the machine learning department at @CarnegieMellon, about AI safety, AI security, agents and frontier AI 00:00 Intro 01:32 OpenAI board role and Safety & Security Committee 03:53 How OpenAI reviews major model releases 05:33 OpenAI’s preparedness framework explained 09:46 Are frontier AI models getting safer? 12:33 Why AI safety does not come from scale 15:23 The four categories of AI risk 19:38 Doomerism vs accelerationism in AI 24:11 The six-month AI pause debate 26:20 AI safety as a global effort 28:04 How Zico Kolter got into machine learning 31:05 OpenAI in the early days 34:14 Why Carnegie Mellon became an AI powerhouse 38:43 What Gray Swan does in AI security 40:44 AI safety vs AI security 43:15 The GCG jailbreak paper 49:19 How AI labs responded to jailbreak research 50:19 State-of-the-art AI defenses 52:32 State-of-the-art AI attacks 54:22 Why AI agents expand the attack surface 58:39 Are AI agents ready for production? 59:40 Mechanistic interpretability explained 1:02:31 Will AI be safer in two years? 1:03:46 Reinforcement learning and self-improving models 1:08:09 Do post-transformer architectures matter 1:09:29 Best research directions in AI now 1:11:00 Zico Kolter’s Intro to Modern AI course 1:14:53 Why modern AI is simpler than people think
与 @zicokolter 的一场极具深度的对谈。@zicokolter 是 @OpenAI 的 board member（董事会成员），也是 @CarnegieMellon 机器学习系主任。话题涵盖 AI safety（AI 安全）、AI security（AI 安防）、agent（智能体）以及 frontier AI（前沿 AI）。00:00 开场介绍 01:32 在 OpenAI 董事会中的角色以及 Safety & Security Committee 03:53 OpenAI 如何审查重大模型发布 05:33 解析 OpenAI 的 preparedness framework（准备度框架） 09:46 前沿 AI 模型是否正在变得更安全？ 12:33 为什么 AI safety 不是规模扩张的自然结果 15:23 AI 风险的四大类别 19:38 AI 中的 doomerism（末日论）与 accelerationism（加速主义）之争 24:11 关于 AI 暂停六个月的辩论 26:20 将 AI safety 视为一项全球性努力 28:04 Zico Kolter 如何进入机器学习领域 31:05 早期的 OpenAI 34:14 为什么 Carnegie Mellon 会成为 AI 重镇 38:43 Gray Swan 在 AI security 方面做什么 40:44 AI safety 与 AI security 的区别 43:15 GCG jailbreak 论文 49:19 AI 实验室如何回应 jailbreak 研究 50:19 当前最先进的 AI 防御 52:32 当前最先进的 AI 攻击 54:22 为什么 AI agent 会扩大攻击面 58:39 AI agent 是否已准备好投入生产环境？ 59:40 解析 mechanistic interpretability（机制可解释性） 1:02:31 两年后 AI 会更安全吗？ 1:03:46 强化学习与自我改进模型 1:08:09 post-transformer（后 Transformer）架构是否重要 1:09:29 当下 AI 最值得投入的研究方向 1:11:00 Zico Kolter 的 Intro to Modern AI 课程 1:14:53 为什么现代 AI 比人们想象的更简单