AI Safety Newsletter

• AISN

  Plus, Measuring AI Honesty. Welcome to the AI Safety Newsletter by the Center for AI Safety. We discuss developments in AI and AI safety. No technical background required. In this newsletter, we discuss two recent papers: a policy paper on national security strategy, and a technical paper on measuring honesty in AI systems. Listen to the AI Safety Newsletter for free on Spotify or Apple Podcasts. Superintelligence Strategy CAIS director Dan Hendrycks, former Google CEO Eric Schmidt, and Scale AI CEO Alexandr Wang have authored a new paper, Superintelligence Strategy. The paper (and an in-depth expert version) argues that the development of superintelligence—AI systems that surpass humans in nearly every domain—is inescapably a matter of national security. In this story, we introduce the paper's three-pronged strategy for national security in the age of advanced AI: deterrence, nonproliferation, and competitiveness. Deterrence The simultaneous power and danger of superintelligence presents [...] ---Outline:(00:20) Superintelligence Strategy(01:09) Deterrence(02:41) Nonproliferation(04:04) Competitiveness(05:33) Measuring AI Honesty(09:24) LinksThe original text contained 6 images which were described by AI. --- First published: March 6th, 2025 Source: https://newsletter.safe.ai/p/ai-safety-newsletter-49-superintelligence --- Want more? Check out our ML Safety Newsletter for technical safety research. Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

  --------  

  11:31
• Superintelligence Strategy: Expert Version

  Superintelligence is destabilizing since it threatens other states’ survival—it could be weaponized, or states may lose control of it. Attempts to build superintelligence may face threats by rival states—creating a deterrence regime called Mutual Assured AI Malfunction (MAIM). In this paper, Dan Hendrycks, Eric Schmidt, and Alexandr Wang detail a strategy—focused on deterrence, nonproliferation, and competitiveness—for nations to navigate the risks of superintelligence. ---

  --------  
• Superintelligence Strategy: Standard Version

  Superintelligence is destabilizing since it threatens other states’ survival—it could be weaponized, or states may lose control of it. Attempts to build superintelligence may face threats by rival states—creating a deterrence regime called Mutual Assured AI Malfunction (MAIM). In this paper, Dan Hendrycks, Eric Schmidt, and Alexandr Wang detail a strategy—focused on deterrence, nonproliferation, and competitiveness—for nations to navigate the risks of superintelligence. ---

  --------  
• AISN #48: Utility Engineering and EnigmaEval

  Welcome to the AI Safety Newsletter by the Center for AI Safety. We discuss developments in AI and AI safety. No technical background required. Listen to the AI Safety Newsletter for free on Spotify or Apple Podcasts. In this newsletter, we explore two recent papers from CAIS. We’d also like to highlight that CAIS is hiring for editorial and writing roles, including for a new online platform for journalism and analysis regarding AI's impacts on national security, politics, and economics. Utility Engineering A common view is that large language models (LLMs) are highly capable but fundamentally passive tools, shaping their responses based on training data without intrinsic goals or values. However, a new paper from the Center for AI Safety challenges this assumption, showing that LLMs exhibit coherent and structured value systems. Structured preferences emerge with scale. The paper introduces Utility Engineering, a framework for analyzing and controlling AI [...] ---Outline:(00:26) Utility Engineering(04:48) EnigmaEvalThe original text contained 4 images which were described by AI. --- First published: February 18th, 2025 Source: https://newsletter.safe.ai/p/ai-safety-newsletter-48-utility-engineering --- Want more? Check out our ML Safety Newsletter for technical safety research. Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

  --------  

  8:56
• AISN #47: Reasoning Models

  Plus, State-Sponsored AI Cyberattacks. Listen to the AI Safety Newsletter for free on Spotify or Apple Podcasts. Reasoning Models DeepSeek-R1 has been one of the most significant model releases since ChatGPT. After its release, the DeepSeek's app quickly rose to the top of Apple's most downloaded chart and NVIDIA saw a 17% stock decline. In this story, we cover DeepSeek-R1, OpenAI's o3-mini and Deep Research, and the policy implications of reasoning models. DeepSeek-R1 is a frontier reasoning model. DeepSeek-R1 builds on the company's previous model, DeepSeek-V3, by adding reasoning capabilities through reinforcement learning training. R1 exhibits frontier-level capabilities in mathematics, coding, and scientific reasoning—comparable to OpenAI's o1. DeepSeek-R1 also scored 9.4% on Humanity's Last Exam—at the time of its release, the highest of any publicly available system. DeepSeek reports spending only about $6 million on the computing power needed to train V3—however, that number doesn’t include the full [...] ---Outline:(00:13) Reasoning Models(04:58) State-Sponsored AI Cyberattacks(06:51) LinksThe original text contained 1 image which was described by AI. --- First published: February 6th, 2025 Source: https://newsletter.safe.ai/p/ai-safety-newsletter-47-reasoning --- Want more? Check out our ML Safety Newsletter for technical safety research. Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

  --------  

  9:00

Más podcasts de Tecnología

Podcasts a la moda de Tecnología

Acerca de AI Safety Newsletter