Lenny's Reads

Ever run an AI analysis on customer data, only to discover the numbers were fabricated and the insights completely generic? In this episode, Caitlin Sullivan, a user-research veteran who's trained hundreds of product and research professionals, shares her four prompting techniques for getting trustworthy, actionable insights out of any LLM. After 2,000+ hours of testing customer discovery workflows with AI, she's identified the failure modes that break AI analysis and the reliable fixes for each one.In this episode, you'll learn:
• How to catch the two types of AI quote hallucinations
• Why AI defaults to useless generic themes and insights
• Which LLM is best for analysis work (and which one fabricates the most)
• How to turn vague signal into actual decision clarity
• The final verification pass that stress-tests everything before it hits a deck
Referenced:
• Caitlin Sullivan: https://www.linkedin.com/in/caitlindsullivan/
• Claude Code for Customer Insights (Maven course): https://maven.com/caitlin/claude-code-insights
• Claude: https://www.anthropic.com/claude
• ChatGPT: https://chatgpt.com/
• Gemini: https://gemini.google.com/
• NotebookLM: https://notebooklm.google.com/
• Maze: https://maze.co/
• Whoop: https://www.whoop.com/
Read the newsletter: https://www.lennysnewsletter.com/p/how-to-do-ai-analysis-you-can-actually
Subscribe to Lenny’s Newsletter: https://www.lennysnewsletter.com/subscribe
Subscribe:
• YouTube: https://www.youtube.com/@lennysreads
• Apple: https://podcasts.apple.com/us/podcast/lennys-reads/id1810314693
• Spotify: https://open.spotify.com/show/0IIunA06qMtrcQLfypTooj
• Substack: https://lennysreads.com/
Follow Lenny
• Twitter: https://twitter.com/lennysan
• LinkedIn: https://www.linkedin.com/in/lennyrachitsky/
• Podcast: https://www.youtube.com/@lennyspodcast
About
Welcome to Lenny's Reads, where every week you’ll find a fresh audio version of my newsletter about building product, driving growth, and accelerating your career, read to you by the soothing voice of Lennybot.


To hear more, visit www.lennysnewsletter.com

If you’re a premium subscriber, add the private feed to your podcast app at add.lennysreads.com

Dr. Marily Nika, longtime AI PM at Google and Meta, shares a simple weekly ritual that rapidly builds AI product sense – the ability to translate probabilistic model behavior into products people can trust. In this episode, Marily walks through the framework for uncovering failure modes before users do.

Listen now: YouTube | Apple | Spotify

In this episode, you’ll learn
• Why Meta added “Product Sense with AI” to its PM interview loop
• The rituals that surface hidden failure modes
• Why generative models confidently invent structure when confronted with mess
• What minimum viable quality (MVQ) means and how to define three critical thresholds
• Five strategic context factors that raise or lower your quality bar
• Why you need to estimate your AI feature’s cost envelope early
• How to design guardrails that protect users from model shortcomings
• Four patterns that cover most real-world failure cases

Referenced
• AI PM Bootcamp & Certification
• AI Product Sense & AI PM Interview prep
• Dr. Marily Nika: https://www.linkedin.com/in/marilynika/
• How to build AI Product sense (Part 1)
• Marily’s AI Product Academy Newsletter
• Thriving as a senior IC PM in the AI era

About
Welcome to Lenny's Reads, where every week you’ll find a fresh audio version of my newsletter about building product, driving growth, and accelerating your career, read to you by the soothing voice of Lennybot.


To hear more, visit www.lennysnewsletter.com

If you’re a premium subscriber, add the private feed to your podcast app at add.lennysreads.com
Your product builder's Product Pass just got even better. In this episode, we introduce the seven new premium partners joining the Product Pass, bringing the total value of a paid newsletter subscription to over $25,000. We cover all the new products, what they do, how people are using them, and the key details you need to know before subscribing.

Listen now: YouTube | Apple | Spotify

In this episode, you’ll learn:
• Key features of each of the seven new products
• Applicable, real-world examples of how people are using them
• Which products are exclusive to Insider subscribers

References:
• Affinity
• Amp
• Canva Business
• Duolingo
• ElevenLabs
• Every
• Factory
• Flourish
• Framer
• Harvey
• Kernel
• Leonardo.Ai
• Lovable
• Manus
• Meta
• Metabase
• Mozart AI
• n8n
• Nvidia
• Perplexity
• Product Pass codes
• Railway
• Sentry


This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.lennysnewsletter.com/subscribe

Colin Matthews, shares how ChatGPT apps are emerging as a powerful new distribution channel. In this episode, Colin shares how discovery will become contextual inside ChatGPT, what the architecture looks like, and why ChatGPT apps represent a rare distribution opportunity. He also covers two fast paths you can take to build and deploy your first ChatGPT app.

In this episode, you'll learn:
• Why chat-native apps could change the tech ecosystem
• How users will discover apps contextually inside ChatGPT
• The three UI modes for ChatGPT apps
• The “one widget per message” constraint
• How tools, widgets, and MCP fit together
• Two build paths: Replit and Chippy

References:
• Adobe
• AllTrails
• Anthropic
• Apple
• Booking.com
• Bolt
• Canva
• ChatPRD
• Chippy
• Colin Matthews
• Coursera
• Datavant
• Descript
• Devin
• DoorDash
• Expedia
• Figma
• Gamma
• GitHub
• Granola
• Instacart
• Linear
• LinkedIn
• Lovable
• Magic Patterns
• Maven
• Mobbin
• n8n
• OpenAI
• Perplexity
• PostHog
• Railway
• Raycast
• Replit
• Shopify
• Spotify
• Stripe Atlas
• Substack
• Superhuman
• Vercel
• Warp
• Wispr Flow
• Zillow

Read the newsletter
https://www.lennysnewsletter.com/p/chatgpt-apps-are-about-to-be-the

Subscribe to Lenny’s Newsletter
https://www.lennysnewsletter.com/subscribe

Subscribe
• YouTube: https://www.youtube.com/@lennysreads
• Apple: https://podcasts.apple.com/us/podcast/lennys-reads/id1810314693
• Spotify: https://open.spotify.com/show/0IIunA06qMtrcQLfypTooj
• Substack: https://lennysreads.com/

Follow Lenny
• Twitter: https://twitter.com/lennysan
• LinkedIn: https://www.linkedin.com/in/lennyrachitsky/
• Podcast: https://www.youtube.com/@lennyspodcast

About
Welcome to Lenny's Reads, where every week you’ll find a fresh audio version of my newsletter about building product, driving growth, and accelerating your career, read to you by the soothing voice of Lennybot.


To hear more, visit www.lennysnewsletter.com

Noam Segal, AI Insights Manager at Figma, shares the results from our comprehensive survey of 1,750 product managers, engineers, designers, and founders on how AI is reshaping their work. The findings are striking: more than half of respondents are saving at least half a day per week on their most important tasks. Since the quality of AI tools are improving at breakneck speed, Noam reveals we're watching the early innings of a compounding productivity revolution.
In this episode, you'll learn:
• Why 55% say AI has exceeded expectations and how it's improving work quality
• Which roles are winning with AI (and which are struggling)
• Where the biggest opportunities are
• Which AI tools have product-market fit
• Which AI tool dominates for most roles
• Why engineers prefer specialized tools
• Why 92% report significant downsides to AI tools
• What AI agents need to take off
References:
• Airbnb: https://www.airbnb.com/
• Anthropic: https://www.anthropic.com/
• Build your personal AI copilot: https://www.lennysnewsletter.com/p/build-your-personal-ai-copilot
• ChatGPT: https://chatgpt.com/
• Claude: https://claude.ai/
• Cursor: https://cursor.com/
• Figma: https://www.figma.com/
• Gemini: https://gemini.google.com/
• GitHub: https://github.com/
• Granola: https://www.granola.ai/
• How to build your second brain with ChatGPT: https://www.lennysnewsletter.com/p/how-to-build-your-pm-second-brain
• Intercom: https://www.intercom.com/
• Lindy: https://www.lindy.ai/
• Lovable: https://lovable.dev/
• Manus: https://manus.im/
• Microsoft Copilot: https://copilot.microsoft.com/
• Midjourney: https://www.midjourney.com/home
• Monday.com: https://www.monday.com
• n8n: https://n8n.io/
• Noam Segal: https://www.linkedin.com/in/noamsegal/
• Notion: https://www.notion.com/
• Perplexity: https://www.perplexity.ai/
• Replit: https://replit.com/
• v0: https://v0.app/
• Zapier: https://zapier.com/
Read the newsletter
https://www.lennysnewsletter.com/p/ai-tools-are-overdelivering-results

Subscribe to Lenny’s Newsletter
https://www.lennysnewsletter.com/subscribe

Subscribe
• YouTube: https://www.youtube.com/@lennysreads
• Apple: https://podcasts.apple.com/us/podcast/lennys-reads/id1810314693
• Spotify: https://open.spotify.com/show/0IIunA06qMtrcQLfypTooj
• Substack: https://lennysreads.com/

Follow Lenny
• Twitter: https://twitter.com/lennysan
• LinkedIn: https://www.linkedin.com/in/lennyrachitsky/
• Podcast: https://www.youtube.com/@lennyspodcast

About
Welcome to Lenny's Reads, where every week you’ll find a fresh audio version of my newsletter about building product, driving growth, and accelerating your career, read to you by the soothing voice of Lennybot.


To hear more, visit www.lennysnewsletter.com