Product Impact Podcast | Formerly Design of AI

How is it that Microsoft and OpenAI’s CEOs are telling us to panic because white collar jobs are going to be replaced by AI,

Then there’s endless evidence of the opposite: Most companies that implement AI see little gains, with execs from over 80% of companies reporting no productivity gains at all.

In this episode of the Product Impact Podcast we tackle Why Your AI Metrics Are Lying to You. We’ll provide you with a framework for improving AI product performance. We discuss how Evals can't answer the most important questions you have about your product's impact and the importance of calibrating your products for success by balancing key pillars.

In this episode you’ll learn:
- Agents hide friction from view, creating dangerous impact blindness
- Balance power, speed, impact & joy to win in the AI era, like F1 cars
- Success doesn’t equal satisfaction—you must measure both outcomes
- Measure outcomes and feelings, not just activity logs and checkmarks

Thank you for listening to the Product Impact Podcast (Formerly Design of AI)
Prove impact. Improve impact. Scale impact.
Learn frameworks and strategies to ensure your product is delivering impact to users, teams, businesses, and communities. We investigate enterprise adoption and highlight builders/startups disrupting value creation.

Hosted by:
Arpy Dragffy Guerrero https://www.linkedin.com/in/adragffy/
Brittany Hobbs https://www.linkedin.com/in/brittanyhobbs/

Subscribe to https://productimpactpod.substack.com for AI Strategy resources

Brought to you by PH1 https://ph1.ca an AI strategy consultancy specialized in improving the success of your AI product.

We started the Design of AI podcast at the end of 2023 at a time when GenAI was a black box of possibilities. Our focus was to help people working in tech navigate a great time of change and unpack how to experiment this new technology. We've now moved into the next era of AI —scaling value and our podcast must adapt too. Where season one was focused on explaining AI and how roles will be forced to change, season 2 will focus on how to measure impact and how to scale impact.

Keep up to date with the new podcast: https://productimpactpod.com

Our focus is highly strategic and pragmatic. We want business, product, design, and research leaders who can unpack the uncomfortable truths about delivering impact at scale. We are seeking dedicated academics and thought leaders who challenge the status quo on how to measure and improve impact delivery. We want voices that challenge the belief that all impact is positive and who can provide a compass to guide the evolution of tech and industries.

Viral agent demos are training product teams to trust spectacle instead of outcomes—and that’s how unsafe automation slips into real workflows. In this episode we welcome Jim Love, one to the most respected voices in technology news to unpack what “Claude Bot / open claw” and Moltbook-style experiments actually prove, what they exaggerate, and why the hardest problems aren’t capability—they’re control, security, and measurement.

In this episode we cover:
Why viral demos distort reality: Hype spotlights novelty, not reliability—so teams miss what breaks when the demo meets real users.

Local agents raise risk fast: Local access turns assistants into operators—writing, deleting, impersonating, and expanding blast radius.

“It learns” is overstated: Many stacks “learn” by saving state—easy to inspect, steal, poison, and manipulate.

Emergence isn’t intelligence: Weird behaviors can emerge at scale without intent—don’t mistake patterns for agency or judgment.

Outcomes > inputs, always: Great teams define success, measure impact, and kill distractions—even when the tech looks magical.

You’ll leave with a sharper lens for evaluating agent stacks before they create collateral damage you can’t see or stop.
Jim Love has spent more than 40 years in technology, working globally as a consultant, leading an international consulting practice, serving as a CIO, and building his own consulting company. He was also CIO and head of content at the iconic publication IT World Canada.
Today he runs a new publication Tech Newsday and hosts two widely followed technology podcasts, Cybersecurity Today and Hashtag Trending. He continues to advise a select group of companies, mostly startups looking to deal with AI.
Jim is the author of both fiction and non-fiction, including Digital Transformation in the First Person.
His latest novel, Elisa: A Tale of Quantum Kisses, explores a near-future shaped by artificial intelligence and became an Audible bestseller shortly after release.
Tech Newsday — Jim Love’s publication covering tech, AI, and security.
https://technewsday.com/
Hashtag Trending — the podcast feed for fast tech headlines + commentary.
https://technewsday.com/podcasts_categories/hashtag-trending/
Elisa: A Tale of Quantum Kisses — Jim’s near-future AI novel (Amazon listing).
https://www.amazon.com/Elisa-Quantum-Kisses-Jim-Love/dp/B0DPFZMDGZ

If this episode helped, follow/subscribe so you don’t miss what’s next. And if you’re listening on Apple Podcasts or Spotify, leave a rating and a review—it’s the simplest way to help more product teams find the show.

Get the ideas, frameworks, and episode takeaways as a written brief—subscribe to the Design of AI Substack.

PH1 Research helps product teams improve digital experiences in the AI era—across strategy, benchmarking, and UX evaluations—so you can measure what matters, reduce impact blindness, and ship systems customers actually trust and adopt. Learn more at https://www.ph1.ca/.

While the world is obsessed with the Moltbot/Clawdbot AI agent, founders like Devi Parikh are laying the foundation for how agents will transform search and shopping—agents that monitor, negotiate, and navigate on behalf of users, securely.
Search is becoming proactive. Shopping is becoming delegated. And the next interface won’t be a results page—it’ll be agents running quietly in the background, surfacing what matters when it matters.
How agents turn search into continuous monitoring

Why shopping shifts from browsing to delegation

Where value shows up first in real workflows

What trust requires before agents can transact

The path from alerts → actions → autonomy

In this episode, Devi breaks down how Scouts reframes search as “future-facing discovery”: track price drops, in-stock alerts, sales leads, funding news, flights, and local events—then get notified the moment conditions change.
We also explore what comes next: moving from monitoring to task completion—where agents can execute purchases and bookings with explicit confirmations, hard guardrails, and a deliberate “trust staircase” designed to prevent surprises.

If you enjoyed this episode, follow the podcast and leave a rating + review—it helps more builders find the show.

Subscribe to the Design of AI Substack for in-depth AI product strategy resources, operator-grade analysis, and frameworks on what makes AI products succeed (and why they fail).

This episode is brought to you by PH1 Research—a strategy + research partner for product leaders shipping AI-enabled experiences. We help teams define success metrics that actually matter, validate value before scaling, and reduce trust and adoption risk through AI strategy, UX evaluation, and evidence-driven product decisions.

Devi Parikh is the co-founder and co-CEO of Yutori, and was previously a Senior Director in Generative AI at Meta and an Associate Professor at Georgia Tech. Her research focuses on human–AI collaboration, generative AI, multimodal AI, and AI for creativity. She holds a Ph.D. from Carnegie Mellon University and has received recognitions including the PAMI Mark Everingham Prize.
Try Scouts: https://scouts.yutori.com/

Blog: The Bitter Lesson for Web Agents: https://yutori.com/blog/the-bitter-lesson-for-web-agents

2026 will reward AI products that get three things right: trust, cost, and orchestration. This episode looks ahead at how those forces are reshaping AI product strategy—and what teams need to pay attention to now.
Brittany and Arpy are joined by Yaddy Arroyo, who has spent a decade designing multimodal AI systems in financial services, where reliability and governance are table stakes. She's also been one of the key community builders amongst the design community who are leaders within AI orgs.
Together, they reflect on what the last two years of AI adoption revealed and how those lessons are directly informing decisions teams are making in 2026.

Why trust now shapes AI product success
Orchestration matters more than prompting
Token costs quietly reshape UX decisions
When small models outperform large ones
How AI design roles must evolve in 2026

Episode chapters
01:21 Reflecting on Two Years of AI Adoption
02:52 The Rise of Copilot and AI's Impact on Creativity
03:37 Challenges and Concerns with AI Safety
04:24 Designing AI for Human-Centric Use Cases
04:53 Meta's Investment and Intelligence as a Service
09:25 Hallucinations and the Reliability of LLMs
11:14 The Business Value and Limitations of Gen AI
18:55 Founders and the Rush to Monetize AI
19:25 Token Optimization and UX Challenges
21:31 Personalizing AI Interactions
21:48 Challenges in AI Adoption
22:27 PH One's AI Solutions
22:53 The Orchestration Problem
24:22 AI's Role in Everyday Tasks
26:08 AI in UX and Design
27:55 Future of AI and Small Language Models
30:35 Human in the Loop and UI Generators
37:35 Accountability and AI's Future
42:39 Closing Thoughts and Future Directions

The conversation connects early generative AI optimism with today’s realities—probabilistic systems, rising costs, and scaling pressure—and surfaces where momentum is building, from smaller models to on-device intelligence.
This episode also marks Episode 50 of Design of AI and two years of conversations with builders, researchers, and leaders shaping AI-powered products—follow the podcast to stay ahead as this next phase unfolds
.
About PH1
The Design of AI podcast is brought to you by PH1, an AI strategy consultancy. PH1 has worked with the biggest corporations in tech to redefine CX in the era of AI through strategic research, prototyping, and aligning product to power. Visit ph1.ca to ask about your project.

Go Deeper
For deeper, unfiltered thinking on AI strategy, governance, and product decisions, our Substack (https://designofai.substack.com) is the best place to follow our work. It’s where we go beyond the episodes—breaking down what’s actually changing, what’s overhyped, and what leaders should do next.

Connect with the Hosts
Contact Arpy if you’re navigating AI product strategy, platform architecture, orchestration, or high-stakes system decisions that need to scale.

Contact Brittany if you need clarity on AI UX, research, service design, or evaluating whether an AI product is actually delivering value for users.