GenAI Level UP

• From QA to AI Improvement Engineer: Navigating the Shift in the AI Era

  Quality Engineering (QE) professionals are well-positioned to transition into AI Improvement Engineering roles due to their deep knowledge of testing, quality assurance, and processes. This transition expands their role significantly – from assuring quality in products to improving the entire system that delivers them. The role demands augmenting traditional QE skillsets with knowledge of AI/ML fundamentals, MLOps, data analysis, and modern DevOps toolchains. Key technical skills include proficiency in AI/ML concepts and workflows, data handling, observability, and automation tools. QEs also need to develop strategic and soft skills, such as mastering continuous improvement methodologies like Lean Six Sigma and Toyota Kata, as well as leadership, coaching, and change management.In the age of AI, the QE role is evolving to encompass new responsibilities like AI governance, ensuring data quality for AI models, and evaluating model performance and reliability. QEs are shifting towards more advisory and strategic positions, overseeing AI agents and verifying their results. This integration of AI into QE is intended to augment human capabilities, allowing engineers to focus on more strategic aspects while AI handles routine tasks. Emerging roles for transitioning QEs include AI Reliability Engineer, MLOps Engineer, Improvement Engineer, and Engineering Excellence Architect.Ultimately, Improvement Engineering in AI is about creating a virtuous cycle, using data and AI to drive improvements, leading to better systems, all fueled by a culture that values growth and excellence. This requires cultivating a continuous improvement culture where quality is everyone's responsibility, promoting a blameless culture, and embracing a scientific mindset for experimentation. By following a roadmap of skill development and cultural change, QEs can reinvent themselves, ensuring organizations not only build the right things but also build things the right way, and keep getting better at it.

  --------  

  45:56
• Defeating Prompt Injections by Design: The CaMeL Approach

  This episode delves into CaMeL, a novel defense mechanism designed to combat prompt injection attacks in Large Language Model (LLM) agents. Inspired by established software security principles, CaMeL focuses on securing both control flows and data flows within agent operations without requiring changes to the underlying LLM. We'll explore CaMeL's architecture, which features explicit isolation between two models: a Privileged LLM (P-LLM) responsible for generating pseudo-Python code to express the user's intent and orchestrate tasks, and a Quarantined LLM (Q-LLM) used specifically for parsing unstructured data into structured formats using predefined schemas, without tool access. The system utilizes a custom Python interpreter that executes the P-LLM's code, tracking data provenance and enforcing explicit security policies based on capabilities assigned to data values. These policies, often expressed as Python functions, define what actions are permissible when calling tools. We'll also touch upon the practical challenges and the system's iterative approach to error handling, where the P-LLM receives feedback on execution errors and attempts to correct its generated code. Tune in to understand how this design-based approach leveraging dual LLMs, a custom interpreter, policies, and capabilities aims to build more secure LLM agents.

  --------  

  28:33
• The Blueprint Behind Google—and the Future of AI Retrieval

  In this episode, we unearth the untold story of how Google engineered one of the most powerful information retrieval systems in history—and why its early design principles still echo through today’s cutting-edge AI.From scavenged servers to hyper-optimized global systems, we follow Google's relentless pursuit of scale, speed, and precision, drawing lessons from Jeff Dean’s landmark 2009 WSDM talk. (video, slides)You’ll hear how seemingly ancient struggles—handling billions of documents, lightning-fast retrieval, caching, real-time updates—directly mirror the modern battles of building Retrieval-Augmented Generation (RAG) systems and AI models today.This isn't just nostalgia. It’s a playbook for the next generation of intelligent systems. Join us to connect the dots between the bold experiments of Google's early days and the challenges facing AI engineers right now—and tomorrow.Chapters include:Why scaling breaks everything—and how to fix itDocument vs. word partitioning warsTricks with doc IDs and early stoppingBirth of caching: the unsung hero of performanceHow a 10,000× speedup rewired web searchLessons from moving the entire index into RAMFrom Universal Search to Universal AI KnowledgeThe eternal race: real-time updates vs. a changing worldBig Idea:The problems that built Google are the problems that will build the future of AI.

  --------  

  18:43
• Running Down a Dream: Bill Gurley’s Roadmap to a Career You Love

  [Level Up With Gen AI Series] We believe in harnessing the power of AI to unlock human potential.We unpack Bill Gurley’s legendary talk “Running Down a Dream”—a no-fluff, high-octane blueprint for building a career that truly lights you up. Through vivid stories of Bobby Knight, Bob Dylan, and Danny Meyer, Gurley reveals five timeless principles that separate the dreamers from the doers:Find your real passion (not your LinkedIn-approved one)Hone your craft with obsessive depthSeek out mentors—and earn their investmentEmbrace your peers as collaborators, not competitorsGive credit, pay it forward, stay humanThis isn’t just career advice. It’s a philosophy for a life well-lived.Whether you’re mid-pivot, just starting out, or simply stuck—this episode is your spark. The dream won’t run to you. So let’s run it down.

  --------  

  14:12
• The Divine Discontent (Constructive Dissatisfaction): Inside Ogilvy's Creative Habits

  [Level Up With Gen AI Series] We believe in harnessing the power of AI to unlock human potential.Are you ready to take your creativity to the next level? In this episode, we explore the philosophy of David Ogilvy, one of the greatest advertising minds of all time. Learn about his concept of 'Divine Discontent' and how it can fuel your ambition, inspire courageous action, and unlock your full potential. We’ll also delve into the specific habits and principles that powered Ogilvy & Mather’s success, giving you the tools to elevate your own work and achieve extraordinary results.The 8 Habits of Highly Creative Communities are explicitly listed in the sources. These habits, along with their corresponding "evil twins" are:* Courage (counters Fear)* Idealism (counters Expedience / The Status Quo)* Curiosity (counters Boring)* Playfulness (counters Tyranny of Politeness)* Candour (counters Cold Arithmetic)* Intuition (counters Bureaucracy)* Free‑Spiritedness (counters Giving In)* Persistence (has no listed "evil twin"; it overcomes obstacles)These eight creative virtues are presented as a way to cultivate David Ogilvy's concept of "divine discontent" within an organization. The practice of these habits aims to prevent "smugness" and encourage a constant striving for excellence, moving beyond being merely "Very Good" to becoming "Very, Very, Very, Very, Very Good". The book argues that embracing these habits can help Ogilvy & Mather maintain its creative edge and remain "gorgeous" despite its size.

  --------  

  10:49

Más podcasts de Tecnología

Podcasts a la moda de Tecnología

Acerca de GenAI Level UP