Implementing Voice of Customer Programs

Last week, I described four design patterns for AI agentic workflows that I believe will drive significant progress: Reflection, Tool use, Planning and Multi-agent collaboration. Instead of having an LLM generate its final output directly, an agentic workflow prompts the LLM multiple times, giving it opportunities to build step by step to higher-quality output. Here, I'd like to discuss Reflection. It's relatively quick to implement, and I've seen it lead to surprising performance gains. You may have had the experience of prompting ChatGPT/Claude/Gemini, receiving unsatisfactory output, delivering critical feedback to help the LLM improve its response, and then getting a better response. What if you automate the step of delivering critical feedback, so the model automatically criticizes its own output and improves its response? This is the crux of Reflection. Take the task of asking an LLM to write code. We can prompt it to generate the desired code directly to carry out some task X. Then, we can prompt it to reflect on its own output, perhaps as follows: Here’s code intended for task X: [previously generated code] Check the code carefully for correctness, style, and efficiency, and give constructive criticism for how to improve it. Sometimes this causes the LLM to spot problems and come up with constructive suggestions. Next, we can prompt the LLM with context including (i) the previously generated code and (ii) the constructive feedback, and ask it to use the feedback to rewrite the code. This can lead to a better response. Repeating the criticism/rewrite process might yield further improvements. This self-reflection process allows the LLM to spot gaps and improve its output on a variety of tasks including producing code, writing text, and answering questions. And we can go beyond self-reflection by giving the LLM tools that help evaluate its output; for example, running its code through a few unit tests to check whether it generates correct results on test cases or searching the web to double-check text output. Then it can reflect on any errors it found and come up with ideas for improvement. Further, we can implement Reflection using a multi-agent framework. I've found it convenient to create two agents, one prompted to generate good outputs and the other prompted to give constructive criticism of the first agent's output. The resulting discussion between the two agents leads to improved responses. Reflection is a relatively basic type of agentic workflow, but I've been delighted by how much it improved my applications’ results. If you’re interested in learning more about reflection, I recommend: - Self-Refine: Iterative Refinement with Self-Feedback, by Madaan et al. (2023) - Reflexion: Language Agents with Verbal Reinforcement Learning, by Shinn et al. (2023) - CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing, by Gou et al. (2024) [Original text: https://lnkd.in/g4bTuWtU ]

Claude Code just shipped a feature that lets you walk away from a pull request and come back to it fully fixed. You toggle auto-fix on and leave. Claude watches your continuous integration pipeline in real time. - If something fails, it pushes a fix.  - If a reviewer leaves a comment, it addresses the feedback automatically. You come back to a green pull request ready to merge. I have been thinking about why this works so well. Continuous integration pipelines already defined what "good" looks like years ago. Every test passes or fails. Every build is green or red. Auto-fix just runs the improvement loop on top of that existing scoring criteria. That is the entire lesson. AI automation works best when there is a clear, measurable definition of quality already in place. The workflows that already have scoring criteria built in will reach full autonomy first. Everything else will follow once the quality definition is clear enough. Full docs on how to set up auto-fix are linked in the comments.

🚀 From Hackathon to Production: Introducing the AI Interview Companion Agent One of the most exciting outcomes of our recent Eightfold AI Hackathon is already live in production at scale. Interviewing today puts an enormous cognitive load on interviewers — remembering what to ask, tracking what was said, probing deeper, spotting gaps or contradictions, and then writing detailed feedback after the fact. The result is inconsistency, fatigue, and delayed decisions. So we built the AI Interview Companion Agent — a real-time AI assistant that works alongside interviewers to bring structure, consistency, and intelligence into every interview. 🔍 What the AI Interview Companion does * Nudge Agent: Surfaces critical unasked questions every ~10 minutes * Response Agent: On-demand guidance like “What should I ask next?” * Smart Feedback: Auto-generates the full feedback form using transcripts + competencies * Real-Time Transcript: Speaker identification with precise timestamps 💡 The impact we’re seeing * ⏱️ 30–40 minutes saved per interview * 📈 40% improvement in question coverage * ⚡ Feedback turnaround cut from 48 hours → <12 hours * 💪 95%+ interviewer confidence * 🎓 Enables junior interviewers to confidently run senior-level interviews What started as a hackathon idea is now a production-grade agent, already helping teams interview better, faster, and more fairly. This is just the beginning. The future of interviewing is real-time, intelligent, and guided — and we’re excited to keep building agents that raise the bar for hiring everywhere.

Most AI agents have no idea if their answers are actually good. Neither do the companies that built them. They ship an agent. They watch ticket volume drop. They call it a win. But deflection isn't resolution. And silence isn't satisfaction. When we started building at Maven, answer evaluation was one of the first things we built. Not the agent. Not the integrations. The evaluation layer. Why? Because I spent years at Google watching this play out. No ranking algorithm made it to production without evaluation. Thousands of queries, scored by humans, before a single end user saw the result. That's how Google News & Search got good. Most AI agent companies skip this step entirely. We built an agentic version of that same rigor. An AI system that evaluates every answer the agent gives not just "did the customer stop asking?" but "was this actually correct, complete, and helpful? if not, why?" That gave us something most companies don't have: a metric. And here's the thing about metrics once you have one, you can move it. You build a flywheel. You find the gaps. You close them. You measure again. Every cycle is ROI-based because you know exactly where to invest. Without it, you're flying blind. You might hit 50% automation and have no idea why you're stuck there. That's how our customers get to 90%+ automation rates. Not by building a better model. By building a better feedback loop. The companies winning at AI aren't the ones with the fanciest architecture. They're the ones who know, with precision, when their agent is wrong.

PART 3: HR Workflows getting automated using Agents in 2025. WORKFLOW 3: Employee Feedback & Review Automation THE PROBLEM: Traditional employee satisfaction surveys suffer from low engagement, biased responses, and lack of real-time insights. Employees often feel surveys are not personalized, making them less likely to participate. Additionally, companies fail to act on feedback promptly, reducing trust in the process. SOLUTION: An AI-powered Employee Satisfaction Survey System automates feedback collection, analysis, and action planning. AI agents conduct sentiment analysis, categorize feedback, and provide real-time insights to HR teams. The system ensures anonymity, dynamic survey questions, and trend analysis over time, leading to better decision-making and increased employee trust in the survey process. Tech Stack :   LLM: GPT-4 & Claude 3   Survey Distribution: Slack API, Google Forms API, HRMS (Workday, BambooHR) Vector Database: Qdrant, Memory Modules: Short-term & Long-term  Agent Framework: Built using Lyzr AI’s Agent API   Dashboard & Reporting: Streamlit & Tableau  Agents: Survey Distribution Agent: Automates survey sending via HR tools Survey Response Analysis Agent: Extracts sentiment, themes, and issues  Employee Feedback Insights Agent: Generates structured reports+ insights AI HR Coach: Provides employees with automated feedback and coaching Trend Analysis Agent: Tracks changes in employee satisfaction over time. #HRworkflows

We built an AI agent system that takes 40% of inbound work off our GTM team   Most teams don’t struggle with inbound replies because writing is hard.   They struggle because replies show up without context, without prioritization, and without a clear way to decide what should happen next.   This is the AI agent orchestration we use so humans only touch the conversations that actually matter.   Here’s how it works, step by step.   STEP 1: CAPTURE ALL REPLIES → Email replies, LinkedIn replies, and form replies are captured in one place using Masterinbox.com → This removes inbox fragmentation and ensures no conversation is missed   STEP 2: SCORE THE LEAD → Each reply is scored from 1 to 10 based on intent and fit using AI scoring in Clay or n8n → This creates an early priority signal before a human ever reads the message   STEP 3: ATTACH CONTEXT → Each reply is linked to the correct contact, account, deal, and last message using CRM data from HubSpot or Attio → This ensures responses are grounded in full conversation history   STEP 4: CLASSIFY INTENT → Replies are classified as interested, not now, out of office, spam, or other using ChatGPT or Gemini → The output is a clear label that downstream logic can reliably act on   STEP 5: PICK THE RESPONSE PATH → Intent and score together determine whether to auto-reply, ask follow-up questions, book a meeting, or route to a human → Decision rules are managed in n8n or Clay   STEP 6: SEND THE RESPONSE DRAFT → Responses are generated using GPT or Gemini → Messages are either sent automatically or prepared for human approval based on risk and value   STEP 7: LOG THE OUTCOME → Conversation outcomes are written back to the CRM in HubSpot or Attio → Attribution is updated so future decisions improve over time   That’s the full orchestration.   Not an AI that replaces humans. Not a generic reply automation.   Just a system that keeps humans focused on the conversations that actually matter.

I just built an automation that sends me AI coaching feedback after every sales call. I started doing some AI consulting lately. After every call with prospects, I wanted feedback, hence this workflow was born. Here's what it does: When a call finishes, it pulls the transcript, sends it to Claude, and delivers structured feedback to Slack. Scores the call, highlights what went well, points out what I missed, and suggests specific improvements. A few thing that worked well: I designed the prompt for the AI to give specific feedback on actual moments from the call which has been great: - i.e. "At 11:32 when the prospect mentioned budget concerns, you moved on instead of digging deeper." Sometimes the webhook-based trigger doesn't come through, so it's good to build a backup option. In my case, I built a manual trigger to do polling. The tech stack is pretty simple, but works well. Fathom API, n8n for orchestration, Claude for the analysis, Slack for delivery. If you take a lot of sales calls and want consistent feedback, this is worth building. I made a quick video walking through how it works, so you can do it too.

Here’s how you can scale scenario feedback across dozens of situations without writing every feedback response by hand. That’s the part of the WhiskerBeans Café leadership training I’ve been building next. One of the hardest parts of creating AI generated scenario-based learning is generating effective feedback. With AI-generated scenarios, one of the biggest challenges is answer quality. Sometimes the model generates three “meh” options where none of them is the clear best choice. Everything sounds kind of reasonable, but nothing teaches what strong judgment actually looks like. So the learner is left thinking: “Okay…what was I supposed to do?” So I rewrote the prompt so the model has to generate structured coaching outcomes for every answer choice, automatically, every time with one answer always being the best one. Here’s how it works: ➡️ The app pulls recent customer reviews from a store location   ➡️ An AI workflow generates one realistic scenario based on those patterns   ➡️ Each answer choice includes its own outcome feedback   ➡️ The feedback explains what happens to the learner based on the choice ➡️ It ends with a takeaway the lead can apply on the job  That means the learner immediately sees coaching tied to their decision, without needing another AI call. I also added a retry mechanic that stays inside the same scenario. ➡️ Retry lets the learner try a different approach   ➡️ The app reuses the feedback already returned for each option  Next improvement I’m adding now is clear visual cues on the feedback screen. One choice will be flagged as the strongest answer, and the front end will use that to show an immediate correct or incorrect indicators, so learners know right away if they picked the best response. More WhiskerBeans updates soon! #LearningDesign #ScenarioBasedLearning #AIinLearning #GenAI #LeadershipDevelopment #eLearning #InstructionalDesign

Someone on our admin team was spending hours every week creating a brand new Microsoft Form for every single class we delivered. A different form. For every class. That meant our feedback data lived in dozens of disconnected places, and when our customer success team needed to summarize feedback at scale, they had no clean way to do it. The data was everywhere and nowhere at the same time. The fix came from an unexpected place. I was playing around in Microsoft Forms one day while prepping for a class and noticed something I hadn't paid attention to before. You can pre-fill form responses and share that pre-filled link with people. That one detail sparked the whole thing. We already had all the session information sitting in a SharePoint List. Course title, start and end times, company name, instructor, session type, all of it. Why create a new form every time when we could dynamically build a pre-filled link using data that already existed? So I built it. Here's what the flow does now: → Every day at 7 AM, the flow runs and checks the SharePoint List for sessions scheduled that day → For each session found, it builds a custom pre-filled Microsoft Forms link using the session details already stored in SharePoint → A single evaluation link is generated per class with the relevant information already populated → Automated emails go out to the instructor and admin team with the link and clear instructions on what action to take Here are a few details I made sure to get right: ★ The pre-filled link pulls directly from the SharePoint List columns so the data in the form always matches the data in our schedule, no copy-paste errors ★ Emails are structured to clearly surface what action needs to be taken so instructors aren't left guessing ★ Because all responses now feed into one form, our customer success team can filter and summarize feedback across all sessions without digging through dozens of separate forms Our admin team got hours back every week. Our customer success team can now surface feedback data quickly and accurately when clients ask for it. And the whole thing runs without anyone touching it. This is the kind of automation I love to build. It didn't require building something complicated. It required noticing one small detail in a tool we already had and asking what was possible from there. Let's start building!

If you're not building feedback loops, you're flying blind. At VAUNT, we’re approaching 80 active real estate teams using the platform week over week. Mind you, they’re using it to run one of the strongest parts of their business: sales, marketing, and customer interactions. The way they rely on the product already feels like an enterprise solution. As we grow, one challenge has become clear: it gets harder to stay close to users, and that’s exactly when you can’t afford to lose touch. When we launched our AI profiling feature, we asked: How do we know, fast, if it’s helping close more deals? We kept it simple. A thumbs up / thumbs down, like YouTube. Since we rolled it out we receive a handful of thumbs up per day. But we're equally interested in the thumbs down. Every thumbs down triggers an auto follow-up. Every response feeds into product. It’s fast, low-friction, and actionable. And way better than waiting for feedback calls. We’ve made a few mistakes learning how to close the loop properly (will share more on that soon), but one lesson is clear: No feedback loop = no iteration speed = no edge. This post is inspired by Erica Wenger 🏕️ to show more of what’s happening behind the scenes, I promised I’d share more of how we build and operate at VAUNT. This is just a glimpse. If you're building, how are you learning from your users? Would love to trade notes.