Developing Experience-Focused KPIs

Over the last year, I’ve seen many people fall into the same trap: They launch an AI-powered agent (chatbot, assistant, support tool, etc.)… But only track surface-level KPIs — like response time or number of users. That’s not enough. To create AI systems that actually deliver value, we need 𝗵𝗼𝗹𝗶𝘀𝘁𝗶𝗰, 𝗵𝘂𝗺𝗮𝗻-𝗰𝗲𝗻𝘁𝗿𝗶𝗰 𝗺𝗲𝘁𝗿𝗶𝗰𝘀 that reflect: • User trust • Task success • Business impact • Experience quality    This infographic highlights 15 𝘦𝘴𝘴𝘦𝘯𝘵𝘪𝘢𝘭 dimensions to consider: ↳ 𝗥𝗲𝘀𝗽𝗼𝗻𝘀𝗲 𝗔𝗰𝗰𝘂𝗿𝗮𝗰𝘆 — Are your AI answers actually useful and correct? ↳ 𝗧𝗮𝘀𝗸 𝗖𝗼𝗺𝗽𝗹𝗲𝘁𝗶𝗼𝗻 𝗥𝗮𝘁𝗲 — Can the agent complete full workflows, not just answer trivia? ↳ 𝗟𝗮𝘁𝗲𝗻𝗰𝘆 — Response speed still matters, especially in production. ↳ 𝗨𝘀𝗲𝗿 𝗘𝗻𝗴𝗮𝗴𝗲𝗺𝗲𝗻𝘁 — How often are users returning or interacting meaningfully? ↳ 𝗦𝘂𝗰𝗰𝗲𝘀𝘀 𝗥𝗮𝘁𝗲 — Did the user achieve their goal? This is your north star. ↳ 𝗘𝗿𝗿𝗼𝗿 𝗥𝗮𝘁𝗲 — Irrelevant or wrong responses? That’s friction. ↳ 𝗦𝗲𝘀𝘀𝗶𝗼𝗻 𝗗𝘂𝗿𝗮𝘁𝗶𝗼𝗻 — Longer isn’t always better — it depends on the goal. ↳ 𝗨𝘀𝗲𝗿 𝗥𝗲𝘁𝗲𝗻𝘁𝗶𝗼𝗻 — Are users coming back 𝘢𝘧𝘵𝘦𝘳 the first experience? ↳ 𝗖𝗼𝘀𝘁 𝗽𝗲𝗿 𝗜𝗻𝘁𝗲𝗿𝗮𝗰𝘁𝗶𝗼𝗻 — Especially critical at scale. Budget-wise agents win. ↳ 𝗖𝗼𝗻𝘃𝗲𝗿𝘀𝗮𝘁𝗶𝗼𝗻 𝗗𝗲𝗽𝘁𝗵 — Can the agent handle follow-ups and multi-turn dialogue? ↳ 𝗨𝘀𝗲𝗿 𝗦𝗮𝘁𝗶𝘀𝗳𝗮𝗰𝘁𝗶𝗼𝗻 𝗦𝗰𝗼𝗿𝗲 — Feedback from actual users is gold. ↳ 𝗖𝗼𝗻𝘁𝗲𝘅𝘁𝘂𝗮𝗹 𝗨𝗻𝗱𝗲𝗿𝘀𝘁𝗮𝗻𝗱𝗶𝗻𝗴 — Can your AI 𝘳𝘦𝘮𝘦𝘮𝘣𝘦𝘳 𝘢𝘯𝘥 𝘳𝘦𝘧𝘦𝘳 to earlier inputs? ↳ 𝗦𝗰𝗮𝗹𝗮𝗯𝗶𝗹𝗶𝘁𝘆 — Can it handle volume 𝘸𝘪𝘵𝘩𝘰𝘶𝘵 degrading performance? ↳ 𝗞𝗻𝗼𝘄𝗹𝗲𝗱𝗴𝗲 𝗥𝗲𝘁𝗿𝗶𝗲𝘃𝗮𝗹 𝗘𝗳𝗳𝗶𝗰𝗶𝗲𝗻𝗰𝘆 — This is key for RAG-based agents. ↳ 𝗔𝗱𝗮𝗽𝘁𝗮𝗯𝗶𝗹𝗶𝘁𝘆 𝗦𝗰𝗼𝗿𝗲 — Is your AI learning and improving over time? If you're building or managing AI agents — bookmark this. Whether it's a support bot, GenAI assistant, or a multi-agent system — these are the metrics that will shape real-world success. 𝗗𝗶𝗱 𝗜 𝗺𝗶𝘀𝘀 𝗮𝗻𝘆 𝗰𝗿𝗶𝘁𝗶𝗰𝗮𝗹 𝗼𝗻𝗲𝘀 𝘆𝗼𝘂 𝘂𝘀𝗲 𝗶𝗻 𝘆𝗼𝘂𝗿 𝗽𝗿𝗼𝗷𝗲𝗰𝘁𝘀? Let’s make this list even stronger — drop your thoughts 👇

Most teams pick metrics that sound smart… But under the hood, they’re just noisy, slow, misleading, or biased. But today, I'm giving you a framework to avoid that trap. It’s called STEDII and it’s how to choose metrics you can actually trust: — ONE: S — Sensitivity Your metric should be able to detect small but meaningful changes Most good features don’t move numbers by 50%. They move them by 2–5%. If your metric can’t pick up those subtle shifts , you’ll miss real wins. Rule of thumb: - Basic metrics detect 10% changes - Good ones detect 5% - Great ones? 2% The better your metric, the smaller the lift it can detect. But that also means needing more users and better experimental design. — TWO: T — Trustworthiness Ever launch a clearly better feature… but the metric goes down? Happens all the time. Users find what they need faster → Time on site drops Checkout becomes smoother → Session length declines A good metric should reflect actual product value, not just surface-level activity. If metrics move in the opposite direction of user experience, they’re not trustworthy. — THREE: E — Efficiency In experimentation, speed of learning = speed of shipping. Some metrics take months to show signal (LTV, retention curves). Others like Day 2 retention or funnel completion give you insight within days. If your team is waiting weeks to know whether something worked, you're already behind. Use CUPED or proxy metrics to speed up testing windows without sacrificing signal. — FOUR: D — Debuggability A number that moves is nice. A number you can explain why something worked? That’s gold. Break down conversion into funnel steps. Segment by user type, device, geography. A 5% drop means nothing if you don’t know whether it’s: → A mobile bug → A pricing issue → Or just one country behaving differently Debuggability turns your metrics into actual insight. — FIVE: I — Interpretability Your whole team should know what your metric means... And what to do when it changes. If your metric looks like this: Engagement Score = (0.3×PageViews + 0.2×Clicks - 0.1×Bounces + 0.25×ReturnRate)^0.5 You’re not driving action. You’re driving confusion. Keep it simple: Conversion drops → Check checkout flow Bounce rate spikes → Review messaging or speed Retention dips → Fix the week-one experience — SIX: I — Inclusivity Averages lie. Segments tell the truth. A metric that’s “up 5%” could still be hiding this: → Power users: +30% → New users (60% of base): -5% → Mobile users: -10% Look for Simpson’s Paradox. Make sure your “win” isn’t actually a loss for the majority. — To learn all the details, check out my deep dive with Ronny Kohavi, the legend himself: https://lnkd.in/eDWT5bDN

Three unglamorous website fixes lifted engagement by 145%. A manufacturer came to us with credibility, demand and traffic. But the post-click experience was making buyers work too hard. The product journey reflected how the business organised its ranges. Not how buyers chose. Proof was buried. The enquiry path sat too far from the moments of highest intent. So we focused on three things: On category pages, we clarified what each range was for, who used it and where to go next. On product pages, we moved useful proof closer to the decision. Then we added contextual calls to action where buyers had enough information to enquire. Not one lonely contact button at the end. We also cleaned up the templates, reduced page bloat and gave marketing more control without turning every change into a dev ticket. After the changes, traffic increased by 49%, new users grew by 20.5%, and user engagement rose by 145%. The campaign earns the visit. The website has to earn the outcome. Where is your site making good traffic work too hard?

📊 Average vs. Percentiles: A Product Manager's Guide to Feature Adoption Analysis - Ever wondered why averages can be misleading? Let's dive into a real-world scenario that showcases the power of percentiles in product analytics. 🎯 Scenario: Analyzing Adoption of a New Collaboration Feature. Imagine tracking user engagement with a new feature in the first month. Here's the engagement count data for 15 users: [2, 5, 8, 10, 12, 15, 18, 20, 25, 30, 35, 40, 45, 50, 60] 📈 Key Percentiles: - 50th (Median): 20 engagements - 75th: 35 engagements - 90th: 47.5 engagements - 100th (Max): 60 engagements 🤔🤔🤔 Why Not Use the Average? The average (25 engagements) seems simple but can be misleading: - Sensitive to Outliers: Poor or Power users skew the number. - Misrepresents Typical Behavior: Doesn't show where most users are. - Lacks Distribution Insight: Misses the bigger picture. 🚀 The Power of Percentiles for Product Managers: - Median (50th): Half of users engage ≤20 times (typical behavior) - 75th: 75% of users engage ≤35 times (great for realistic goals!) - 90th: Only 10% engage >47.5 times (your power users) 💡 Actionable Insights: - Aim to increase 75th percentile to 40 engagements/month. - Learn from 90th percentile users - what drives their high engagement? - Improve experience for below-median users to boost overall adoption/ 🎉 Key Takeaways: Percentiles offer a clearer picture of user behavior, helping you: - Identify user segments (casual vs. power users). - Prioritize improvements and plan A/B tests. - Set realistic, segmented goals. - Communicate feature performance effectively to stakeholders. Thanks Shikha Pandey for sharing this input.

How I helped a growth org 2x their impact... With one metric change! The context: - My team and I were supporting a head of growth - B2C subscription business - Trying to increase retention The problem: - Growth teams were working on different areas - Each trying to optimise slightly different metrics - Local metrics were moving but global metrics weren't - Turns out local metrics rewarded cannibalisation effects The solution: - Build a single proxy metric - Make it broad enough to connect to the global topline - But also made sensitive enough to pick up leading signals - Align all teams to use this as their primary decision-making metric - Maintain local metrics for secondary/supporting purposes The results: - Radical alignment around shared goal - Near 2x increase on the cumulative uplift generated on this metric over 6 months (backtested with past experiments) - Correlational evidence of improved movement in the global topline metric The takeaway: 𝗜𝗳 𝘆𝗼𝘂'𝗿𝗲 𝘁𝗮𝗰𝗸𝗹𝗶𝗻𝗴 𝘁𝗵𝗲 𝘀𝗮𝗺𝗲 𝗴𝗼𝗮𝗹, 𝘀𝗵𝗼𝗼𝘁 𝗳𝗼𝗿 𝘁𝗵𝗲 𝘀𝗮𝗺𝗲 𝗴𝗼𝗮𝗹 𝗽𝗼𝘀𝘁𝘀. 𝗬𝗼𝘂'𝗹𝗹 𝘀𝗰𝗼𝗿𝗲 𝗮 𝗹𝗼𝘁 𝗺𝗼𝗿𝗲. Data, used properly, is a great aligner for large teams.

Certainly, while wishlists have emerged as a valuable tool for gauging consumer interest, there are several other methods and metrics that e-commerce platforms can use to measure consumer interest: 1. Cart Abandonment Rate: Observing how many customers add products to their carts but don't complete the purchase can provide insights into potential hesitations or barriers. 2. Product Views: The number of times a product is viewed can indicate its popularity or interest level. 3. Time Spent on Page: Monitoring the average time consumers spend on product pages can hint at their level of interest. 4. Product Reviews and Ratings: A high number of reviews or ratings, even if mixed, can signify strong interest or engagement with a product. 5. Search Query Analysis: Observing which products or categories users are searching for on the platform can indicate trending interests. 6. Social Media Engagement: Shares, likes, comments, and mentions related to products can provide insights into consumer preferences. 7. Referral Traffic: Analyzing traffic from external sites or social media can show where the interest is coming from and which products are driving it. 8. Customer Surveys and Feedback: Directly asking customers about their preferences or interests can yield detailed insights. 9. Sales Data: A straightforward metric, but analyzing which products are selling the most can clearly indicate consumer interest. 10. Click-Through Rate (CTR): Observing how often people click on a product after seeing it in a recommendation or advertisement can be a strong indicator. 11. User-Generated Content: If consumers are posting pictures, videos, or blogs about a product, it showcases genuine interest and engagement. 12. Repeat Purchases: Products that are frequently repurchased can indicate high levels of satisfaction and interest. 13. Customer Service Inquiries: The number and nature of questions related to a product can offer insights into areas of curiosity or concern. 14. Heatmaps: Tools that show where users most frequently click, move, or hover on a page can help in understanding which products or sections grab their attention. 15. Newsletter and Email Open Rates: If consumers are frequently opening emails about specific products or categories, it can be an indication of their interest areas. 16. Retargeting Campaign Success: The conversion rate of retargeting campaigns can provide insights into the residual interest of consumers after their initial interaction. By leveraging a combination of these methods, brands can gain a comprehensive understanding of consumer interest, helping them to tailor their offerings and marketing strategies more effectively. #ecommerce #LinkedInNewsIndia

A critical part of journey management in any large organisation is measuring how your journeys perform. 📊 By setting clear goals, monitoring performance, identifying gaps, and measuring improvement impact, you create a continuous cycle of management and enhancement. Measurement surfaces opportunities and kickstarts improvements. 🚀 Yet many organisations struggle: data sits in silos, teams measure inconsistently, and dashboards report numbers without a coherent story. Product, marketing, sales, service, and digital teams collect valuable insights, but without a common language, they never combine into a unified performance view. The result? Plenty of activity, little clarity on what actually improves customer experience and business performance. Measuring performance along specific journeys—rather than isolated KPIs—provides the right context: the journey itself. 🗺️ This approach transforms your journey framework into an engine for improving both customer experience and business performance holistically, creating a shared structure and language where different KPIs unite. 🧭 Inspired by the Balanced Scorecard, this pragmatic 3x3 Matrix structures performance measurement across two dimensions: 👉 First, it distinguishes 3 performance metric categories: - Customer performance (behavior and sentiment) - Commercial performance (conversion, customer base, revenue) - Operational performance (cost, efficiency, reliability) 👉 Second, it distinct three journey hierachy levels: - Overall customer lifecycle - End-to-end product or service journey - Individual customer tasks These intersecting dimensions ensure each metric sits logically within a complete, coherent view. The visual below shows example metrics for all nine sections, helping you build a balanced measurement framework for journeys. This matrix delivers three immediate benefits: ✨ 1. It aligns siloed KPIs and contextualizes them into a shared journey 2. It enables drill-down and aggregation through connected KPIs across journey levels 3. It surfaces trade-offs and synergies between performance metrics A few quick tips to take into account when drafting or structuring your own journey-driven measurement framework 👇👇👇 🐌 Consider both leading and lagging indicators for a robust measurement approach that balances early warning signs with outcome metrics.  🤲 Don’t collect everything. Start with a North Star KPI for each journey, and add a small set of supporting metrics. Less is more. 💬 Always mix performance metrics with more qualitative feedback and insights that will help you determine why performance is down and how to fix it. Happy measuring! 🎉

I’ve reviewed over 100 games in the past few years. Here’s my step-by-step process for reviewing a new game: 𝟭. 𝗥𝗘𝗦𝗘𝗔𝗥𝗖𝗛 𝗖𝗢𝗠𝗣𝗘𝗧𝗜𝗧𝗢𝗥𝗦 𝗙𝗜𝗥𝗦𝗧 Before playing, I focus on understanding the competition: • Read user reviews in app stores • Play 3-5 competitor games • Take screenshots of key moments • 𝗔𝗻𝗮𝗹𝘆𝘇𝗲 𝗸𝗲𝘆 𝗮𝘀𝗽𝗲𝗰𝘁𝘀: —— Gameplay tempo – is it fast or slow? —— Monetization strategies – how are purchases introduced? —— Engagement hooks – what makes players return? —— Strengths and weaknesses 𝟮. 𝗣𝗟𝗔𝗬 𝗧𝗛𝗘 𝗚𝗔𝗠𝗘 𝗧𝗪𝗜𝗖𝗘 Each playthrough serves a specific purpose: 𝗙𝗶𝗿𝘀𝘁 𝗽𝗹𝗮𝘆𝘁𝗵𝗿𝗼𝘂𝗴𝗵 (𝟭 𝗵𝗼𝘂𝗿) – Focus on the big picture: • General feel – Does it engage from the start? • User flow – How intuitive is it for a new player? • Core loop – Do main mechanics fit together? • Goals – Are short- and long-term objectives clear? • Progression – Is there a sense of steady improvement? • Gameplay tempo – Fast or too slow? • Monetization – Do I want to pay and why? 𝗦𝗲𝗰𝗼𝗻𝗱 𝗽𝗹𝗮𝘆𝘁𝗵𝗿𝗼𝘂𝗴𝗵 (𝟮 𝗵𝗼𝘂𝗿𝘀) – Dive deeper: • Onboarding – Is the tutorial clear? • Game mechanics – Do they evolve or get repetitive? • Boosters – Impactful yet balanced? • Level design – Does it introduce new challenges? • Art and UI – Consistent and intuitive? • Monetization – When offers and ads appear? What do they offer? • Game balance – Fair resource flow? • Technical aspects – Bugs or glitches? • Social mechanics – Are they well-integrated? • Narrative – Is it interesting and well-paced? • Live operations – Which events and tasks appear, and when? → Throughout both sessions, I take detailed screenshots. → I also read user reviews here to confirm my impressions. 𝟯. 𝗔𝗡𝗔𝗟𝗬𝗭𝗘 𝗚𝗔𝗠𝗘 𝗠𝗘𝗧𝗥𝗜𝗖𝗦 Once the playthroughs are done, I review key metrics: • Retention ↳ How well does the game retain players (Day 1, Day 7, etc.)? • Engagement ↳ Average playtime, daily levels completed • Onboarding completion ↳ Tutorial completion rate • Level funnel and churn ↳ Points where players quit • Monetization ↳ How is revenue split between IAP and ads? The distribution between sources? • Difficulty ↳ Win Rate, game difficulty • Game balance ↳ Currency sources and sinks This data shows if the game meets benchmarks or needs changes. 𝟰. 𝗗𝗘𝗟𝗜𝗩𝗘𝗥 𝗦𝗧𝗥𝗨𝗖𝗧𝗨𝗥𝗘𝗗 𝗙𝗘𝗘𝗗𝗕𝗔𝗖𝗞 • No vague comments like “the game is too easy.” ↳ Instead, explain why it feels easy, e.g. “𝘗𝘭𝘢𝘺𝘦𝘳𝘴 𝘤𝘢𝘯 𝘴𝘵𝘢𝘯𝘥 𝘴𝘵𝘪𝘭𝘭 𝘸𝘪𝘵𝘩 𝘢 𝘭𝘦𝘷𝘦𝘭 1 𝘸𝘦𝘢𝘱𝘰𝘯 𝘢𝘯𝘥 𝘥𝘦𝘧𝘦𝘢𝘵 𝘦𝘯𝘦𝘮𝘪𝘦𝘴 𝘸𝘪𝘵𝘩𝘰𝘶𝘵 𝘦𝘧𝘧𝘰𝘳𝘵.” • Go beyond criticism, offer solutions: ↳ “𝘐𝘯𝘵𝘳𝘰𝘥𝘶𝘤𝘦 𝘢 𝘳𝘦𝘭𝘰𝘢𝘥 𝘮𝘦𝘤𝘩𝘢𝘯𝘪𝘤 𝘵𝘰 𝘧𝘰𝘳𝘤𝘦 𝘮𝘰𝘷𝘦𝘮𝘦𝘯𝘵. 𝘓𝘪𝘮𝘪𝘵𝘦𝘥 𝘢𝘮𝘮𝘰 𝘦𝘯𝘤𝘰𝘶𝘳𝘢𝘨𝘦𝘴 𝘦𝘹𝘱𝘭𝘰𝘳𝘢𝘵𝘪𝘰𝘯.” • Provide examples from competitors • Prioritize feedback by urgency, attach screenshots ↳ Helps developers have a clear roadmap

Your CEO asks 'how's the product doing?' and you panic because you genuinely don't know the real answer That moment of panic isn't about imposter syndrome - it's about not having the right metrics at your fingertips. You might know your MAU is growing and your latest feature got great feedback, but do you actually know if your product is healthy, profitable, and sustainable? The best PMs never get caught off guard because they track metrics that tell the full story: user behavior, business impact, and early warning signals. They can confidently answer not just "how's it doing?" but "where is it heading?" and "what should we do next?". Stop tracking vanity metrics. Start tracking product health. Here is a short guide on how to stay in touch with all metrics you and your team should keep up with: 1️⃣ BASICS Foundation metrics every PM should monitor DAU / WAU / MAU → Unique users daily, weekly, monthly Stickiness (DAU/MAU) → How often monthly users return daily Monthly New Users (MNU) → New users per month Utilization → Core feature usage frequency Feature Adoption Rate → % of users using a feature Time to Value (TTV) → Time to reach core benefit 2️⃣ PIRATE METRICS (AARRR) The classic growth framework that still works Awareness → Brand exposure and reach Acquisition → How users discover your product Activation → First successful experience Retention → % of users returning Referral → Users recommending you Revenue → Conversion to paying customers 3️⃣ NORTH STAR METRIC (NSM) Your single most important growth indicator Revenue → Direct monetization growth Customer Growth → Expanding user base Engagement Growth → Depth of usage Consumption Growth → Volume of usage User Experience → Satisfaction and usability Growth Efficiency → Output per effort 4️⃣ RETENTION & CHURN The metrics that predict your product's future Churn Rate → % of users lost Retention Rate → % of users who stay Monthly Churned Users (MCU) → Users lost per month 5️⃣ STARTUP / GROWTH METRICS Advanced metrics for scaling products Net Promoter Score (NPS) → Likelihood to recommend Feature Adoption → New feature uptake rate Customer Acquisition Cost (CAC) → Cost to acquire customers Customer Lifetime Value (CLV) → Revenue per customer lifecycle LTV:CAC Ratio → Acquisition investment efficiency Monthly Recurring Revenue (MRR) → Subscription revenue (more in the post) ---------------------- Bookmark this before you forget and spend another 3am googling 'important product metrics help please' 😭 Tag a PM friend who definitely needs to see this (we all know who) What's your current metric nightmare? Spill it below 👇