Conducting Skills Assessments

ACING a job interview does not guarantee high performance on the job ... and poor performance at one does not rule out success ... When asked during a job interview to “tell me about yourself”, some people freeze. Their mind goes blank and palms go all sweaty. They forget half of what they know makes them great at what they do. On the job, the same person is very different. They are unstoppable, solve problems, manage even the trickiest of clients and take on tasks others avoid. Yet none of that shows up when they are sitting in front of an interview panel. Being good at interviews and being good at the job are two very different things. We tend to assume the person who nails the interview will also nail the job. But that logic is flawed. More often than not, interviews reward confidence rather than competence. They favour quick thinkers but not necessarily deep thinkers, leaving quieter candidates or those who struggle under pressure at risk of being overlooked. It is not that these candidates lack skill or motivation. They just do not shine in artificial settings that favour polish over potential. As a result highly capable people miss out even though their resumes stack up and references are glowing. But they stumble through awkward introductions, second-guess their answers and walk away from interviews feeling like they have blown it. The interview system just does not play to their strengths. Interviews are a blunt instrument when it comes to assessing the full scope of someone’s ability. Their focus on questions like “tell me about a time when …” often lead to rehearsed, generic answers rather than useful insight. Even candidates who have the gift of the gab can leave an interview feeling like it was more an interrogation than a conversation. It is easy to dismiss candidates who are challenged by the traditional interview as being not interview-ready. Maybe the better question is whether the interview is job-ready. Most roles do not require someone to sit in a room and answer abstract questions about hypothetical situations. They demand persistence, teamwork, judgment and follow-through. They require people who can get on with the task, not just talk about how they would do it. For employers, there is a growing case for thinking beyond traditional interviews. Adding in work samples, job trials or even a casual chat over coffee can give a more rounded sense of a candidate’s capabilities. The risk is not just passing over a perfectly good candidate – it is missing out on someone who would have made a lasting contribution. Some of the best workers are simply not the best interviewees. And until we stop confusing interview ability with job suitability, we will keep getting it the wrong way around. #work #worplace #humanresources #management #leadership #aimwa #recruitment Cartoon used under licence: CartoonStock

If you’re an AI engineer trying to understand how reasoning actually works inside LLMs, this will help you connect the dots. Most large language models can generate. But reasoning models can decide. Traditional LLMs followed a straight line: Input → Predict → Output. No self-checking, no branching, no exploration. Reasoning models introduced structure, a way for models to explore multiple paths, score their own reasoning, and refine their answers. We started with Chain-of-Thought (CoT) reasoning, then extended to Tree-of-Thought (ToT) for branching, and now to Graph-based reasoning, where models connect, merge, or revisit partial thoughts before concluding. This evolution changes how LLMs solve problems. Instead of guessing the next token, they learn to search the reasoning space- exploring alternatives, evaluating confidence, and adapting dynamically. Different reasoning topologies serve different goals: • Chains for simple sequential reasoning • Trees for exploring multiple hypotheses • Graphs for revising and merging partial solutions Modern architectures (like OpenAI’s o-series reasoning models, Anthropic’s Claude reasoning stack, DeepSeek R series and DeepMind’s AlphaReasoning experiments) use this idea under the hood. They don’t just generate answers, they navigate reasoning trajectories, using adaptive depth-first or breadth-first exploration, depending on task uncertainty. Why this matters? • It reduces hallucinations by verifying intermediate steps • It improves interpretability since we can visualize reasoning paths • It boosts reliability for complex tasks like planning, coding, or tool orchestration The next phase of LLM development won’t be about more parameters, it’ll be about better reasoning architectures: topologies that can branch, score, and self-correct. I’ll be doing a deep dive on reasoning models soon on my Substack- exploring architectures, training approaches, and practical applications for engineers. If you haven’t subscribed yet, make sure you do: https://lnkd.in/dpBNr6Jg ♻️ Share this with your network 🔔 Follow along for more data science & AI insights

Most high-performing professionals approach interviews the wrong way. They show up trying to prove they’re qualified, listing accomplishments, reciting prepared answers, hoping they say the “right” thing. But here’s the truth: If you made it to the interview, they already believe you can do the job. Now they need to know if you’re the right fit to lead, partner, and grow with the business. Here’s how to shift your approach: 1. Don’t just answer questions, guide the conversation. Talk like someone who belongs at the decision-making table. 2. Share insights, not just experience. Know the company’s pain points and speak directly to how you’ll solve them. 3. Align with the future, not just your past. Frame your story around where they’re headed, and how you fit into that vision. 4. Ask questions that show you’re evaluating them too. Senior professionals don’t just want any offer. They want the right one. The best interviews aren’t about performance. They’re about positioning. You’re not there to earn approval. You’re there to show up as the expert they’ve been looking for.

The Hidden Interview Questions You Didn't Know Were Being Asked I spent Tuesday meeting five candidates for a senior sales role. By the time the last one left, I noticed something fascinating. Each person was answering questions I never actually asked. Here's what I mean: When Sarah arrived 15 minutes early, she showed me she values preparation and respects others' time. When Michael kept checking his phone, he told me his priorities might be elsewhere. And when Emma asked thoughtful questions about our company culture, she revealed her interest went beyond just getting a pay cheque. You see, the interview starts well before you sit down. As an experienced headhunter, I can tell you that hiring managers are constantly gathering data points that candidates don't realise are being assessed. Some of these hidden assessment moments include: How you treat the receptionist or junior staff Whether you researched the company properly Your body language while waiting How you handle unexpected hiccups (like a delayed interviewer) The questions you ask at the end I once worked with a client who rejected an otherwise perfect candidate because they were dismissive to the office assistant. That 30-second interaction outweighed an hour of brilliant answers. Think about your last interview. What signals might you have sent without knowing it? That email you took three days to respond to? The thank-you note you forgot to send? The next time you're up for a job, remember that everything from your arrival to your departure is part of the assessment. The most successful candidates understand that actions speak louder than rehearsed answers. #Recruitment #HiringTips #TalentAcquisition

🦜Beyond Stochastic Parrots: Understanding How LLMs Really "Think". Recently, I've been fascinated by the mechanics of how LLMs choose/sample their next words. Some spicy stuff going on in the land of LLM samplers. Here's what I've learned about token selection in LLMs: 🔹For each potential word, the model assigns probabilities across its entire vocabulary ~120,000 tokens 🔹The selection process can be tuned through "temperature": • Low temperature = focused, precise responses • High temperature = more creative, exploratory outputs 🔍 What's exciting is the emergence of dynamic temperature adjustment. The idea is to automatically adapt the creativity level(temperature) of the LLM based on the models confidence(entropy): 🔹If you have high LogProbs → High confidence of next token → Low Entropy State → Increase Temp → We can afford to increase temperature and force the model to be more creative. 🔹If you have low LogProbs → Low confidence of next token → High Entropy State → Decrease Temp → We need to reduce the likelihood of the model going off the rails and rein in the creativity. This adaptive approach allows LLMs to be both precise when needed and creative when appropriate - much like human thinking! The next time someone tells you LLMs are just "stochastic parrots," remember: they're implementing sophisticated probability-based decision making that can dynamically adjust to the task at hand. Resources: 🔸Nice Explanation of LLM Samplers: https://lnkd.in/gjHQ4EVA 🔸Slides on Dynamic Temperature: https://lnkd.in/gBy3V4uX 🔸Notebook on Dynamic Temp Sampler: https://lnkd.in/geTwBwsV 🔸Deepmind Paper on Dynamic Sampling: https://lnkd.in/gyYxHCN2 🔸MinP Sampling Paper: https://lnkd.in/gD3uzj8M 🔸Entropix – Entropy based sampling: https://lnkd.in/gAhxz_e9

If your legal AI makes a “judgment,” it’s not just following rules. 𝗜𝘁’𝘀 𝗲𝘅𝗽𝗿𝗲𝘀𝘀𝗶𝗻𝗴 𝗮 𝘄𝗼𝗿𝗹𝗱𝘃𝗶𝗲𝘄. Most software operates on if/then logic. Input X produces Output Y. Every time. Deterministic. LLMs are fundamentally different. They're probabilistic, and new research shows just how much that matters. A recent paper, "Evaluative Fingerprints" by Wajid N., studied 9 frontier LLMs evaluating the same content using the same rubric. The results are striking: 𝗜𝗻𝘁𝗲𝗿-𝗺𝗼𝗱𝗲𝗹 𝗮𝗴𝗿𝗲𝗲𝗺𝗲𝗻𝘁 𝘄𝗮𝘀 𝗻𝗲𝗮𝗿 𝘇𝗲𝗿𝗼. Yet individual models were remarkably consistent 𝘄𝗶𝘁𝗵 𝘁𝗵𝗲𝗺𝘀𝗲𝗹𝘃𝗲𝘀, just not with each other. The researchers could identify which model produced an evaluation with 𝟴𝟵.𝟵% 𝗮𝗰𝗰𝘂𝗿𝗮𝗰𝘆 based solely on its scoring patterns. Even GPT-4.1 and GPT-5.2 (same provider, different versions) were distinguishable 99.6% of the time. The paper calls this the "reliability paradox": models don't agree on what "good" means, but 𝘁𝗵𝗲𝘆'𝗿𝗲 𝘀𝗼 𝗰𝗼𝗻𝘀𝗶𝘀𝘁𝗲𝗻𝘁 𝗶𝗻 𝗵𝗼𝘄 𝘁𝗵𝗲𝘆 𝗱𝗶𝘀𝗮𝗴𝗿𝗲𝗲 that their evaluation patterns function as fingerprints. 𝗪𝗵𝘆 𝘀𝗵𝗼𝘂𝗹𝗱 𝗹𝗲𝗴𝗮𝗹 𝗰𝗮𝗿𝗲? In many ways, LLMs behave like people, who are also probabilistic. We've always known that different lawyers assess risk differently, interpret contract language differently, prioritize issues differently. Now we're deploying AI with the same characteristics. Consider a legal department implementing an agentic workflow that escalates matters based on risk assessment. This research suggests the choice of model isn't an implementation detail. It's a substantive decision that shapes outcomes. As we build agentic processes that assert judgment (contract review, risk triage, compliance monitoring), we need to understand that model selection is a methodological choice with real consequences. The question isn't whether to use AI for legal judgment. 𝗜𝘁'𝘀 𝘄𝗵𝗲𝘁𝗵𝗲𝗿 𝘄𝗲 𝘂𝗻𝗱𝗲𝗿𝘀𝘁𝗮𝗻𝗱 𝘄𝗵𝗮𝘁 𝘁𝗵𝗲𝗼𝗿𝘆 𝗼𝗳 𝗷𝘂𝗱𝗴𝗺𝗲𝗻𝘁 𝘄𝗲'𝗿𝗲 𝗲𝗻𝗰𝗼𝗱𝗶𝗻𝗴 𝘄𝗵𝗲𝗻 𝘄𝗲 𝗽𝗶𝗰𝗸 𝗮 𝗺𝗼𝗱𝗲𝗹.

Small variations in prompts can lead to very different LLM responses. Research that measures LLM prompt sensitivity uncovers what matters, and the strategies to get the best outcomes. A new framework for prompt sensitivity, ProSA, shows that response robustness increases with factors including higher model confidence, few-shot examples, and larger model size. Some strategies you should consider given these findings: 💡 Understand Prompt Sensitivity and Test Variability: LLMs can produce different responses with minor rephrasings of the same prompt. Testing multiple prompt versions is essential, as even small wording adjustments can significantly impact the outcome. Organizations may benefit from creating a library of proven prompts, noting which styles perform best for different types of queries. 🧩 Integrate Few-Shot Examples for Consistency: Including few-shot examples (demonstrative samples within prompts) enhances the stability of responses, especially in larger models. For complex or high-priority tasks, adding a few-shot structure can reduce prompt sensitivity. Standardizing few-shot examples in key prompts across the organization helps ensure consistent output. 🧠 Match Prompt Style to Task Complexity: Different tasks benefit from different prompt strategies. Knowledge-based tasks like basic Q&A are generally less sensitive to prompt variations than complex, reasoning-heavy tasks, such as coding or creative requests. For these complex tasks, using structured, example-rich prompts can improve response reliability. 📈 Use Decoding Confidence as a Quality Check: High decoding confidence—the model’s level of certainty in its responses—indicates robustness against prompt variations. Organizations can track confidence scores to flag low-confidence responses and identify prompts that might need adjustment, enhancing the overall quality of outputs. 📜 Standardize Prompt Templates for Reliability: Simple, standardized templates reduce prompt sensitivity across users and tasks. For frequent or critical applications, well-designed, straightforward prompt templates minimize variability in responses. Organizations should consider a “best-practices” prompt set that can be shared across teams to ensure reliable outcomes. 🔄 Regularly Review and Optimize Prompts: As LLMs evolve, so may prompt performance. Routine prompt evaluations help organizations adapt to model changes and maintain high-quality, reliable responses over time. Regularly revisiting and refining key prompts ensures they stay aligned with the latest LLM behavior. Link to paper in comments.

If you use LLM-as-judge, this one is worth reading. (bookmark it) It's actually one of the most effective ways to use LLM-as-a-Judge for evals. Holistic judge scores hide both their reasoning and their ceiling effects. BINEVAL decomposes each evaluation criterion into atomic yes-or-no questions, answers each independently per output, then aggregates the verdicts into calibrated multi-dimensional scores. Every question-level verdict is inspectable, so you can diagnose exactly why an output scored low, and the same verdicts feed straight back as targeted prompt-improvement signal. Across SummEval, Topical-Chat, and QAGS, it matches or beats UniEval and G-Eval, training-free, with especially strong results on factual consistency.

Major preprint just out! We compare how humans and LLMs form judgments across seven epistemological stages. We highlight seven fault lines, points at which humans and LLMs fundamentally diverge: The Grounding fault: Humans anchor judgment in perceptual, embodied, and social experience, whereas LLMs begin from text alone, reconstructing meaning indirectly from symbols. The Parsing fault: Humans parse situations through integrated perceptual and conceptual processes; LLMs perform mechanical tokenization that yields a structurally convenient but semantically thin representation. The Experience fault: Humans rely on episodic memory, intuitive physics and psychology, and learned concepts; LLMs rely solely on statistical associations encoded in embeddings. The Motivation fault: Human judgment is guided by emotions, goals, values, and evolutionarily shaped motivations; LLMs have no intrinsic preferences, aims, or affective significance. The Causality fault: Humans reason using causal models, counterfactuals, and principled evaluation; LLMs integrate textual context without constructing causal explanations, depending instead on surface correlations. The Metacognitive fault: Humans monitor uncertainty, detect errors, and can suspend judgment; LLMs lack metacognition and must always produce an output, making hallucinations structurally unavoidable. The Value fault: Human judgments reflect identity, morality, and real-world stakes; LLM "judgments" are probabilistic next-token predictions without intrinsic valuation or accountability. Despite these fault lines, humans systematically over-believe LLM outputs, because fluent and confident language produce a credibility bias. We argue that this creates a structural condition, Epistemia: linguistic plausibility substitutes for epistemic evaluation, producing the feeling of knowing without actually knowing. To address Epistemia, we propose three complementary strategies: epistemic evaluation, epistemic governance, and epistemic literacy. Full paper in the first comment. Joint with Walter Quattrociocchi and Matjaz Perc.

The candidate with perfect answers lost the job to someone who said "I don't know". (Save this for your next interview) After coaching dozens of managers through interviews, I noticed something surprising: Hiring managers aren't just listening to your polished answers. They're watching how you think when you don't have one. Just last week, a client bombed the "Tell me about a time..." questions. But she still got the job. Why? Because when they asked her: "What would you do if half your team quit tomorrow?" She lit up. No script. No polish. Just raw problem-solving in real-time. She said: "I don't know... but here's how I'd figure it out." Then walked them through her thinking process. That's when it hit me: Behavioral questions are the warm-up. The real test? How you think under pressure. 𝗛𝗲𝗿𝗲'𝘀 𝘄𝗵𝗮𝘁 𝘁𝗵𝗲𝘆'𝗿𝗲 𝗮𝗰𝘁𝘂𝗮𝗹𝗹𝘆 𝗲𝘃𝗮𝗹𝘂𝗮𝘁𝗶𝗻𝗴: ✅ How you handle ambiguity Do you freeze or flow when faced with the unexpected? → They want to see you think out loud, not recite. ✅ Your questions, not your answers Smart candidates ask: "Can you share more about the context?" → That shows strategic thinking, not just storytelling. ✅ Your collaborative instincts Do you make it a conversation or a performance? → They want teammates, not actors. 𝗦𝗼 𝘄𝗵𝗮𝘁 𝘀𝗵𝗼𝘂𝗹𝗱 𝘆𝗼𝘂 𝗽𝗿𝗮𝗰𝘁𝗶𝗰𝗲 𝗶𝗻𝘀𝘁𝗲𝗮𝗱? • Ask a friend to throw you curveball scenarios (Like: "How would you handle a client who wants to fire your best performer?") • Practice saying: "Let me think through this..." • Get comfortable with 5-second pauses • Show your thinking, not just your conclusions Because here's the truth: They can train you on systems. They can't train you to think. Your rehearsed STAR story? They've heard 50 today. Your authentic thinking process? That's what gets you hired. The best candidates don't have all the answers. They have better questions. 💬 What's one interview question that completely caught you off guard? ✅ Save this post 🔄 Share it with someone prepping for interviews 🔔 Follow me for insights that challenge conventional career advice