Skill Assessment Methods

𝗠𝗲𝗮𝘀𝘂𝗿𝗶𝗻𝗴 𝗟𝗲𝗮𝗱𝗲𝗿𝘀𝗵𝗶𝗽 𝗗𝗲𝘃𝗲𝗹𝗼𝗽𝗺𝗲𝗻𝘁 I've been asked this at least 3 times in the last two months. "How do I know that my leaders are improving?" This is where we distinguish knowing from application. 10% of capability comes from learning from formal sources. 20% comes from networks and interactions. 70% comes from application to portfolios and projects. One thing that sets this all apart are data points. Even if I apply skills to my projects, how do I know I did it well? Most large companies have a 360-degree or leadership assessment process in place. So, I'll share my thought process for this in case you are attempting to develop this for your own organization. Step 1: Determine organizational strategy and business outcomes. This is necessary to align expectations of desired behaviors. This is where a Balanced Scorecard can come in handy. Step 2: Assess expectations of leaders. You'll then assess them across leadership behaviors for new, mid and even senior managers. Granularity of differences supports focus and clarity. Often, a list of pre-existing behaviors/competencies are used to make the exercise easier. Validated psychometric tools such as the 16PF help to anchor it to scientific rigor. Organizational psychologists like me conduct surveys to gather insights. Then, focus groups are used to drill down to details information. After that, we'll create categories basedon the information and produce working behavior-based definitions. Step 3: Prioritize the list Now, the leadership team decides which behaviors are more important by way of ratings. Step 4: Build the 360 We then build a 360-degree feedback survey questions. These questions are reviewed for validity. Step 5: Allocate the survey A system specializing in the 360 (there are many) can be used. Feedback Recipient selects 6 to 12 people to rate them. In organizations, to avoid selection bias, leaders of the feedback recipient can review and veto the people doing the rating. Then, the participant does the survey too (self-rating) Step 6: Debrief of survey Usually, participants need guidance from a trained coach who understands feedback requirements. This is to provide grounding and objective input. Often, 360 surveys tend to be met with resistance unless the coach is skilled in facilitating the reflection conversation. Step 7: Action Planning The participant then produces a set of actions for improvement. This plan and the priority of focus should be made known to the feedback givers. Step 8: Pulse Surveys After a designated time (within 6 to 12 month period) a validated pulse survey is set up for the observers to rate improvement in specific behaviors. Step 9: Continued Leadership Coaching, Mentoring and Peer Support A combination of these can be used to enhance development. Step 10: Final Comparison Survey Toward the end of the year, a comparison survey is done to see how the key areas have improved or not. ---

WHAT IS A SKILL MATRIX? A Skill Matrix is a structured visual tool used to map and assess the skills and competencies of employees against the tasks or operations required in their roles. It helps organizations understand the current capabilities of their workforce and identify skill gaps that need training or development. Purpose in Manufacturing: In manufacturing, a skill matrix serves several important functions: It shows which employees are trained and capable of performing specific tasks, operating machines, or handling processes. It identifies gaps in skills where training is needed. It ensures the right person is assigned to the right job. It supports workforce flexibility, job rotation, and succession planning. It helps maintain production continuity during absenteeism or peak loads. How It Works: A skill matrix typically includes a list of employees and a list of required skills. Each employee is rated on how proficient they are in each skill, often using a scale from 0 to 3 (or 0 to 5). These scores represent the level of expertise, ranging from no knowledge to expert who can train others. Common Skill Levels: 0 – No knowledge: The employee is unaware of the task or has never performed it. 1 – Basic: The employee has some knowledge but needs supervision. 2 – Competent: The employee can perform the task independently. 3 – Expert: The employee is highly skilled and can train others. Benefits of Using a Skill Matrix: Improves visibility into team strengths and weaknesses. Supports training plans by clearly showing who needs development. Helps with compliance for audits and certifications (ISO, IATF, etc.). Aids in planning for job rotation, workload balancing, and cross-training. Enables better decision-making in assigning work or promotions. Applications in Manufacturing: Assigning machine operators based on their skill levels. Ensuring only qualified personnel handle critical or high-risk tasks. Supporting TPM (Total Productive Maintenance) and lean initiatives. Building multi-skilled teams to increase flexibility and reduce downtime. Maintaining audit readiness by documenting workforce capability. Best Practices: Review and update the matrix regularly (e.g., monthly or quarterly). Use input from supervisors, trainers, or certification results for accuracy. Visualize with color coding (e.g., red for 0, green for 3) for easy understanding. Integrate with performance reviews and training plans. Use it as a living document — not just for compliance, but as a driver for development.

Formative assessment is like tasting the soup as you cook. It is ongoing, diagnostic, and designed to guide learning. It gives students timely, actionable feedback during the process so they can adjust before the dish is served. Examples include quizzes, exit tickets, drafts, reflections, and teacher-student conferences. Formative assessments help identify misconceptions, adjust teaching strategies, and personalise support. They build student confidence and competence one taste at a time. Summative assessment is the finished dish served to the guests. It evaluates what students have achieved at the end of an instructional period. It measures mastery against learning outcomes and is used to judge the effectiveness of instruction. Examples include final exams, projects, performances, and standardised tests. Summative assessments do not shape the meal as it is being prepared, but they reflect every taste, every correction, and every adjustment that came before.

Catalina S. told me something that completely reframes how we should think about skills validation. After 10+ years leading workforce transformation at Vodafone, T-Mobile, and DataCamp, she dropped this truth bomb during our latest Business AI Playbook episode: "Companies don't just want employees to know things, they want employees who can do things." Most L&D teams are still stuck measuring completion rates and quiz scores. But Catalina's seeing something different work: Evidence-based skill validation that proves real-world capability. Here's what she's implementing right now: → AI-powered surgical feedback — Johns Hopkins is using AI to analyze actual surgical videos, providing objective feedback on technique and precision, not just theoretical knowledge → Peer-led GenAI Scouts — A global engineering org turned employees into instructional designers, achieving 90% engagement and 20-40% time savings on repetitive tasks in just 6 months → Real-world retail simulations — AI roleplay environments where new hires practice customer interactions, earning badges only after demonstrating 3 successful and 3 unsuccessful scenarios with lessons learned → Skills data as strategic inventory — Finally giving companies visibility into their actual internal capabilities while supporting employee growth aspirations Catalina's challenge to every L&D leader: "We need to shift from knowledge retention to evidence-based skill validation." The companies getting this right aren't just improving training metrics. They're fundamentally changing how their workforce approaches capability development. 🎥 Watch the full conversation below 🔄 Share this if you think proving skills matters more than passing tests What's the most creative approach you've seen to validate real-world skills? #BusinessAIPlaybook #LearningInnovation #SkillsValidation #AITransformation #FutureOfWork

Measuring the Intangible: Psychometrics at TISS HRM & LR Back in my engineering days at National Institute of Technology Jamshedpur, I often wondered, how do you quantify soft skills? We know how to measure efficiency in algorithms, the tensile strength of materials, or the electrical conductivity of a circuit. But how do you measure leadership? Teamwork? Emotional intelligence? The lack of clear metrics often makes people dismiss these subjects as vague, generic, or lacking rigor. But the reality is, the mastery of soft skills lies not just in understanding them but in learning how to measure them, what methods to use, and most importantly, what to infer from the results. Psychometrics (Stats for Psychology), a course at TISS HRM & LR, which is the polar opposite of what I studied, yet follows the same structured, analytical approach. Why Measure Psychological Variables? Every possible test be it aptitude test, personality assessments, cognitive ability exams, or emotional intelligence evaluations is built on psychometric principles. Hence, selecting good variable is important. At its core, psychometrics seeks to answer: "How are individuals different?" Organizations today consider individuals as a collection of attributes, each with varying levels. Different jobs require different attributes, and individuals likely retain their rank order relative to one another, even after training. Take creativity as an example. If a company wants to hire "creative" individuals, what are they really measuring? Is it artistic ability?, divergent thinking? The way creativity (read variable) is defined, measured, and normed impacts the hiring process. The Science Behind Intelligence and Ability One of the key frameworks we explored is Spearman’s G-Factor Model. As per meta analysis individuals with high G (General Intelligence) are likely to perform better. So, should they be prioritized in hiring? Not necessarily. Research shows that those with a high G also have a high need for cognition (constant growth, intellectual stimulation). But hiring solely based on intelligence might overlook other skills like adaptability and collaboration. There is nothing inherent called Intelligence. It accounts for Personality, Attitude & Stimulus Discrimination (constructs based on observation and prone to bias). Can Intelligence Be Changed? The Flynn Effect suggests that intelligence is not static, it increases by 1 SD over generations due to better education, environment, and societal changes. Psychomotor abilities, on the other hand, are always in reference to something external, how fast we react, how accurately we perform a task under pressure, etc. We used JASP (open-source software for statistical analysis) to measure such constructs. Understanding psychometrics fundamentally changes the way we view hiring, leadership, and workplace performance. It’s not just about asking if someone is a "good fit"; it's about systematically measuring what "fit" even means.

People and Talent Leaders: Is your hiring process really adjusted to the level of talent you’re targeting? Because the steps look very different depending on whether you’re hiring for more accessible roles or competing for top-tier talent. For roles where the talent bar is more accessible (say high-volume, less specialised), the process can be summarised quite simply as “interviews”: - Screening Call - Hiring Manager Interview - Final Interview - Offer → Candidate Accepted But when you’re competing for scarce or high-bar talent, the process cannot be summarised as just “interviews.” And it shouldn’t be. For senior, niche, or high-impact roles, the process should look more like: - Problem Alignment (why the role exists / clarity of success metrics) - Value Alignment (candidate sees the mission, EVP, team fit) - Skill Validation (structured assessment of technical/functional skills) - Stakeholder Alignment (multi-disciplinary panel, cross-team buy-in) - Decision & Offer Alignment (timing, competing offers, motivators) These stages track specific agreements and validations that have to be made in the hiring journey. They usually have clear entry/exit criteria and defined actions. For Skill Validation, it might look like this: Entry Criteria - Candidate has already shown alignment with role scope and career trajectory. - Candidate is motivated to invest time in assessment. Actions - Run structured technical or practical assessment. - Ensure panel has consistent scoring rubric. - Calibrate with benchmarks (internal & external). Exit Criteria - Panel agrees candidate meets or exceeds the hiring bar. - Feedback documented in structured format with rationale. 👉 The catch: depth always trades off against speed. - Go too “deep” and you lose top talent to faster-moving competitors. - Go too “fast” and you risk poor signal or mis-hires. The best scale-ups calibrate their process so each stage is both rigorous and lightweight. For example: - Replacing generic take-home tasks with short, live working sessions. - Combining multiple stakeholder conversations into a single panel. - Pre-aligning on what’s “must-have vs nice-to-have” so decisions don’t drag. ⚖️ The art is finding the balance: rigorous enough to protect quality, but lean enough to win talent before someone else does.

I’ll never forget my first year in recruiting. I had no background in production or distribution, yet I was responsible for hiring talent in those fields. My “training” consisted of watching an old VHS tape in the back of the staffing office—great for interview tips, but not exactly a deep dive into the actual skills needed for the job. So, I learned the hard way. I talked to candidates, toured workplaces, and observed the roles firsthand. But the real aha moment came when I sat down with a production manager to review candidates. Halfway through, he stopped me and said: "Jessica, you’ll never find someone with this exact experience in San Jose. It doesn’t exist. What I need is someone who has a craft hobby or likes to wrench on their car—someone who enjoys working with their hands and problem-solving. I can teach them the rest." That conversation changed the way I think about hiring—forever. It made me realize that hiring based purely on past job titles or industry experience is a huge limitation. Why Skills-Based Hiring Matters More Than Ever Traditional hiring—relying on degrees, years of experience, and job titles—creates unnecessary bottlenecks. It’s a model that is quickly becoming outdated because: 🔹 Skills evolve faster than job descriptions 🔹 Non-traditional paths produce top-tier talent 🔹 Rigid credential requirements exclude high-potential candidates 🔹 Overlooking skills-based talent reduces diversity and innovation According to a 2023 McKinsey report, 87% of organizations either have skill gaps today or expect them in the near future. If companies don’t shift toward a skills-first mindset, they risk falling behind. How to Implement Skills-Based Hiring ✅ Look Beyond the Résumé – Like that production manager, recognize that a candidate’s hobbies, past projects, and work history might indicate strong, transferable skills. Ask: What projects have they worked on outside of work? How have they adapted to new tools or technology? ✅ Use Competency Testing and Behavioral Evaluations Competency testing and behavioral interviews can help assess a candidate’s skills more effectively than a résumé alone. Practical assessments: Real-world tests where candidates complete a task relevant to the role. Behavioral interview questions: Instead of asking about general work history, ask: "Give me an example of a time you had to solve a complex problem with limited resources." "How did you stay engaged and ensure quality results?" ✅ Evaluate Longevity in Projects and Roles Rather than focusing solely on job titles, assess: Have they shown commitment and follow-through in past work or personal projects? Do they take on complex challenges and see them through? Have they demonstrated adaptability in different work environments? How has skills-based hiring impacted your team or organization? Have you uncovered great talent by looking beyond traditional requirements? Share your experiences in the comments! 👇

When you interview, you’re not just checking skills — you’re trying to spot a high performer. Use this: 9 micro-screens for the 9 signs (save for your next interview). Takes initiative → “Tell me about something you shipped that nobody asked for. Why? Outcome?” Delivers consistently → “Show me a timeline of commits/launches. What was the weekly cadence?” Adapts quickly → “When the plan changed mid-project, what did you change in 48 hours?” Owns mistakes → “Describe a miss you caused. How did you detect it and prevent the repeat?” Raises the standard → “What bar did you lift for the team? How did others follow it?” Seeks feedback → “Share a doc/design with comments. What did you keep vs. change and why?” Stays focused → “What did you say no to last quarter? What did that protect?” Supports the team → “A time you made someone else successful at a cost to you.” Leads without title → “Project you led with zero authority. How did you align skeptics?” How to score: look for specifics, artifacts, numbers, and clear “I/We” ownership shifts. Bonus: finish with a 10-10-10 sprint (10 min portfolio → 10 min decision deep-dive → 10 min task sim). Leaders hire potential; great teams hire evidence.

How do you know if AI is good at X versus Y? Other fields figured this out by moving from aggregate scores to diagnostic evaluation: Aviation doesn't ask "can you fly?" They assess specific maneuvers: stall recovery, crosswind landings, instrument approaches. Pilots get detailed skill profiles showing exactly where they excel or struggle. Medical education evolved beyond multiple choice The OSCE uses rubrics that separately score communication, physical exam technique, clinical reasoning, and treatment planning. Financial AI stopped trusting aggregate scores Researchers built frameworks mapping performance across 70+ specific financial concepts. Result? Two models both scoring 85% had completely different capability profiles — one excelled at tax calculations, the other at revenue analysis. Scale AI recently showed that when they evaluated models using detailed rubrics instead of simple accuracy, performance gaps jumped 31% on HealthBench. The rubrics revealed which specific skills each model actually possessed. The conceptual shift is from "did you get the right answer?" to "do you have the specific capabilities required for this task?" I have a new Substack - the first in a series - exploring diagnostic evaluation approaches from aviation, education, finance, and how healthcare AI can apply these lessons https://lnkd.in/gaDQsFF6

How Do We Know That Our Dental Students Are Competent and Ready to Perform Certain Procedures on Patients? Having been in dental education for over three decades, I frequently ask myself: “Is this dental student ready to prepare a crown for this patient?” or “Is this resident ready to perform a full-mouth rehabilitation?” These are critical questions that define the safety and quality of care our students provide. Systematically, we rely on competencies and assessments to evaluate whether a learner—whether a predoctoral student or a resident—is ready to perform specific patient care procedures. However, dentistry is not a one-size-fits-all discipline, and different procedures require different benchmarks for competency. For example: • A student learning crown preparation might require six to eight or more practice attempts on a typodont before demonstrating competency. • Intraoral scanning might require three to five attempts on live patients to become efficient. • Tooth polishing, a simpler procedure, may only require one or two practice sessions before competency is achieved. Thus, applying a uniform competency threshold across all procedures can be misleading. If a program uses a rigid numerical requirement (e.g., “X number of procedures = competency”), it risks shifting the focus away from true skill development and progression. Instead of promoting growth and refinement of skills, it creates a checkbox mentality where the learner and faculty may mistakenly assume that completing a fixed number of cases equates to readiness. The Case for Entrustable Professional Activities (EPA) A more effective way to assess readiness is through Entrustable Professional Activities (EPA)—a framework that evaluates a learner’s ability to independently perform a task based on real-world observation rather than just a predefined number of attempts. EPAs consider: • The context of the procedure • The student’s decision-making process • The complexity of the case • The supervising faculty’s confidence in the learner’s ability to perform the task safely and effectively This approach shifts the focus from just completing a requirement to demonstrating competency in a dynamic, patient-centered way. How Should We Implement This? 1. Define clear EPA guidelines for key procedures that align with patient safety and clinical complexity. 2. Encourage progressive assessment, allowing students to develop skills at their own pace while ensuring readiness at every stage. 3. Integrate faculty calibration so that evaluators consistently assess readiness and entrustability. 4. Use technology and data analytics to track skill progression beyond just a number of completed procedures. Ultimately, competency in dental education should not be about rigid numerical thresholds but about ensuring that students are truly entrustable to perform patient care with confidence, skill, and safety. #DentalEducation #CompetencyBasedEducation #EntrustableProfessionalActivities