After more than 25 years of mitigating risks, ensuring compliance, and building robust security programs for Fortune 500 companies, I’ve learned that looking busy isn’t the same as being secure.
It’s an easy trap for busy cybersecurity leaders to fall into. We rely on metrics that tell a story of the tremendous efforts we’re expending – how many vulnerabilities we patched, how fast we responded – but often vulnerability management metrics get associated with operational metrics because traditional approaches to measuring and implementing vulnerability management does not actually reduce risk. So, we resort to various ways of reporting on how many patches were applied under the traditional 30/60/90-day patching method.
I call these vanity metrics: numbers that look impressive in reports but lack real-world impact. They offer reassurance, but not insights. Meanwhile, threats continue to grow more sophisticated, and attackers exploit the blind spots we’re not measuring. I’ve seen firsthand how this disconnect between measurement and meaning can leave organizations exposed.
In this article, I’ll explain why vanity metrics are not enough to protect today’s complex environments and why it’s time to stop measuring activity and start measuring effectiveness.
Vanity metrics are numbers that look good in a report but offer little strategic value. They’re easy to track, simple to present, and are often used to demonstrate activity – but they don’t usually reflect actual risk reduction. They typically fall into three main types:
Vanity metrics aren’t inherently wrong – but they’re dangerously incomplete. They track motion, not meaning. And if they’re not tied to threat relevance or business-critical assets, they can quietly undermine your entire security strategy.
When vanity metrics dominate security reporting, they may do more harm than good. I’ve seen organizations burn through time and budget chasing numbers that looked great in executive briefings – while critical exposures were left untouched.
What goes wrong when you rely on vanity metrics?
I’ve seen breaches occur in environments full of glowing KPIs. The reason? Those KPIs weren’t tied to reality. A metric that doesn’t reflect actual business risk isn’t just meaningless – it’s dangerous.
If vanity metrics tell us what’s been done, meaningful metrics tell us what matters. They shift the focus from activity to impact – giving security teams and business leaders a shared understanding of actual risk.
A meaningful metric starts with a clear formula: risk = likelihood × impact. It doesn’t just ask “What vulnerabilities exist?” – it asks “Which of these can be exploited to reach our most critical assets, and what would the consequences be?” To make the shift to meaningful metrics, consider anchoring your reporting around five key metrics:
Taken together and continuously updated, meaningful metrics give you more than a snapshot – they provide a living, contextual view of your threat exposure. They elevate security reporting from task tracking to strategic insight. And most importantly, they give both security teams and business leaders a common language for making risk-informed decisions.
Vanity metrics offer comfort. They fill dashboards, impress in boardrooms, and suggest progress. But in the real world – where threat actors don’t care how many patches you applied last month – they offer little protection.
Real security demands a shift from tracking what’s easy to measure to focusing on what actually matters. That means embracing metrics grounded in business risk. And this is where frameworks like Continuous Threat Exposure Management (CTEM) come into play. CTEM gives organizations the structure to move from static vulnerability lists to dynamic, prioritized action. And the results are compelling – Gartner projects that by 2026, organizations implementing CTEM could reduce breaches by two-thirds.
The metrics you choose shape the conversations you have – and the ones you miss. Vanity metrics keep everyone comfortable. Meaningful metrics force harder questions, but they get you closer to the truth. Because you can’t reduce risk if you’re not measuring it properly.
Note: This article is expertly written by Jason Fruge, CISO in Residence at XM Cyber.
