Archistry

Survivability by Design™ since 2006

  • Home
  • About
    • Who Is Andrew?
    • C2T System™
    • The Agile Security System™
  • Contact
You are here: Home / Archistry Daily / “Good math” vs. “bad math” in risk assessments

May 21, 2020

“Good math” vs. “bad math” in risk assessments

Photo by Antoine Dautry on Unsplash

A long time ago, I heard someone say: “Lottery tickets are a tax for people who are bad at math.”

Which is pretty accurate. Have I ever bought one? Well, yeah—but as a conscious choice in a game of “Wow, wouldn’t it be really funny if I won $18 gazillion,” rather than, “I can’t pay my rent, so I know that this time, I’m going to win. I just know it!”

Come to think of it, there’s a lot more in common between buying lottery tickets and risk assessments than you might think. But the biggest one is a bit of an understanding of statistics and probability.

I have to admit, for a guy with a Computer Science degree and being only 1 course shy of getting a Math minor, statistics is one of the things I’ve had to work pretty hard to overcome an aversion the size of the Moon. It wasn’t always this way. In grade school all the way up to High School, I thought it was really cool. I had some great teachers, but then, I got to university, and ended up in a Stat 101 class with Dr. Mumbles.

Every day, he would walk up to the board, brace the book between it and his belly, and mindlessly copy verbatim from the book as he mumbled what he was writing into his chest. It was terrible! And it made me want to get as far away from him – and, unfortunately, statistics – as I wanna be right now from the front-lines of COVID-19.

In fact, I ended up failing the course—at least the first time. So, I’m probably one of the last people you should listen to about how to do risk assessments properly, because it ultimately all boils down to statistics.

But what I’ve noticed is that I must not be the only risk and security professional with an historic aversion to statistics, because there’s an awful lot more qualitative risk assessments done every day than there are ones based on hard probabilities. Of course, there’s a reason for this. And it’s a reason that technology people – and even a few psychology people – will disagree with:

Decisions are fundamentally based on emotion, not logic. And, yes, I know this may imply that everything is just confirmation bias, but I think that’s a bit simplistic.

There’s actually been a lot of clinical research with people with brain injuries to the areas responsible for emotion and their ability to make rational decisions. What this actually means is that it’s virtually impossible to make a decision “strictly based on the numbers” because we can’t avoid how each of those potential outcomes represented by “the numbers” is going to make us feel.

You wanna make “rational” decisions based on only numbers, then let the computers do it. I’m sure the YouTube suggestion algorithm can manage running the world just fine, not to mention your security program investments.

Even when I used to teach the official SABSA Foundation course, I used to say that because humans make the decisions, everything ultimately boils down to a qualitative scale—even when presented with quantitative evidence. Until the last couple of years, I just didn’t have access to the science to prove my theory.

But, since this goes against our Mr. Spock ideal of the logical, rational decision, we want to reject the validity of qualitative data—especially in risk assessments. Of course, because it’s so subjective, and it’s also very difficult to get reliable and repeatable results, when there’s millions of currency units on the table with each decision, people want to either:

  1. be able to justifiably cover their arse, or
  2. be able to effectively blame someone else if the decision goes pear-shaped.

Two sides of the same coin.

So, we’re scared of labels – with good reason, which I cover briefly as part of Module 3 the BESA program (which is still open for registrations, BTW) – and because we don’t want to be deemed an “emotional decision-maker”…

…we invent “new math” that might prop up our subjective assessments with more upstanding numerical values.

So instead of “Very Likely”, we call that a 5 on a 5 point scale. And then we do the same for impact, giving the “Moderate” label (whatever that really means) a value of 3/5.

We’ve just taken a giant leap down the slippery slope that will land us squarely in the ice cave of the Wampa on Hoth without our lightsaber (let’s face it, we’re not Luke Skywalker).

Because the very next thing we do in our beady little brains is go:

“Hey, we have two numbers, but *two* numbers are really hard to track, because then you need to explain each one and that opens the door to many questions I can’t really answer…so, I know! I’ll combine them together, and then we get only one!”

And hence, the single value “risk rating” is born: (3 + 5)/2 => 4

So, there you have it, the risk exposure went from something that we needed to talk about because it was qualitatively represented to a “hard number” that’s 4. And since 4’s awfully close to the top of our scale, then we need to mitigate it immediately!

Yeah…sure.

And the risk of monkeys flying outta my butt is 4.398.

So, the “bad math” of risk assessments is much, much more dangerous than the intelligence tax of buying lottery tickets—because if you buy a lottery ticket, you’re mostly exposing yourself and your family to the consequences of getting it wrong.

If you present a whole host of recommendations to the executive management team based on “bad math” risk assessments, the consequences can be much more far-reaching indeed.

There are solutions, however—even when you’re faced with a “risk rating” of “cucumber”. But you need to have a reliable way to untangle the mess and be able to highlight the real factors involved in the real decisions to be made.

The decision to be made is not, what should we do with this risk rated at 4.398. The decision to be made is buried far, far deeper and is about a business choice rather than making numbers smaller.

If you want to know how to do that, then you might find the forthcoming May issue of Archistry’s Security Sanity™ newsletter useful. If you’re happy with single-value risk ratings driving the security decisions and investment of your organization, then you can probably give it a miss. What I’m going to be talking about will most certainly fall into the “too much work” bucket for you.

Obviously, existing subscribers will get it automatically, but if you’re not one, then the only way you’re going to get it is to get on the bus before the end of the month using this link:

https://securitysanity.com

Stay safe,

ast
—
Andrew S. Townley
Archistry Chief Executive

Article by Andrew Townley / Archistry Daily / Accuracy, Agile Security, Risk Assessments, SABSA

  • Email
  • LinkedIn
  • Twitter
  • YouTube

EMAIL NEWSLETTER

Want to get DAILY email tips on how to build a more effective security program so you can prove your security investments deliver value to the business?

You can always unsubscribe at any time, and we won't sell your data to third parties.

About Us

Archistry works with you to ensure what you want to achieve actually gets done, linking strategy, risk, governance and compliance to enable sustained exceptional performance Read More…

Testimonials

Andrew is a highly skilled and experienced information systems architect and consultant, which in my view is a rare thing. He is innovative in his thinking and merits the title of 'thought leader' in his specialist domains of knowledge—in particular the management of risk. Andrew has embraced SABSA as a framework and, in doing so, has been a significant contributor to extending the SABSA body of knowledge."

— John Sherwood, Chief SABSA Architect

"Fabulous person to work with. Very engaging and insightful. Extremely good technical knowledge with ability to relate concepts together and overcome differing opinions. Makes things work."

— Kevin Howe-Patterson, Chief Architect, Nortel - Wireless Data Services

"Andrew was able to bring clarity and great depth of knowledge to the table. His breadth of thinking and understanding of the business and technical issues along with a clear and effective communication style were of great benefit in moving the process forward towards a successful conclusion."

— Doug Reynolds, Product Manager, MobileAware

"Andrew is a fabulous consultant and presenter that you simply enjoy listening to, as he manages to develop highly sophisticated subjects in very understandable way. His experience is actually surprising and his thoughts leave you without considerable arguments for any doubts in the subjects he covers."

— Biljana Cerin, Director, Information Security and Compliance

Recent Posts

  • If you want better security, you’d better have a better security architecture
  • The ultimate security song to keep you focused on what you’re doing
  • Security heroes
  • There’s always a people problem
  • Putting your data flow diagrams out to pasture…for good

Looking for something else?

  • Home
  • About
  • Contact

  • Terms of Service
  • Privacy Policy
  • Cookie Policy
  • Copyright © 2006-2025 Archistry Incorporated or its affiliates

"Archistry", the stained glass window logo, "Pragmantix" and the Pragmantix™ logo, "Archistry Execution Framework (AEF)", "Archistry Execution Framework, Cybersecurity Edition (ACS)", "The Agile Security System", "The Agile Business System", "Baseline Perspectives", "Architecture Wall", "Archistry Execution Engine", "Renegade Security", "Renegade Security System", "Security Value Delivery System (SVDS)" "Collapse-to-Traction", "Collapse-to-Traction System", "Adaptive Trust & Governance Model (ATGM)", and "Adaptive Trust & Governance Model for Organizations (ATGM4O)" are trademarks of Archistry Incorporated or its affiliates.