Now, over the last couple of weeks, and more recently today via Jeremiah Grossman on Twitter) folks have been talking to me about metrics, and more importantly measuring "web based exposure to risk" ... right. We can talk about this all we want, but talking it to death won't make it any more real.
At the heart of the problem is the fact that we're essentially being asked the same question your parents asked you when you asked, begged, and pleaded for $toy: "what will happen if you don't get $toy ... right now? Can you wait until Christmas?"
There are two camps that have formed over the last several years, when it comes to answering that question ...
- The "FUDs" - You know who you are. "We'll be hacked if we don't buy product X" ... sound familiar? Seeing Fear|Uncertainty|Doubt only works for so long before they catch on!
- The "Franks" - With the mentality of an accountant and the numbers to back it up, the "Franks" typically don't give any emotion into their work, it's all risk and numbers
So we've got a problem. There are 3 things that are going to save Info Security ... web app security more specifically - metrics, magic, and failure. Don't laugh.
- Metrics: You can measure damn near anything these days, and if you look hard enough and network with enough people eventually you'll get some pretty decent numbers on things like how many times the average credit card authorization site for a mid-market company gets attacked (actually attacked, not "scanned") in a given quarter. You can even scam your way into getting people to giving you metrics on how many times companies in your industry, maybe even your competitors, (anonymously of course) have been attacked and maybe even hacked. You may even get someone like Gartner, Forester or IDC to publish a report on the metrics of being hacked and how much it costs per record lost, or hour of downtime. This is all great information but it's entirely worthless if you don't have the right context. By context I'm referring to the specific context that makes sense to the metrics you're collecting ... clear as mud right? Here's an example -if you're a very large online retailer it only makes sense to collect metrics of like context ... from other businesses that are of similar size, market, and exposure. In this instance collecting metrics from an industrial business won't help your cause any ... or will it?
- Magic: When metrics alone just don't make the case, make the case with a little bit of home-brew magic. Security folks are known for our passion and ability to manufacture a reality that fits our ends. I'm not saying to make things up, that's what the FUDs often do, I'm simply telling you to get creative. Make the numbers pop without making them up ... it's harder than it sounds. Being a magician also involves getting numbers from places that you can't talk about ... friends in your network, your own systems or other sources that can't clearly be cited for anonymity. Adding a pinch of magic to your pitch will make the metrics get someone's attention and scream "You can't ignore me!"
- Failure: It's a fact, nothing helps along a security agenda like a catastrophic hack. Once you've been breached your executives will write checks for amounts you've never seen before... in record time. The problem is once you've had a major breach your job may be on the line and then things get really crazy (see my post on the dangers of a disaster-driven security program) while everyone is running around with their hair on fire. The trick is to not be the victim of a major breach but something big enough to spark attention and make people inside your company paranoid, that's when you get to work your magic. (see the Magic above). Some great person once said "Without failure, we cannot know what it is to succeed" ... and that's very true even today in InfoSec.
Speaking of risks ... how do you measure that, exactly? Do you measure what you find ($found_vulns) against the function of the application ($function) and the value ($value) combined with exposure ($exposure) ... ? And riddle me this Batman... what about those vulns which aren't found? How do you measure risk with that many unknowns?
Sadly, if I had the answers to those questions I think I would be writing this post off a yacht in the South Pacific, but alas, I'm sitting at my desk in Chicago... (damn it's cold for September).
For starters, and maybe as a spring-board for a good risk formula I have used the following elements successfully ...
- Dollar-value of the asset being assessed/analyzed -should be the easiest metric to gather
- Relative exposure to defined threats (notice I say defined threats) - exposure is relative to the industry, type of business, type of technology, type of asset, etc ... this can have multiple components within it (to be discussed at a different time)
- Relative complexity of asset - remember, complexity is the enemy of security
Good luck! More coming soon...