Saturday, February 1, 2014

Guest Post: Follow up to "Where Risk Calculations Fall Apart [Again]"

In a previous post "Where Risk Calculations Fall Apart [Again]" I made the argument that a complex formula variable in a risk calculation like "likelihood-of-exploit" is essentially (at best) undesirable, and at worst detrimental if not nonsensical. I posted the blog link to Twitter and as expected debate struck up. I think I'm going to write another follow-up on this because there still seems to be some confusion as to what I am arguing ... I appreciate all the replies and discussion so far. I even received an email from a colleague who agreed with my viewpoint and had put together a very comprehensive reply but couldn't fit it into the comments section so instead here it is in it's entirety ... I encourage you to read Heath's lengthy, but extremely well-thought-out reply.

I’d like to speak to “likelihood of an event,” conversation because I think Raf is correct in that this term can be a red hearing.  True, we all have to engage in predicting the future just to get up in the morning, but the way we flippantly talk about probability in this business is a concern to me. In the end, it is a never-ending task that will rarely provide results in a sustainable way because, let’s face it: no one knows the future. No one especially knows the future about the hard problems that we are all asked to answer ( easy, predictable questions are another matter). If there are relevant studies that say otherwise I’m all ears, but as far as I know, from stock picking to the weather, forecasting is still as much art as it is science.

For our average corporate audience, when we state the probability of something happening to the firm based on the frequency of it happening in the past (or multiple frequencies all aggregated and wrapped up nicely), we are essentially making a declarative analytic claim about what the future will be.  The nuance of to what degree our conclusion is an abstract concept is lost on the average audience. Therefore, we have to be much more conservative in the language we use to make claims about the likelihood of an event. We have to add nuance and probabilistic language, because the audience may actually think we are talking about the actual future when we are talking about the output of a model with a degree certainty. The declarative approach may seem immediately gratifying, but it eventually will lead to questions about underlying assumptions, which if answered truthfully, will reveal just how precarious our conclusions are. This conversation won’t necessarily make our conclusions false. It will just make them look less absolute, because there is a long way to fall from claiming that the probability of x is y over the next year. If you work at a national laboratory or your boss is asking you for this level of information that is another story. I bet that is not the case for the majority of firms. Raf’s suggestion of focusing on the basic controls and resiliency seems to make sense for most firms, leaving the modeling to those who are competent enough to understand their models and speak about them responsibly to generalists.
This conversation reminds of the debate in finance as to whether or not markets approach perfect efficiency, and therefore appear random (random walk), or whether they are inefficient and repeat themselves according to behavioral patterns (  I haven’t made all the linkages myself yet, or looked at it closely since undergrad, but I think there is helpful fodder for us in security. The idea is that if all previous information is already integrated into financial valuation strategies, and strategies are instantly executed, and reflected in a stock price, then what comes next is based purely on whatever is new information. That means that the past will not inform the future because it will be a function of only brand new, unexpected events. All the old info is already absorbed, already expected.  The practice of investing from this premise was to simply invest in long-term averages since short-term outcomes were mostly due to luck and no one knows what happens next.

The parallel here in my mind is the idea that attackers are efficient and swift, they've absorbed all the known info about our defenses, and their next move will simply be a matter of what is entirely new (vulnerabilities that present themselves or political causes to rally against, etc). This action will appear random because it has never been seen before.

The other side of the debate says that markets are not efficient and that strategies and behavior repeat themselves. Markets may move information around quickly, but it is not accurately reflected in the stock price because man’s judgment is (consistently) flawed. The idea is that we can potentially predict market behavior if we plug in enough variables because people will make the same mistakes over again and conditions will repeat themselves. The assumption is that previous facts do inform the future. The parallel in security is the belief that previous attack patterns will repeat themselves, regardless of the fact that former vulnerabilities should be mitigated in our defenses. The truth is not every attacker is cutting edge, and not every firm is perfectly patching. Therefore, previously seen attacks are more likely to be repeated. If they do repeat themselves, then probability of incidents are relevant.
I’ve heard both approaches be blamed for financial crises. From the late 90s, those who believed markets were efficient were accused of overestimating the market’s ability to correct itself without more proactive analysis (What if we all just invested in the SP500 instead of individually researching companies? Their valuations would start to be less accurate. What if we all just took it for granted that bad attackers would be bad attackers and focused on resilience? We would start to lose understanding of attackers.) . At the same time, quant-based model builders have been blamed for hubristically thinking they could predict the future. The wizards that gave us the more esoteric mortgage-backed derivative products fall into this camp. The parallel in my mind is a security team that thinks it can predict outcomes by studying the past and trying to create a universal model for risk.  It occurs to me, that this would be a critique of modeling in general, which I would not want to be accused of. I love models! It’s not the modeling I’m leery of about, it’s how we talk about the modeling.

The question is in what do we put our faith? How much faith do we put in either one of these approaches? Also, how much money do we want to spend to manage security?

Wait! Please not another lecture on ALE just yet. Hear me out. It would be very convenient to have a scorecard. How do we know which approach in finance works the best? We can easily compare analysts’ price targets to the price that occurred after the fact. We can also easily compare the performance of different strategies (Simply indexing over the long term or active investing?).

Where are we doing that as security practitioners? One area is the comparison of CVSS scores and the very few vulnerabilities that actually result in a majority of breaches. More could probably be done along those lines in terms of the reporting transparently of breaches and then having academia establishing a line of research comparing the breaches to previous strategy. How about actually studying firms that do not engage heavily in probability-based analysis and simply focus on the big 20 security controls for their high value data: do they experience more events than those that take a more active approach? What we are missing is a way to publish strategies and predictions and then rank the performance of those strategies and predictions.
I am skeptical of any approach that tries to predict the future. My world view says that history makes fools of us all. At the same time, I would not have us living in caves nor would I have the good folks at SIRA stop trying to refine risk science. I also do not hesitate from leaning into the future with my corporate audience as far as my conscious will allow. But I equate the advance modeling with the type of practice advance finance quant shops would use to value financial instruments. Their work is digested in conjunction with other approaches. Their successes last for various lengths of time until attackers attempt to outsmart the model. Not every firm will be able to afford this kind of analysis. But we need folks pushing the envelope anyway.

For the vast majority of firms, focusing on the basics of defense, building resilience and letting the professionals take care of the rest through outsourced services is cheaper than trying to establish the true risk by defining probability, and then attempting to speak about it appropriately and in a relevant way, all at the firm level. The majority of us should probably be index investors. There are important exceptions with firms operating in the scientific industries, high security environments or with enlightened managers, but these are the minority. I’d like it if we started delineating which audience we are targeting with our remedies instead of making absolute statements about what is appropriate in all cases (unless it’s completely justified ;))

Heath Nieddu is a Senior Information Security Analyst at Providence Health & Services, where he tries to keep his CISO entertained with policy-relevant analysis and metrics. His comments are his own and do not represent the views of his employer.

No comments: