This current discussion on the DailyDave regarding the APT1 report Mandiant put out (one year on) list is seriously boiling my bunny(tm).
I've written about the challenges and necessity for attribution before, on my corporate blog back on Dec 24, 2011 - so I won't rehash old points. The discussion of attribution quickly goes into the deep end of the pool as techniques and technologies vary from one researcher and technology to another - this whole topic seems to come up over and over. Here are some points for discussion.
Cyber-attribution isn't unlike finding a fingerprint in the victim's blood and matching it against the killer... you have to be absolutely sure before making serious accusations and there are (at least in the physical world) standards that must be met. The same goes for comparing DNA, there are a number of markers which must match before the identification can be made positive. So when we look at cyber attribution - identifying the who behind a breach, a campaign (a collection of related actions), or something else we're interested in requires careful analysis. This analysis must be done out in the open - you can't tell me that you have data that you can't show me. Publishing a subset of data, holding some bits of it 'secret' and then telling me that the conclusion I reach based on empirical analysis of the data you published is wrong - is ridiculous. I understand that the intelligence community (public/private) has some things that must be kept secret - but if you're going to do attribution in a public document you simply can't hold information back. Either publish and be transparent, or consider the entire matter 'secret'. You can't have both. The discussion on DailyDave seems to be originating from this point, and I believe the researchers arguing the point are absolutely in their right to either demand data, or an admission of "gut feeling".
Here's another thing I'm having a problem with - while attempts at a published standard for cyber attribution exist today (CyBOX (a structured language for cyber observables), OpenIOC) they lag from non-collaboration between corporations, researchers and others with a stake in the game. Things like IP address, OS language settings, strings inside exploit code, etc are all things that can be changed by the attacker, and I suspect a smart attacker will do a fair bit of misdirection and randomization to throw off those who would track them. I think this reply is absolutely brilliant - http://seclists.org/dailydave/2014/q1/74 and goes in great depth to try and pull back the veil of black magic and voodoo. Assigning attribution isn't a trivial exercise, because if it was any old hack could do it. Fact is, there is a mountain of research and profiling that goes on to tie an action to a person or persons, including psychology - all of this (as Th3 J35t3r reminded me) requires HUMINT (human-based intelligence gathering). So realistically, if you're serious about attribution you're going to want some sort of human-based interaction (setting a trap to coerce or force interaction) to gather good data. This is starting to sound like we need a human profiler. Maybe we do... maybe that's already happening. So you want to talk IOCs (Indicators of Compromise)? You'll undoubtedly run into people and systems speaking CAPEC (for software security), MAEC (for malware analysis), Stix (for generic cyber threats), Taxii (for automated indicator exchange) - and there are many more proprietary methodologies too. It's enough to make you crazy!
Finally - what I think many of us are missing is this - maybe some/most of the researchers understand the subtleties of attribution on a global scale - but once the marketing departments get a hold of these reports and information - many times sanity goes out the window. I keep thinking of that scene in Zero Dark Thirty:
Maya: “100% he’s there. OK, 95% because I know certainty freaks you guys out, but it’s 100% he’s there.”This isn't a movie and it most certainly isn't a game. Your company, and your self-promotion isn't the goal here. People's lives and global politics are at stake many times, for fear of being overly dramatic. Throwing around thinly supported accusations of state-sponsored attack has the potential to spark wars. If you think this is ridiculous - read your history more closely, wars have been started for much, much less.
It's not just one vendor's credibility that's at stake either, this is a case of collectively succeed and collectively fail. Many of the cyber threat reports which identify specific state-sponsored, alright "APT" (now drink), actors are used as scare tactics to get CIOs and CISOs to "do something". Sadly that something is often to buy yet another 1U rack-mountable "APT Defender 2.0 Super Deluxe edition with the Anti-China add-on" box (we call this a solution if you've been to RSA Conference). In many cases, executives are scared into buying many of these, and spending their company's resources including people and time on implementing something that will protect them. Yet we know protections fail. All of them. And then we're wondering why corporate executives have a less-than-healthy distrust for the security vendors that court them.
I'll leave it here, I can sense myself starting to rant...
Call to action
Now I throw out the proverbial "Something must be done" request. Except this time, I think the community of threat researchers has to handle this, internally. Self-police.
First, Collectively provide expert analysis on some of these obviously farcical publications which are thinly veiled marketing documents for their companies or services. Take the report. Take the data. Collectively analyze, peer review and provide rebuttal where necessary.
Second, come up with some industry-accepted, peer-researched-backed standards for reasonable attribution of cyber. What constitutes x% certainty - since I'm damn sure we'll never be 100% of the stuff that's on the wires. Create a framework, publish, leave for peer review, formalize. Stick to it.
Skip the drama, the name-calling, and the "I know more senators than you" game. Let facts speak for themselves, with solid, peer-reviewed analysis.
Otherwise we only have ourselves to blame for this mess.
[Edit 3/25/14 - 5:35pm]
I'm adding in two extremely relevant paragraphs from Kyle Maxwell's reply (on-list) which I think nicely tie a bow on this whole discussion. Kyle seems to put it more eloquently in a few paragraphs what I've been trying to ramble on in a whole blog post:
"The issue of trust comes up here as well, because things changed sometime after the APT1 report release. Many of us in this community have even more trouble than we might have had before in accepting assertions based solely on "NSA" and "DoD" and "US government" labels, to the extend we ever *did* accept them.
The idea of reproducibility is a key part of inquiry, whether in science or intel analysis or anything else where critical thinking matters. We're not shamans. In the 21st century, we should expect to have our conclusions and methodology challenged (as I do every day). In any case, if one's response to criticism is to withdraw from the discussion, onlookers will not draw good conclusions. The audience is listening, as I believe I've heard once or twice."
 Thanks to J. Oquendo for the peer review and added data which I was missing initially
Links (all in one place) as resources:
- CAPEC - http://capec.mitre.org/
- MAEC - http://maec.mitre.org/
- STIX - http://stix.mitre.org/
- TAXII - http://taxii.mitre.org/
- CyBOX - http://cybox.mitre.org/
- OpenIOC - http://www.openioc.org/
- Thinkst (interesting post on Mandiant APT1 report from Feb '13) - http://blog.thinkst.com/2013/02/thinkstscapes-2013-ah1-on-china-report.html
- J. Oquendo's analysis of (published) Mandiant data for APT1 - http://www.infiltrated.net/aptredux/