Saturday, February 13, 2010

Further Proof You Don't Get It

At OWASP AppSec last year I had some remarkably accurate conversations with Josh Abraham (of Metasploit, Rapid7 fame)... and I've gone back to those in my mind several times now over the last few months and scoured the 'net for signs that someone is running with those thoughts and ideas.

Sure as the DC snowpocalypse ... nope.

It's a little scary to me that Josh's thoughts on "Pen testing with a purpose" (paraphrasing) hasn't gotten any buzz. I know he's worked hard on it and even wrote a pair of blog posts on it a while back ("Goal Oriented Pentesting - The New Process for Penetration Testing" and the follow-up).

I still see penetration testers using the buckshot approach where they're hired to "break into a site" and they just go at it like rabid wolves on a wounded impala.  I don't get it - how do you know when you've succeeded?  Do you count it a success when you've found XSS?  SQL Injected the database?  Broken authentication?  What's the logic?  Just sad that this topic has been rejected from conferences since so many testers need to hear it!

I say this is further proof that "you don't get it" (well, maybe not you, know, you) because this seems to be the sorry state of the security field right now.  Call me insane but I would love to have a standards-based penetration testing framework which everyone built SoWs (statements of work) from.  Would it be too much to ask for some sanity?

Look, I just got done working with a customer who decided they didn't need an internal security program for their 1,000+ web applications because, and I quote, they "had an external 3rd party assessing their apps once a year".  It's so ridiculous I couldn't make this stuff up!  What's worse is that this isn't the type of company that can afford to be lax with data protection... They were kind enough to introduce me to one of the "Statement of Work" documents which basically outlined what this 3rd party (a well-known security firm, by the way...) was going to do.  "Assess the external exposure of {company} web applications"...  I made the effort to call their penetration testing "guy" and had a quick chat to see how they were doing things.

"We have repeatable processes we follow.  We basically try to identify all the stuff that's broken in the site."  {mouth agape}  I asked how they knew what the important targets within the site were - was it based on a site profile, some insider knowledge, or what... "Nope, it's just a web site, the goal is to break in".


I could have sworn there were already some pretty good penetration testing frameworks (at least for web app pen testers) that could help focus an effort like this to things of importance within the site.  Goal-oriented penetration testing is the only way to test some of these behemoth web sites (applications) we're all seeing ... I can't be in the minority of people who thinks so, this isn't a revolutionary thought.

So - is it time to add an OWASP project for this?  (Or maybe it already exists and isn't well known?)  Let's hear it ... there has to be something I'm missing.


Kim Guldberg said...

Some of us have been using Jabra's approach even before he blogged about it

DaftDoki said...

The linked original article is very apt, and I remember reading it originally and agreeing very much with the premise. For a while I've been telling my customers to save penetration testing for when they have clear goals, and feel that they've done their best to take care of the low hanging fruit; otherwise it's a waste of time and money.
Spend something like ~80% of your time and resources taking care of the obvious, and then use the remaining ~20% for precision activities.

Penetration testing should be about precision testing since presumably you're paying people who are specialists and at the top of their game; these folks don't come cheap.

Ben said...

The problem is two-fold. First, the pentesting community is one that prides itself on art, not science. Especially many of the younger/greener pentesters like "the scene" more than advancing the science. It's part of that whole breaker culture I think, such as you see at DEFCON.

The other problem is that people in this sector of the industry simply don't know about the formalized approaches. Vulnerability testing has it's place, but there's also a place for threat testing (red teaming). It seems increasingly important to include red teaming in the mix and not just focusing on how many vulns you can enumerate or exploit in a given period of time. Anyway...

CG said...

previous commenters nailed it. The idea of having a goal besides grabbing 50 shells isnt new.

issues that arise are:

companies need solid risk assessments and solid vulnerability assessment programs before they can even determine a realistic goal for their pentesters.

you really need longer than 1 or 2 weeks to adequately test a good "goal" because getting domain admin on internal pentest is a stupid goal.

whats the point of having someone come in a do a full scope pentest if they could get 50 shells via autopwn flavor of the month. you have MUCH bigger issues

Lastly, how do you expect OWASP or anyone else to model full scope testing when the whole point is that there is no scope?

I dont expect most people that have that level of knowledge and experience to freely educate the competition. I also don't expect most companies to pay for it either. But i'm frequently wrong, so it will fun to see how it progresses.