Thursday, July 2, 2009

"SecurityMetrics" - Another Site Security Certification Seal

You know how much I hate these things...

You also know how much I hate it when these people sell "security" carefully wrapped in bullshit and smoke... under the pretense that their "scan" will actually do anything to achieve some measure of security.

While looking for some new hockey pants (yes, I have destroyed my current ones) on I saw this interesting seal. Clicking on it made me cringe even more. This is a measure of PCI Compliance? ... and this is supposed to make me feel good about the actual security of the site? Clicking on the damn thing brought up the "Site Certificate" which should be an immediate red light for anyone looking to do business on this site.
First off, this is a quarterly certification... holy crap! The last "Certification Date" is shown as May 14, 2009... which immediately makes me worry since I can't recall the last time I saw an e-commerce site that stayed static for almost 45 days... but let's move past that because after all, compliance is a point-in-time thing... right?

Alright, this next part really gets my blood pumping and feeling like a bull out of the chute... read the first sentence of the text dead center of the Site Certificate carefully...
"On May 14, 2009 met the PCI dada security requirements by passing a Securitymetrics Site Certification vulnerability scan"
Come again? Maybe I read that wrong. Nope... read it again and it still sounds just as idiotic.

So, let me get this straight... SecurityMetrics has managed to figure out how to achieve the full spectrum of PCI-DSS Security Requirements via a vulnerability scan? How is that even possible? Since SecurityMetrics is scanning the site from the "outside"... how do they know if the various sections are all met properly? Are desktops being equipped with properly updated anti-malware agents? Are default passwords not used? Something smells like a steaming pile of bullshit.

At least these guys don't make outrageous claims such as that they are "Hacker Proof" or "Hacker Safe"... and instead do say that the scan "significantly reduces the risk that this site will be compromised..." and while I wouldn't give them significantly, I may agree that it does reduce overall risk but only as much as me wearing goloshes in the rain reduces my risk of catching the H1N1 (Swine Flu) bug.

So let's investigate this genius PCI Compliance scanning service that will magically achieve PCI Compliance for their customers a little further, shall we?

From the Site Certification Overview page...
Is Site Certification Easy? It is easy. Site Certification does not require any software installation, software configuration, training or costly maintenance. All your technical support is included and there are no hidden fees. SecurityMetrics does not require confidential system information or access to your systems. You simply enroll and the service is scheduled to run at your convenience.
D'oh! I'm going to ask again... how do they determine any measure of PCI-DSS compliance without access to merchant systems?! Are we doing Scanless PCI again?

Their FAQ Page has a priceless little illustration of the devilish "hacker" exploiting "security holes" in the web server... which is so funny I had to stop a minute to quit laughing. Bulletpoint 3 appears to hint that SecurityMetrics does some measure of web site security testing... to me that means testing for things like SQL Injection, Cross-Site Scripting (XSS), CSRF and other common security vulnerabilities, yet there seems to be no mention of these common vulnerabilities. Instead the site's Product Comparison talks about how many ports they can scan and how many "vulnerabilities" they can identify and scan for.

My absolute *favorite* page on their entire site is the Sample Test Results.I love it! Take a look at this for 5 seconds and tell me this isn't a blatant rip from the Nessus results reports? Take that back... Nessus looks much better these days than this poorly-constructed "report". My guess... they're just Nessus scanning sites and calling them PCI Compliant. [bangs head on keyboard].

One last thing I need to point out, this page which is a List of Vulnerabilities that SecurityMetrics scans for. Out of the total of 5,882 checks (as of today) they break down to 4,486 vulnerabilities, and "if telnet or ftp is enabled the vulnerability assessment engine will test 698 names and passwords common to these services." [mouth wide open... *gasp*]

Let me just say that I read through this list of vulnerabilities and it amounts to nothing more than some basic pattern-checking and typical vulnerability scanner type crap. There are no checks for CSRF (Cross-Site Request Forgery), no checks for XSS (Cross-Site Scripting) that don't involve a vulnerability in a particular application package (i.e. .Net XSS), and no checks for non-specific SQL Injection vulnerabilities... once again - a complete failure of a security service.

The thing I have to wonder is (and I already know the sad answer) why do site owners keep using these services?! For example, JetBlue is apparently one of their customers [not to self: avoid JetBlue website/services at all cost]. SecurityMetrics is not a known brand in security and they have a non-starter product so what draws people to use them? Is it the prospect of having a "PCI Certification" seal somewhere on their website causing them to lose their better judgement?

Logic fails here ladies and gentlemen. Why doesn't someone from the PCI Council do something about companies like this? Isn't it [or shouldn't it be] illegal to claim you can certify someone as PCI Compliant with this rediculous service - when in actuality that's not even close to true?

So... anyone know of any fun XSS vulnerabilities in JetBlue's site, or any of the other SecurityMetrics testimonial customers they'd care to share?


Gunter Ollmann said...

Interesting. I'm not familiar with the organization, but perhaps they're being unfairly criticized?

Surely, as seasoned security professionals and contributors to security evangelism, have we not been the ones that have failed here?

A market obviously exists for this kind of "certification" (or validation, etc.) because average-Joe doesn't know any better. What should we be doing to further educate the market in why these kinds of certification hold no technical anti-threat value?

We know that there is an established (and growing) market for "good enough security". How should we really proceed in raising that level and the consumers expectation?

Anton Chuvakin said...

Rafal, it sounds weird but I'd side with Gunter. Think about this in the following manner: SM is not a security company at all, they simple sell website seal that increases merchant sales since it makes his customers feel better.

See, there no scam - as long you remember that this is not about security; they just picked a particularly "unusual" name for their company :-)

Rafal Los said...

Gunter/Anton - while I do agree with you that this is NOT a security company - I don't think that gives them the right to mislead people. This is akin to me opening up a "rust proofing shop" and charging $100/yr to "rust proof your car" when in actuality I get under your car and spray it with water once a month/year/quarter... right?

It's a placebo that's being sold as the real deal - so it's still a scam as far as I'm concerned. You can't have it both ways.

Anton Chuvakin said...

Well, not with water, but with regular paint. And you don't claim rust-PROOFING just "improved rust resistance" :-)

mckt said...

The security seal is nothing but snake oil, and it's training users to expect (and worse, to trust) that snake oil. If that isn't a scam, I don't know what is.

The only value this service has is that of any other Nessus scan (yes, that's basically what their scans are)- finding out of date software, poor configuration, and known vulnerabilities. You won't find any new issues with it, and you certainly can't test for PCI compliance.

That said, I'm somewhat familiar with the guys at Security Metrics, and some of them actually do know their stuff... but those people aren't in sales or marketing. I don't think they're the ones pushing the security seal either.

Trey said...

Man Raf- you're striking at something near and dear to my heart. You are also calling out those who perpetuate the myth- for fun I will still people to

Gunter, Anton, and Mike all have offered great banter that I agree with- I will summarize my thoughts in line with that saying that there is apparently a market for some form of a 'security seal.'

Consumers seem to respond to them, and the merchants know this. McAfee did too ... follow the redirect :o)

I am still deliberating how we might attack this problem, as many out there still think that a network vuln scan implies PCI compliance- certain 'security seal' programs perpetuate that kind of madness.

As Gunter put it so very well, the "seasoned security professionals and contributors to security evangelism" would serve our contingency well to petition the PCI Council to remove all Web App Sec references from the 'ASV Validation Requirements' as commissioned in PCI DSS Requirement 11.2

Very solid post- you and I will be expanding in this further very soon!

Anton Chuvakin said...

I think this is a lost cause, sorry guys. Maybe you can call me a cynic, but I suspect the battle of explaining "how is a vuln scan =/= PCI" is lost.

Thus I suspect the only solution is to build a wall between those looking to "buy seals" (or sea lions or sea elephants or whatever other fat sea critters) and those looking to implement information security.

Clerkendweller said...

Thanks for alerting us to this one...

Anonymous said...

SM just has a good marketing strategy. We are a small non profit that uses a ticketmaster spinoff called iats as our payment processor. We recently received a letter from securitymetrics stating that IATS has contracted with them providing PCI compliance certification. IATS is going to bill us $99 /year and another 19.99a month until we sign up with Securitymetrics. I called IATS and verified, since SM had our merchant ID. Well, if I want to use seomone else to do PCI compliance testing, I can't avoid the $99. I have to get a credit after it's already been billed. After talking to SM, I have a feeling that once they runner their "rigorous tests" I will probably "need" to purchase more stuff from them to ensure my compliance. So, that's how they're getting customers. In the end, I will probably go through the farce, because I've already spent more than $99 worth of my time confirming that this is all BS...

Anonymous said...

Please people! Just take a quick peek under the covers please! You will find that the PCI Standards were created by and maintained and supposedly enforced by banks and large credit card companies! Does this suggest anything to you? It sure as hell does to me! The banks and credit card companies are going to protect you from security fraud or intrusions? Give me a break, this seems to be an obvious way to create a revenue stream for these guys. Start to dig up info regarding these security review and compliance companies and you may find a line back to the financial industry that is telling us this is required.

Mostly fees from the merchant card processing firms, the test, review and compliance firms and the threat of fines from the banks and credit card companies is what you will read on the surface. These are called scare tactics and if allowed to follow through by the banks and credit card companies it will probably work. I call it legalized extortion, kind of like insurance ya know?!

Worried Citizen

Alex said...

100% SCAM, NO EXCUSES, are you on the Security Metrics Payrol Gunter? Chuvakin, a company which is not a security company thats sells security is a FRAUD.

Hayes said...

Unfortunately, some UK companies have NO CHOICE but go with these muppets and HAVE to pay them £100 per year for the privilege!

Barclays now charge a higher percentage for payment processing without the SM certificate.

Anonymous said...

So any security experts want to knock Security Metrics off the block then figure out how they got top billing for getting several merchant services.
Was it via the credit card companies (V/MC/DV/AMX)?
It is a moot point to joe the business operater if they really provide PCI compliance testing. The real issue is the bottom line of the cost of 'compliance' & losing the ability to process cards at all. This small company paid the monthly fee instead of going through the greater expense(at the time) of Security Metrics then we got a letter this year saying that if we did not become 'PCI compliant' then we would no longer be able to take credit card payments. So we played the game & used Security Metrics because we are a small air conditioning company that must be able to use the credit cards from our customers. Looking forward to getting another letter from our credit card processor with a choice of two or three services instead of just Security Metrics.

John R. Morgan said...

Thanks for this post. We got a phone call from these idiots this morning. The hilarious part is that ALL of our payments are processed via PayPal! Sure we use terminals in our stores, but all of those are direct connectedto networks (no connected WiFi, servers, client computers, etc.) that are totally separated from our main business network.