Friday, February 20, 2009

Response to How to Choose a Pen Tester

So a response to "How to Choose a Pen Tester"

Let me start with that I agree with the core of Steve's argument. Yes if I pay someone to come in and do "anything" on my network I want to be able to trust them not steal info, plant trojans, or air my dirty laundry out on the net when they are done.

I don't disagree with that.


A few comments not sure they are quite counterpoints

1. I personally don't see a big prevalence of pentest shops doing pentests and posting customer data on the net in any form. If there are examples show me. He mentions in 6 months he heard ONE story about someone that did that and didn't provide a link...ummm ok. Is it believable that it does happen/has happened/could happen?...yes. That every pentest shop is doing it (except his which is really the point of the post)... doubtful. Its not a smart business decision to 1) as a company do that or 2) allow your testers to do that on their personal blogs.

2. As David Hull mentioned, what is the problem with talking about a pentest as long as the customer cant be derived from the post/presentation/email or there isnt enough actionable information to conduct the attack? If companyX was vulnerable to SQLI 6 months ago and I went in and found it using some creative method and i decided to share that experience on my blog or at a conference what is the problem with that? The company isn't vulnerable any more and if I had to figure out some new method of doing "whatever" unless it was explicitly in the contract not to share "new pentest methods" aren't those mine to share as a I see fit? It helps the community when others talk about things they have seen on a pentest even if its just to make the other guy feel a tad bit better than someone else lives in jacked up network hell. Even though I always get alot more out of peoples posts about their pentests.

3. I realize that Steve proposes you do a scorecard but really....trustworthiness over competence? Why on earth would you ever even consider doing business with someone you didn't trust? I don't see how anyone with half a brain would put themselves in the position of...hmmm do I choose the trustworthy CEH or the untrustworthy l33t ass hacker....ummm NEITHER! You pick a company that hires intelligent, competent, trustworthy, and the rest of the stuff on his scorecard people. Is there really that many companies that are that piss poor that even make it past a scoping call? and more importantly do the decision makers for choosing the testers not have the ability to pick the good from the bad?

I should insert a shameless plug here but I don't think its necessary :-)


dp said...

I agreed with Branigan's post. Trustworthiness is of prime import when hiring a pen tester. Do I think a pen tester would be in biz long if they weren't also trustworthy? Obviously they wouldn't last.

I do agree with your point that there's no harm done and in fact, it benefits the community to share findings as long as they don't divulge details about who the client is.

Anonymous said...

Damn. I wish I would have read you post earlier.

After reading the first post, I used the score card approach. I picked my Grandmother to do my pentest. I trust her a lot, and know she would never do anything to harm me.

I am looking forward to a successful engagement.

Thurso said...


"First-ever" wireless firewalls...// easily implemented that easily
erases (most professionals use words like 'patch' or 'mitigate') vulnerabilities...and best of all
Patent-pending technology (== fail)

the rest of his techno-babble just confirms this is just another half-assed attempt at security.

Anonymous said...

Hey Thurso...get a grip! Open discussion of findings is needed in this community. Most items can be related to business processes, not individual ass-hats. Hiding the problem does not help the whole, talk about what you find in the non-attribution context of systemic failures and we all gain.

as for the thin-blue-line security site..yea FAIL.

dmc said...

I wonder if he'd feel different if he was compromised by a vulnerability the trustworthy but not as technically skilled pen tester failed to highlight?

I think it's fair to assume a pen tester is trustworthy if you're hiring a reputable company.

CG said...

@dmc that was pretty much the point i was trying to make. Unethical behavior wouldnt be tolerated by any halfway decent business.

Bob said...

Are references from other companies the best means through which you gauge trustworthiness of a pentester?

Has anyone had a client request full time monitoring of pentester actions throughout the course of an engagement?

I guess my question becomes, "Would more l33t skills outweigh trustworthiness if you could provide some preventative or detective comfort (e.g. monitoring, traffic logging, etc) in the pentesting process?"

Just a thought, I may be out of my mind...

dean de beer said...

Hey Bob,

I've been 'shadowed' on quite a few engagements. The reasoning has been everything from wanting to learn to security protocols. I'm sure at some level trust was also a reason.