Dean and I have talked about this more times than i can count and finally a discussion has taken place over on the pentest list about automated pentesting and a pentester's experience. The thread is here: "Penetration Testing Techniques" I wont get into all the issues wrong with whats going in the post. I'm going to harp on experience and certifications
from thread: http://seclists.org/pen-test/2008/Apr/0039.html
"Well, the results are definitely verified through nmap as well.OS is
win 2k3 running IIS 6.0 and only 80 being open.Yes indeed the client
has assigned us the job to perform the pen test and knows about it.
I do have the CPTS training dvd and am going through that, but it will
take time to digest that horde of information.Also downloading web
goat to get my hands wet with web app testing."
While the thread is initially about CORE IMPACT not finding any vulnerabilities with this particular server, the underlying issue is the lack of experience someone has and them being hired to do a pentest. Its a reoccurring thread on other sites as well; "Hey, I got my CEH, who wants to hire me to be a pentester" :-(
Bottom line, tools are just tools, they help humans get jobs done. They aren't and shouldn't be the only thing used on a pentest. The other point is experience is king, granted the original poster is getting experience, but giving CORE to a brand new tester is not going to help them get better. there is a reason A LOT of subjects are taught the hard way first then you get taught "the shortcut." Oh, and passing a multiple choice test is not a real demonstrable measure of ability.
Let me also add that if one of my employees posted some crap like that, i'd seriously be considering them finding another place to get their experience.
want to learn the right way? check out LearnSecurityOnline's Learning Model. LSO isnt the end all be all of security, but i think the Learning Model and the Core and Advanced Competencies is a solid foundation for any security professional.
Here are the Core & Advanced Competencies:
Four Core Competencies
• Operating Systems
• IT/IT Security Resources
• Documentation, Policies, Procedures, Disaster Recovery
• Penetration Testing
• Security Industry Certifications