Wednesday, January 14, 2009

When automation is too much automation or where's the TLC??!!

Like the"hiring geeks" post by ax0n said, we automate. Automation is a good thing but when it comes to pentesting I really think there can be too much automation. Too much automation leads to "fire and forget" activities and lack of TLC.

For example there are a couple scripts out there that try to automate your whole scan, enumerate, and exploit process and all the pentest frameworks have some sort of autohack feature and they all suck (as much as it pains me to say that --especially because I am such a msf fanboy).

There is a certain amount of diligence I think that should be applied come actual "exploit" time. Scripts that automate or allow a "tester"(?) to script too much of the pentest while handy can cause real damage on a network, not to mention MISS things, possibly IMPORTANT things.

I've heard that some people will spend a ton of time writing a tool that will run everything from nmap and a bunch of different exes, some that do automated exploitation like adding rogue user accounts if it finds null passwords or something, or whatever random exe's that can find on the net. run that script and go for lunch.

The problems arises that:

1. That much output really saves you no time if you go back and actually go through and validate the results.

2. Seems like no one knows how to enumerate and certainly no one teaches it. Automating all the steps between scan and exploit don't help the lack of enumeration either.

2. There is no "test" you just ran a bunch of tools, the script did all the "work."

3. There is no personal experience or tester analysis if you just run a script. There is no thinking outside the box or expertise involved if you hit the autohack button or ./autohack.rb

4. What about stealth? what about tactics? what about proper footprinting? what about emulating anything besides a script kiddie attacker?

5. Where is the fun and challenge in having the script do all the work for you?

6. Every pentest is (at least should be) a battle of minds against the tester and the people admining and securing the network. If you've got any kind of decent admin the easy low hanging fruit should be patched up but that doesn't mean there still aren't vulnerabilities to be found and exploited by an experienced tester. Its all about finding the one thing that guy missed and then digging in from there.

7. There's no TLC with autohack, for the amount of cash you paid for a "real" pentest, there should be some love and work from your tester, that nessus report just aint cutting it.

Moral of the story, show some TLC, get good (better) at your craft and don't rely on the latest autohack script to do things for you.


ax0n said...

Nessus and ISS Internet Scanner were the big names when I was pen-testing for a living. eEye was just stepping into the scene. I've (obviously) been out of that gig for a while.

Automating mundane and repetitive things (such as having to manually re-start this one process on a weekly basis) is great.

Automating an entire vulnerability assessment is dangerous and plain stupid.

I'd usually automate some things. I had scripts to do simple host/service enumeration tasks. I had scripts to automate some common web app problems, but keep in mind those at the time were things like Cold Fusion examples being left out on the web root, using the Microsoft IIS ::DATA Alternative data stream to get Database Username/Passwords and things of that nature.

Pen testing should definitely be primarily manual. Most of the time, my "automation" was limited to a small bunch of command-line snippits and tricks that I came up with to quickly pipe a bunch of stuff together for output clarity.

Ulisses Castro said...

Like my friend said, "We can reach nothing without love." auto-magic-tools has no LOVE!

Nice post!

gabeleblanc said...

word CG it's mostly manual for me, but that's cuz i'm too stupid for scripting.
But for nailed it again!

CG said...

the real haxors have spoken... i evidently dont know shit