Security Conferences, pen tests and incident response

My first post on the new blog is going to be more of a philosophical rant than a technical entry, but hopefully still enjoyable. I've been having interesting conversations with Ed Skoudis, Peter Silberman and tons of other people informally about these topics for some time, and decided to share some of my thoughts for whatever its worth.

First; security conferences. Originally they were places to meet up with friends you don't see often, exchange ideas, and learn about new things. Increasingly they have become advertising venues for your product, company, or researcher brand name. Now that's not all bad, it helps people move forward, get opportunities and advance the industry but we should try to remember our roots and not lose that sense of discovery and pursuit.

An aspect of this that I think we are all guilty of is: "I've got this great new idea / tool / 0day and it blows your socks off, but I'm not going to tell you about it unless you come to the con." Somewhat necessary to promote talks but starting to get out of hand, and often the talk doesn't really provide what the advertising offered.

Next is the fact that the security industry has the same problem the music industry has, which is, if something "works" i.e. gains a lot of press and general interest, they try to milk it with 100 variants following the same formula for some period of time. (Boy bands, britney spears types, emo, hypervisors, rootkits, cloud).

Another issue is that a lot of talks are ones the speakers would like to see, which generally trend toward the more theoretical and academic. While there is definitely a place for this, what I hear over and over from general audience members is that they come to cons hoping for something they can take home with them and use. An hour lecture on operating system design, virtual memory management or the latest tricks on upcoming architectures aren't much help when the enterprise back at home is still running XP SP2 and they just want a baseline security audit or some way to help from getting owned by today's threats.

Now I've been involved in talks like these which usually gain interest from other speakers but I've also given practical tool and technique driven talks which usually get 10x the appreciation from audience members. (validated by feedback numbers) I have become a big fan of talking about actual hacking at hacking cons rather than just computer science.

For penetration testers (big consumers of conferences btw) the unfortunate situation has arisen (or maybe always been there) that companies have a vested interest in buying a poor pen test. Many companies have not yet developed the ability to identify, document or even discuss the real risks to their business and are barely holding on by figuring out whatever regulations they need to follow and checking off the boxes. They need to pass. Shinking budgets mean they need it cheap. This means that pen testers are selling something with little real world, but lots of bureaucratic, value.

Therefore being the best pentesters, using the latest tools and techniques can actually have a negative effect rather on your ability to gain customers, rather than a positive effect. I haven't come up with a lot of creative ways to fix this, but one idea was to couple incident response with pen testing somehow. If the in-brief, or pre-roe includes real world examples of how places got hacked, and the actual effect on business perhaps it would be easier to sell some of those less "conventional" but more realistic techniques. I think pen testers need to do a better job at both describing the real threats as well as expending beyond the scan/exploit numbers game into physical, SE, and monitoring what the bad guys are actually doing.

This leads finally into incident response. There are two kinds of organizations out there in my experience (I encourage other examples!). One that wants to know exactly what the bad guys did, what they were after, capabilities, techniques and tools. The more data about what happened the better. The other type is one that doesn't care what happened, just want's the attacker gone and their infrastructure back up. I'm finding its very important to figure out which one of these two you are dealing with going into a situation because the response is completely different. This is probably old hat to IR people but I'm used to dealing with organizations that want to know what happened.

So lets see, compare infosec to pop music? check. Dis the academic community? check. Tell pen testers their work is for naught? check. Now that I've alienated just about everyone in the infosec industry, I think this is a good place to wrap up the post :). Hey at least I didn't discuss vuln disclosure.

Thanks for reading,

V.

Trackback URL for this post:

http://carnal0wnage.attackresearch.com/trackback/361

Securing over the top - in depth vs. just sketchy

About good people in IT being too good and bad guys playing better

Bugs flew in the Eniac and caused errors. Since ages bugs cause trouble in IT. Now it's time to exterminate them?

Comments

Disagree

Ok, so I fall into the speaker category here, so this reply may mean nothing to you, but I don't know anyone else who would appreciate one of these talks, but this is probably because the people I know like to learn & think for themselves.

Conferences usually have trainings attached to them which do exactly what you're talking about: giving some useful information about how to achieve things in practice, and those usually have clear pre-requisites.

If that's what people want, then why not go to those instead of the conference?

People have varied skill levels, so you're never going to get a talk that's perfect for anyone, but more advanced talks mean everyone learns something, if you don't understand something, then the background that you need to understand it is usually pretty clear and if you're want to learn about it, there are probably tonnes of resources. Why even bother going to a conference to hear someone talk at you for 1 hour about something that you could probably find just as easily online?

Regarding "I've got this great new idea / tool / 0day and it blows your socks off, but I'm not going to tell you about it unless you come to the con.", would you prefer "I've got this great new idea / tool / 0day and it blows your socks off, but I'm not going to tell you at all, since I don't want my bugs to die." or "I've got this great new idea / tool / 0day and it blows your socks off, but I'm not going to tell you at all because it's just too much hassle and I have no motivation to disclose."?

P.S. DEFCON still seems to have a lot of 101 talks with absolutely nothing new or interesting.

re: incident response

the majority of IR clients I see just want to know if they have to disclose the breach, maybe some remediation steps to put under their pillow at night, but not much else.

Going to pull out what I

Going to pull out what I think is the biggest and best line in your post, and I agree with:

"I have become a big fan of talking about actual hacking at hacking cons rather than just computer science."

That's really it right there. Some things are best left to academics or white papers, and not a talk at a hacking con, imo. Give me something I can actually use or be interested in enough to take some nearly immediate action on. Theory and CompSci is great in moderation at a con.

I wonder if some of this is due to two things as we move forward year after year:
- the same old crap still works
- the new crap is getting complicated (virtualization, cloud...)

-LonerVamp

can't put the whole blame on the security industry

While it's convenient to put the blame on the security community as many in the anti-sec movement try to do, I feel they only are partially to blame for some of the sociological phenomenon surrounding the practice of hacking, security and how it fits into our economy and society. You must remember that serious criminal malware exists and is being used with serious consequences. PCI regulations for instance are likely to have been put into place by the number of credit card data breaches taking place. Who was behind most of those breaches? NOT the security industry, but criminals. Criminals, who may be using hacking techniques but this does not make them 'hackers' in my definition. Sure, the security industry loves FUD, hype and is a snake eating it's own tail in many ways. But how many people are working in the security industry who have an opportunity to engage in the work that they are passionate about, that gives them the sense of exploration and development that they crave? I'd guess there are more than a few, who have taken their passion and interest and motivation and turned it into something meaningful from an economic and business perspective. It would be nice to think that hacking and security conferences can exist outside this reality but they don't, or at least they cannot escape it's influence that permeates nearly every area of our commercialized society. Granted, some conferences are less filled with corporate bullshit than others. Some still have the spirit that I've enjoyed for many years. Filter out the bad, strive to enhance the good by contributing directly to those areas and groups that are putting your vision into practice, and that will help build up the spirit in a way that I think will be a positive situation. But the whole situation is hardly a black-and-white scenario. There are shades of grey, nuance and subtlety to this discussion.

-timelord

The Underground Myth, revisited, Phrack 65-13

The Underground Myth, Phrack 65-13
http://www.phrack.com/issues.html?issue=65&id=13#article

The Security Industry has killed the classic age of hacking/phreaking where information was the goal.

I see daily the checkbox mentality that regulations have imposed. Sad really.

further preaching

(I can't figure out how to register, perhaps I should hang up my hacking shoes now?)

cg, I think you may have hit on the solution and not realized it. What the cons need is more actual tracks to teach you stuff. Like, 101 talks for a given class of talks. In order to go to, and understand this track/class of talks, you should go to these 101 talks which are on the first day. The problem of course is logistics (it always is), a.) you have a bunch of folks that don't need to 101 stuff, so what do you do with them on day one? and b.) your 101 talks will have to be developed _after_ your speaking engagements are set. Well, for a) why not do the crazy 'academic' stuff on that day as a 400-track? And for b) I'm sure folks that are running these cons, between the bunch of them, can come up with introductory slides/talks on any number of these subjects, as it seems most of the clueful folks are 'generally' knowledgeable.

Something to think about, at least.

I think the con should be a place to meet, share, and learn (not in that order of importance). Frequently, the only folks that are learning are the folks that already grok all the basics, or the folks that already have a set group of friends that they spend time hacking with. This kind of interaction is frequently lost on those of us without connections that attend out of the blue for the first time. Workshops really help this.

FWIW it's not really all that bad at these things now, I am hardly a grizzled Defcon veteran, but I was happy to see that it had all the interactiveness of a Shmoocon in terms of side-booths with stuff to do and things to learn. There still is a barrier to entry as you need to know someone to get started, though, and sometimes (for the shy) this can be a hard bridge to cross. Actually, the best example I've seen of spontanious, in-con learning is the TOOOL folks w/Deviant Ollum. That's the way to do it.

Right now, the only cons that seem really dedicated to teaching/learning interactively are the SANS ones, and unfortunately the price is a barrier to entry for lots of folks.

I'm sure financial concerns govern most of what I'm suggesting, but it's food for thought...

--Mish

(mishley@gmail.com, @mishou)

preach it! as far as cons go,

preach it!

as far as cons go, I actually liked it when Defcon and other cons did the 101 track, and I think there is still a need for a 101 track in most cons.

This shift of essentially only talking about "possible" attacks because all the "probable" attacks have been talked about or "not con worthy" was a shift in the wrong direction. There are still a ton of people that would get benefit from 101 style talks, the speaker pool grows, and usefulness goes up. What good is some fancy obscure way to do SQL injection if you dont understand the basics? I for one like talks where i can take that info back to work and start using that in my pentests or at least sit there and go "hmmm never thought of that or i'll have to try that." Some attack that will work on 1 out of every 1000 pentests that will probably be fixed soon or WAS fixed when i heard about it isnt as useful as something maybe less technical, cutting edge but more functional.

Great Post

One of the hardest lessons as someone trying to straddle the fence between researcher and pen tester is the sad truth you write above: "For penetration testers (big consumers of conferences btw) the unfortunate situation has arisen (or maybe always been there) that companies have a vested interest in buying a poor pen test...Therefore being the best pentesters, using the latest tools and techniques can actually have a negative effect rather on your ability to gain customers, rather than a positive effect."

I'm still trying to figure out what that means for me professionally. It's hard for skill and passion for your work to have negative consequences.