Socs vs Greasers: The Pentesting Debate

During the Podcasters Meetup at Shmoocon 2010 a conversation began about the worth of penetration testing in the corporate environment. On one side of the tracks, business folks questioning why it was necessary to active exploit business systems while penetration testers argued for such. The conversation and debate has continued on many mediums since Shmoocon and many involved have made some valid points. I have not heard much from anyone who is in the trenches of security operations, however. Since this is my primary role for the small enterprise I support, I thought I could add some of my own perspective to the discussion.

I am the typical one man security show that is not uncommon within business the size of my employer. I deal with all aspects of security for the organization including vulnerability scanning and penetration testing. Other responsibilities include regulatory compliance, incident response, patch/vulnerability management, and security architecture. So my view on penetration testers and the services they have to offer is the same as any other consultant or contractor that walks through my door. I welcome the second set of eyes and assistance.

The reality is with all aspects of my daily responsibilities, I am going to miss things, make configurations errors, and downright fuck up from time to time. The fact the matter is I get tired, have a family, and often don't know my systems as well as I may think I do. I am a juggling clown balancing on a unicycle with a warped rim riding right down the middle of the train tracks separating these two groups.

This debate is not new and many others have already touched upon some of the pros of penetration testing. Defense in depth by way of post exploitation testing is one such argument that is completely valid. There are a few additional arguments I would like to make in regards to the usefulness of penetration testing, however.

1. Your penetration tester should not be testing things that you know are broken. This wastes the consultant's time, your money, and does no one any good. If you know it is broken, evaluate the risk then fix it or put the appropriate mitigation in place so that it can be tested during the next engagement.
2. Sometimes exploitation is the only way to verify something is broken. The Symantec exploit I blogged about last October is a great example of the risk assessment and patch management process failing within an organization. This was a situation where the only way to verify that a system was vulnerable even though it was patched was to run the POC on it. Such situations, while not the norm, are also not unusual. If you are trusting your Vendors to secure your environment, you are doing it wrong. It should be noted that the vulnerability was weaponized several months later as reported by dshield.org here.
3. Incident Response! You do have an Incident Response plan right? Thought so! Do you review and practice it? What better time to see how well your IR plan works than when you're actively being attacked. A Penetration Test is a great time for the entire team to have a "fire drill" of sorts. I recently had the opportunity to listen to Andy Ellis speak about incident response. Andy serves as Akamai's Senior Director of Information Security and Chief Security Architect. His statements about availability made an impression on me. If your management is really serious about maximizing up time, then you better have a lean, mean Incident Response team. It is not a matter of; if you have a compromise, it is a matter of when, and how well you respond to it.

What about APT do you ask? I will leave you with a recent description of APT from Matt Olney of the VRT Sourcefire team;

APT: There are people smarter than you, they have more resources than you, and they are coming for you. Good luck with that.

Matt's advice includes building a security team with "... at least one very bad person" on it. For the small business security professional, that person is often the penetration tester. Besides they are usually much more fun to have a beer with than senior management.