Professional exploit tools

Last week I was presented with a proposal to 'pen-test' my network with industry class exploit tools like Core Impact, Nessus and MSF Pro. The notion  made me chuckle to myself, as I sat quietly through the remaining slides in the deck.  This company are clearly way off the mark with their thoughts around security and the benefits of penetration testing, even more so in the understanding of what these tools can and will achieve. 

Now don't misunderstand, automated tools are great, they reduce the workload, reduce the skill level required and they output to nice, easy to read reports which flow easily up to top tier management.  They provide nice dashboards for easy review and they also integrate nicely with other tools in the enterprise.  There are however, drawbacks..

I decided to do a little test to see how many of my lab machines could be compromised by these tools vs the methods I've used, to get a feel for just how effective they are in a real world scenario. 

I set up 10 lab machines, with some flaws that are reasonably obvious through to others that are either obfuscated or require a degree of thought to compromise them.  My lab machines are designed to be as you might find them in the real world, and all were set up with exploits that are publicly available.

The results
M Pro - 3/10
CI 4/10

The results are fairly poor, less than 50% for both (they would have been tied at 4 each, but although correctly identified, the exploit wouldn't fire a shell using the other tool).

I was able to compromise all of these lab machines without these tools using good old fashioned enumeration before targeted exploitation, in roughly half the time it took to run the tools themselves, with a fraction of the network traffic.

These tools are designed to bridge a skills gap, and they are truly excellent for that purpose, they are reasonably easy to pick up and "point and sploit" but they are not a panacea, nor do they offer full assurance that the environment has been fully enumerated and tested.  Even in the hands of skilled pen-testers these tools have obvious limitations, (although I must disclose, that its difficult to run this test unbiased as I already know the weaknesses in my lab machines)

The worrying thing, some businesses see these as a means to replace expensive penetration testing, others (like the company presenting to me last week) believe they are the foundation of the testing methodology and are absolute in the identification of weaknesses in the environment.

My advice, these tools in the hands of someone skilled will get you  further than someone who can "point and sploit" , but the last mile, should probably be entrusted to someone who steps beyond the boundaries of professional grade tools.   

Use them as a means to reduce the cost of professional testing, but not to replace it altogether.