Nearly every large IT security group makes some use of penetration testing these days, whether they do so in-house using open source tools or outsource it IT service firms. Either way, it’s happening. And the trouble here lies not so much in the testing itself, but in how the results are interpreted and used.
To start with, penetration test (or pen test) results can be interpreted on at least two levels — technical and procedural. All too many organizations focus on the technical findings of a pen test as a measurement of their security posture. This leaves the organization with, at best, a false sense of security (or insecurity if the test results are egregiously bad).
You see, fault testing, by its very nature, is only capable of testing for the presence of flaws, not their absence. Further exacerbating this weakness is the fact that the crop of available tools, whether commercial or open source, typically only tests for an, arguably, small and known set of vulnerabilities.
So, at its most fundamental level, a pen test does not test your security. It partially tests your insecurity. That’s hardly the sort of ”warm and fuzzy” feedback that I’d be comfortable with taking to the board of directors.
If there is any value to be found in pen tests, and I believe that there is, then it lies in an area that few people pay attention to when they look at their pen test findings — the procedural aspects. By that I mean that the pen test findings can point to problems in systems administration processes and procedures.
After all, the flaws discovered during a pen test are really just symptomatic of procedural shortcomings in the systems management processes.
See Ken’s complete column on e-Security Planet.