7 Common Mistakes Choosing Penetration Testing

1. Not understanding what you are getting

Photo by RapidEye/iStock / Getty Images

Many organizations offer highly discounted "penetration tests" that are really automated vulnerability assessments. A vulnerability assessment does not equal a penetration test. A penetration test is much more thorough, includes multiple tools, and manual analysis. Vulnerability assessments typically use a single tool. For our penetration tests, we run a vulnerability assessment tool, in addition to numerous other tools and manual methods.

2. Not asking for a retest

Once you receive the penetration test report and remediate the findings, you should test the remediation to see if it actually worked. Many organizations receive a penetration test report, "fix" the discovered problems, but never validate their fix actions worked. The validation often comes in the form of an unwanted attack or breach.

3. Not asking for a Letter of Attestation

A Letter of Attestation provides sanitized proof that you had a penetration test performed and highlights the results of the penetration test. Hopefully, after you've done the retest, the results show no critical, high, or medium findings. The Letter of Attestation is great to show clients, partners, and anyone that needs assurance that your systems have been penetration tested by an independent 3rd. party.

4. Using a vendor without a documented process

The vendor should follow a documented process. This should include, at a minimum, clearly defined Rules of Engagement for the penetration test, explicit authorization forms, and cloud provider penetration test request guidance. The Rules of Engagement process is one of the most critical because it defines the systems to be tested, the timeframe, the source IP addresses of the attacker machines, escalation procedures, points of contact, etc.

5. Not receiving guidance on the type of penetration test you should receive

A Black Box Penetration Test is one of the most common. This means the penetration testing vendor only knows the IP address or URL of your systems.  If you have web applications or systems you have developed though that require user-level access, you should also receive a Gray Box Penetration Test. The Gray Box Penetration Test is a penetration test where the testing is performed with user-level, authenticated access. This can uncover flaws in your web application - flaws, such as a user being able to access another user's information or a user being able to elevate permissions from user-level to administrator-level. All of our Gray Box Penetration Tests include the Black Box portion as well. The penetration testing organization should determine what type of tests you need during a scoping discussion.

6. Not asking for a detailed debrief or report walk-through

After the penetration test is completed and the report is delivered, you should receive an in-person or online debrief of the report. The penetration testing organization should walk through the report in detail with your team, so you understand the findings and fix actions.

7. Not asking for a report with detailed steps and reproducible results

Many penetration testing organizations simply run dozens of tools without understanding any of the tools or the techniques the tools use. They will often tell you they hacked into one of your systems, but they cannot explain how they broke into the system, nor can they reproduce the result. Without knowing the exact vulnerability and technique used, it makes it impossible for you to fix your system or validate the fix. The penetration test organization should know exactly how they hacked into your system and have the steps thoroughly documented.