"We already run vulnerability scans, so we don't need a pentest." It is a common line, and it rests on a misunderstanding: the two activities answer different questions. A scan asks which known flaws exist in my environment. A pentest asks what an attacker can actually do with them. Mixing them up leads to paying for an unnecessary test or to trusting an automated report that never tested anything.
What each one does
A vulnerability scan is an automated sweep. A tool identifies assets, services and versions and compares what it finds against databases of known vulnerabilities and configuration checks. It is fast, repeatable and cheap per run, which means it can run often and across the whole estate. The trade-off lies in its limits: it points to the possibility of a flaw, produces false positives and is practically blind to business logic errors, authorization flaws between users, or combinations of small weaknesses that together open a path to an attack.
A penetration test (pentest) is a people-driven exercise, with an agreed scope and rules, that simulates a real attacker. Testers use tools, scanners included, but the core of the work is manual: validating whether a flaw is exploitable, chaining findings, trying to escalate privileges and showing how far an intruder would get. NIST's technical guide to security testing, SP 800-115, draws exactly this distinction: scanning identifies potential vulnerabilities; penetration testing validates whether they exist and what their impact is.
In one sentence. Scanning gives breadth and frequency; pentesting gives depth and proof of exploitation. Neither replaces the other.
When to use each
Some typical situations:
- Ongoing hygiene of the estate. Recurring scans covering every asset, internal and internet-facing. This is the foundation of any vulnerability management program.
- Launch or significant change. A new application, a new integration, a cloud migration or a network redesign calls for a pentest before or right after going live.
- Critical web applications and APIs. Authentication, access control and business logic flaws rarely show up in a scan. Here, manual pentesting is what finds the problem.
- Regulatory or contractual requirement. PCI DSS, for example, asks for both: internal and external vulnerability scans at least once every three months, with external scans run by an ASV, a scanning vendor approved by the PCI SSC (requirements 11.3.1 and 11.3.2), and internal and external penetration tests at least once every 12 months and after any significant infrastructure or application change (11.4.2 and 11.4.3).
- Validating controls. When the question is "do the firewall, the EDR and the segmentation hold up against a real attack?", only a test that actually tries to attack can answer it.
In practice, most organizations need both, at different cadences: scanning all the time, pentesting periodically and aimed at what matters most.
How to choose the right pentest
Start with scope
A poorly scoped pentest produces a report that doesn't answer the business question. Before requesting a proposal, define which assets (external perimeter, internal network, web application, API, mobile app, cloud, Wi-Fi), what is out of scope, testing windows, emergency contacts and what is forbidden (denial of service, social engineering, production testing during business hours). This agreement, known as the rules of engagement, protects both parties and should be in writing.
Black box, gray box or white box
- Black box. The tester starts with no internal information, like an outside attacker. It is the most realistic option for the perimeter, but part of the time goes into reconnaissance, and coverage tends to be lower.
- Gray box. The tester receives partial information, such as a regular user's credentials or basic documentation. It balances realism and coverage and is the most common option for applications, because it simulates a malicious customer or employee.
- White box. Broad access to architecture, configurations and sometimes source code. It is the deepest and most efficient way to find flaws, at the cost of less realism about what an outside attacker would see.
A recognized methodology
Ask which methodology the provider follows and how it shows up in the report. The most widely used references are NIST SP 800-115 (planning, discovery, attack and reporting), the OWASP Web Security Testing Guide for web applications, ISECOM's OSSTMM, focused on operational security measurement, and PTES (Penetration Testing Execution Standard), which structures the work from pre-engagement to reporting. Methodology is not red tape: it is what ensures that two tests of the same environment are comparable and that nothing important is skipped by oversight.
Assess who will do the testing
- Real manual testing. Ask how the work is split between tools and manual analysis. Reformatted scanner output is not a pentest.
- Team qualifications. Certifications with a hands-on exam, such as OSCP, say more about the ability to run a test than purely theoretical ones.
- Experience with your scope. An internal network pentest requires solid Active Directory knowledge; an application pentest requires solid knowledge of web development and APIs.
- Sample deliverables. Ask for an anonymized sample report and, if the test will be used in an audit, a template attestation letter.
What the report should include
A good pentest report includes:
- Executive summary. In business language: what an attacker could achieve, which critical assets are at risk and what to fix first.
- Scope and method. What was tested, when, how, and what was left out.
- Findings with evidence. Each flaw with a description, steps to reproduce, evidence (screenshots, requests), affected assets and a risk rating in the context of the environment.
- Attack chains. How medium-severity findings combine into a serious compromise.
- Actionable recommendations. Specific fixes, not generic phrases like "apply patches".
Insist on a retest. A pentest without a retest ends with a list, not with reduced risk. The retest confirms that the fixes work and did not open other paths. PCI DSS (requirement 11.4.4) requires exploitable vulnerabilities found during penetration testing to be corrected and testing to be repeated to verify the corrections.
How this fits into continuous vulnerability management
Treating "scan or pentest" as a one-off decision is the underlying mistake. Both are inputs to the same cycle: identify, assess, prioritize, remediate and verify. Scanning feeds that cycle with volume and frequency. Pentesting feeds it with depth and with confirmation of which flaws are actually exploitable in your environment, which is valuable data for prioritization. To order the remediation queue, it pays to combine severity, likelihood of exploitation and exposure, as we discussed in how to prioritize vulnerabilities with CVSS, EPSS and KEV.
The flow also runs the other way. A pentest finding that a scanner missed points to a coverage gap: missing credentials in the scan, an asset outside the inventory, a class of flaw that needs a different control. Closing them improves the whole cycle.
Finally, the results need to reach governance: ISO 27001, PCI DSS and LGPD audits increasingly ask for practical evidence that controls work, and a history of scans, pentests, fixes and retests is that evidence.
Frequently asked questions
Does a vulnerability scan replace a pentest?
No. A scan identifies known flaws at scale, but it does not validate whether they are exploitable, nor does it find logic flaws, authorization flaws or attack chains. A pentest does, but it covers a narrower scope at specific points in time. The two complement each other.
How often should we run a pentest?
A common market benchmark is PCI DSS: at least once every 12 months and after any significant change to the environment. Critical applications that change often may justify testing closer to releases. Scanning, being automated, should run far more often.
Which approach should we choose: black, gray or white box?
It depends on the question. To learn what an outside attacker sees, black box. For applications with authenticated users, gray box usually strikes the best balance. To find as many flaws as possible in a critical system in a short time, white box.
Sources consulted: NIST SP 800-115, Technical Guide to Information Security Testing and Assessment; OWASP Web Security Testing Guide; ISECOM, Open Source Security Testing Methodology Manual (OSSTMM 3); Penetration Testing Execution Standard (PTES); PCI DSS v4.0.1, requirements 11.3 and 11.4 (vulnerability scans and penetration testing). This article is educational; check the official text of the standards before defining the scope and frequency of your tests.