Authorised testing. Evidence that matters.Atlant Security
Pentest/ServicesBY ATLANT SECURITY
Build your scope Free brief builder

Testing methodology

Security Testing Standards: Turn WSTG, ASVS, NIST and PTES into Evidence

Compare WSTG, ASVS, NIST SP 800-115 and PTES by purpose. Build a requirement-to-evidence matrix and avoid vague methodology claims in pentest proposals.

Discuss your requirements
Technical reference volumes arranged in an architectural library

Choose security testing standards by the question they answer

Security testing standards are useful when they make scope, methods and evidence more precise. They are less useful when a proposal lists familiar acronyms without explaining what will actually be verified. A buyer needs to know which requirements apply, which techniques will be used, what access is necessary and how untested items will be reported. The framework name alone answers none of those questions.

OWASP ASVS, OWASP WSTG, NIST SP 800-115 and PTES are often mentioned in penetration testing discussions, but they do not serve identical purposes. Treat them as complementary references rather than interchangeable certifications. This guide explains how to turn methodology language into a practical statement of work and evidence matrix. The example matrices are original procurement tools, not official reproductions of any standard.

If you are commissioning penetration testing services, begin with the system and decision you need to support. Releasing a multi-tenant application, evaluating an internal network and checking a corrected authorisation flaw require different evidence. A reference becomes valuable when it helps define that evidence and the limits of the resulting conclusion.

Separate requirements, techniques, engagement process and obligations

A requirement states a security property you expect the system to satisfy. A testing technique describes how to investigate that property. An engagement process governs authorisation, preparation, execution and reporting. A contractual or regulatory obligation states what a particular organisation must do. Confusing these layers produces scopes that sound comprehensive but cannot be accepted objectively.

For example, an application may be required to enforce object ownership on the server. A technique may compare requests from two approved users against separate synthetic objects. The engagement process determines which environment and accounts may be used. A customer contract may require independent testing before release. These statements relate to each other, but satisfying one does not automatically satisfy the others.

Write the layers separately in the scope. Ask the provider to map the relevant requirements to planned techniques and evidence. Ask your own legal, compliance or assurance owner to interpret applicable obligations. A technical assessor can supply useful observations without claiming to deliver a statutory opinion or a universal compliance certificate.

LayerQuestionExample evidence
Security requirementWhat must be true?A defined authorisation property
Test methodHow will it be examined?Paired permitted and forbidden requests
Engagement processUnder what permission and constraints?Signed rules and agreed contacts
Applicable obligationWhy is this evidence needed?Contract or requirement interpreted by its owner

Use ASVS to make application requirements explicit

The OWASP Application Security Verification Standard provides application security requirements. Its stable release at this review date is 5.0.0. Use a versioned reference when specifying a requirement because numbering and content can change between releases. An unversioned identifier can become ambiguous when a report is reviewed months later.

The practical procurement question is which applicable requirements will be evaluated and by what evidence. Some checks can be investigated through external behaviour; others need design information, configuration or source access. If the provider receives only an unauthenticated URL and a short testing window, do not assume that every selected requirement can be conclusively evaluated. Mark access dependencies before the engagement begins.

Create an applicability register. Record the requirement reference, why it applies, the component, evidence method and result category. Where a requirement is not applicable, give a reason tied to architecture rather than convenience. Where evidence is unavailable, use an unverified status. These distinctions make a requirements-based review useful without pretending that a spreadsheet full of ticks is itself assurance.

Requirements should connect to identifiable verification evidence.
Requirements should connect to identifiable verification evidence. Conceptual illustration.

Use WSTG to discuss application testing techniques

The OWASP Web Security Testing Guide is a practical reference for investigating web application security. Its stable guide includes areas such as authentication, authorisation, session management and business logic. In procurement, use it to make the intended testing approach more concrete, then tailor that approach to the application’s actual workflows and architecture.

A statement that testing follows WSTG should be followed by a coverage explanation. Which application roles, interfaces, workflows and relevant test areas are included? Which are excluded because the application does not use them? Which are constrained by access or time? The same named technique can produce very different depth when applied to one anonymous page or a complex authenticated multi-tenant workflow.

Ask how exploratory testing will supplement the planned checks. A useful methodology guides investigation without turning it into a rigid script that ignores what the tester learns. If a shared permission service appears to affect several endpoints, the assessor should be able to adjust the test plan within the agreed scope. Document material changes and the reason for them.

Use NIST SP 800-115 for assessment planning context

NIST SP 800-115 is a technical guide to security testing and assessment, published in 2008. It discusses planning, conducting assessments, analysing findings and mitigation. Its age should be visible when it is cited; it is not a current catalogue of every cloud or application technology. Use it for assessment structure while using appropriate current technical references for the target systems.

A useful statement of work can draw on that planning discipline without claiming NIST certification. Name the objective, scope, permissions, constraints, evidence handling and reporting expectations. Explain the limitations of the chosen techniques. A vulnerability scan, configuration review and penetration test can all contribute evidence, but they do not establish the same things.

For a modern cloud application, combine the process with target-specific knowledge. Identity federation, service accounts, managed services and application workflows require evidence that a general assessment guide cannot supply by itself. The provider should explain how the chosen methods fit the architecture rather than relying on a broad reference to make the scope appear complete.

Technical techniques need an authorised and controlled assessment environment.
Technical techniques need an authorised and controlled assessment environment. Conceptual illustration.

Treat PTES as an engagement-methodology reference, not a badge

PTES, the Penetration Testing Execution Standard, is commonly used as a reference for structuring penetration testing engagements. If a provider cites it, ask which parts of its engagement process and deliverables the reference is intended to describe. Do not treat the acronym as proof of an independent accreditation, a particular tester qualification or coverage of every system in your organisation.

The useful buying questions remain concrete: how will scope and permissions be agreed, how will the target be understood, how will testing be controlled, how will findings be validated, and what will the report contain? Ask for an anonymised example that demonstrates the proposed evidence quality. A methodology statement should make those answers easier to evaluate, not replace them.

Record the exact source or edition the provider uses in its proposal. If an online methodology is unavailable or changes, retain the agreed reference with the project documents. This is especially helpful for long procurement cycles. The acceptance decision should depend on the agreed work and evidence, not on whether a logo or methodology name appears on the report cover.

Compare references without flattening their differences

A concise comparison helps procurement teams ask the right follow-up questions. It should not imply that choosing one reference excludes all others. An application assessment may use requirements to define expected controls, testing guidance to investigate behaviour and an engagement process to govern execution. The provider should explain that combination in plain language.

The table below is a buying aid, not an official crosswalk. It deliberately avoids equating a requirement identifier with a test case. One requirement can need several observations, and one test can reveal evidence relevant to several requirements. The mapping must be justified for the particular system and should remain reviewable after the engagement.

ReferenceUseful emphasisQuestion for the proposal
OWASP ASVSApplication security requirementsWhich versioned requirements are applicable and verifiable?
OWASP WSTGWeb application testing guidanceWhich techniques, roles and workflows will be covered?
NIST SP 800-115Technical assessment planning and methodsHow are objectives, constraints and limitations documented?
PTESPenetration testing engagement methodologyWhat process and report evidence does the reference commit to?

Do not score a proposal higher simply because it mentions more references. A focused, traceable plan can be more useful than a page of acronyms. Ask the assessor to demonstrate the relationship between the target architecture and the proposed coverage. If that relationship is unclear, request a revised scope before comparing prices.

Build a requirement-to-evidence matrix

Start the matrix with business-sensitive capabilities, such as tenant administration, document export, payment approval or account recovery. For each capability, identify the security property that matters and the relevant reference. Then specify an evidence method, prerequisites and an acceptance condition. This reverses the common habit of selecting a long checklist first and trying to fit the application into it afterwards.

For example, a document export capability may require that only authorised users receive records from their permitted tenant. Evidence might include an architectural explanation, paired requests using two synthetic tenants and an inspection of the resulting export. The prerequisite is not simply a URL; it includes suitable accounts, synthetic records and visibility into the asynchronous job result.

Add a result vocabulary before testing: verified, failed, not applicable and not verified are useful starting points. Explain partial coverage separately. If only one of three export mechanisms is assessed, the result should not appear as an unconditional pass for the entire capability. A short coverage note prevents later readers from overextending the conclusion.

Compare the required security property with observed software behaviour.
Compare the required security property with observed software behaviour. Conceptual illustration.

Match access to the claims you want the report to support

Black-box, authenticated and source-assisted assessments provide different visibility. None is universally best. An unauthenticated external review can investigate the exposed attack surface, while an authenticated review can examine role boundaries and business workflows. Source-assisted work can help assess implementation choices that are difficult to infer from responses alone. Choose access based on the questions you need answered.

Document which accounts, roles and tenants will be supplied. For a multi-tenant service, one administrator account in one tenant may be insufficient to test cross-tenant boundaries. For delegated access, the assessor may need an owner, a delegate and a revoked delegate fixture. These are scoping inputs, not administrative trivia. Missing them can make a claimed methodology impossible to execute meaningfully.

Limit privileges to what the agreed evidence requires. More access is not automatically better, and access should be time bounded, attributable and removable. Where direct access is inappropriate, supervised exports or owner-run collection may be suitable. State how those alternatives affect independence and verification. A transparent limitation is more useful than an unsupported assurance claim.

Worked example: a multi-tenant procurement portal

Imagine a fictional procurement portal with supplier accounts, buyer accounts, tenant administrators and a background document export service. The buyer wants evidence that one organisation cannot read another organisation’s documents, that supplier changes require the right authority and that exports preserve the same boundaries as interactive screens. These objectives are more precise than asking for an OWASP test.

The scope names two synthetic tenants, representative roles, the web interface, API and export workflow. The matrix links expected authorisation properties to the selected requirements and techniques. The assessor receives synthetic documents, an architecture discussion and controlled visibility into export completion. Testing includes permitted actions as well as forbidden cross-tenant requests, with evidence tied to the target release.

If the export worker is unavailable during the engagement, the report records that dependency as not verified. It does not mark the whole authorisation workstream passed because the web interface behaved correctly. If the API reveals a separate permission gap, the finding identifies the affected operation and the evidence. The framework references help organise the work, while the actual observations support the conclusion.

CapabilityAcceptance questionNecessary fixture
Document readCan only permitted identities retrieve the object?Owned and unowned synthetic documents
Supplier changeDoes the caller have the required authority?Separate supplier and buyer roles
Background exportAre tenant boundaries preserved after processing?Controlled export job and result access
Role removalDoes withdrawn access cease on relevant paths?Revoked delegate and active session

Make sampling and time limits explicit

Most assessments operate under finite time and access constraints. Sampling can be reasonable, but the selection logic must be visible. Group similar components by implementation, privilege and business consequence. Explain why selected examples are representative and where that assumption could fail. A shared code path may justify a sample; merely sharing a visual design does not.

Ask how the provider allocates effort to complex workflows. A site with thousands of repeated product pages may be simpler than a small portal with intricate delegated approvals. Counting URLs alone is a weak measure of security testing effort. Roles, trust boundaries, state transitions and integrations often determine the work more directly.

At the end, request a coverage record that separates planned, completed, constrained and excluded work. If time was consumed by a significant finding, explain what was not reached. That is useful information for the next assessment and for release risk decisions. A report that hides incomplete work behind a methodology label is difficult to use responsibly.

Scope boundaries should be visible before and after testing.
Scope boundaries should be visible before and after testing. Conceptual illustration.

Do not substitute tool output for verified coverage

Automated tools can efficiently collect evidence and identify candidate issues. They do not by themselves prove that a complete application requirement set has been evaluated. A scanner may not understand the business meaning of a cancellation, the relationship between two tenants or whether a delegated role has expired. The provider should explain where automation helps and where human investigation is required.

Ask how candidate findings are validated. A useful report distinguishes confirmed behaviour from tool warnings and records the conditions needed to reproduce the issue safely. False positives waste engineering time, while unexamined assumptions can conceal real weaknesses. Validation is part of the service, not an optional cosmetic step performed only when the buyer challenges a finding.

Similarly, a clean scan does not establish that all selected requirements passed. The report should say what the tool could observe, what authentication it used and which manual work supplemented it. If the service is primarily automated scanning, label it accurately. Buyers can then decide whether that evidence meets their objective or whether a deeper assessment is needed.

Specify the report before buying the test

Request a report structure that includes scope, environment, dates, methods, coverage, findings, limitations and recommended next actions. Each finding should explain the observed condition, affected component, business consequence and practical correction. Technical evidence must be sufficient for the engineering team to understand and reproduce the issue through authorised means, without exposing unnecessary sensitive information.

Ask for a separate coverage appendix or matrix if the assessment is requirements based. The finding list alone cannot show which controls were examined and found satisfactory, which were not applicable and which could not be verified. This distinction is especially important when a customer or assurance team later asks what the engagement actually covered.

Agree how retesting will be documented. A corrected finding needs a dated verification result tied to a release and environment. The original report should remain traceable. Our pentest retest acceptance guide explains positive controls, nearby variants and residual-risk statuses that make closure evidence more useful.

Versioned records keep requirements, observations and corrections traceable.
Versioned records keep requirements, observations and corrections traceable. Conceptual illustration.

Questions that expose vague methodology claims

Ask the provider to show one example mapping from a business capability to a security requirement, a test method and a report observation. This small exercise is often more informative than a long capability presentation. It reveals whether the team understands the difference between a framework reference and the evidence needed to support a conclusion.

Ask what would cause an item to be reported as not verified. Strong answers identify access, environment, time and evidence constraints. Weak answers imply that every checklist item will pass or fail regardless of visibility. Ask who will perform the work, how findings are reviewed and how the team handles unexpected operational impact. These details affect the reliability of the result.

Finally, ask what the proposed engagement will not establish. A useful provider can explain that a point-in-time application test does not guarantee future security, cover unlisted systems or replace secure development. That explanation should be specific to the proposed work. The goal is a well-defined decision aid, not an impossible promise.

Write acceptance language that can be checked

A practical acceptance clause can require delivery of the agreed coverage matrix, evidence-backed findings, an explicit list of limitations and a remediation discussion. It can require versioned references and a record of material scope changes. Avoid language that demands the application be declared secure regardless of findings; that creates the wrong incentive and an impossible technical standard.

Define how disagreements are resolved. If a developer believes a result is a false positive, the assessor should review the evidence and the intended behaviour with the relevant owner. The record should preserve the final rationale. A disputed severity can be discussed separately from whether the underlying behaviour occurred. This keeps the conversation grounded in evidence instead of competing labels.

State how sensitive material will be exchanged, retained and removed. Reports, raw requests and architecture documents can expose valuable information about the environment. Access to those records should be deliberate. The procurement process should make evidence usable without requiring it to be distributed indiscriminately across email chains or unrestricted collaboration spaces.

Procurement becomes easier when acceptance is defined in observable terms.
Procurement becomes easier when acceptance is defined in observable terms. Conceptual illustration.

Turn the reference list into an actionable scope

Before requesting quotes, write down your main systems, roles, integrations, target release and the decisions the test should support. Identify whether you need application testing, API testing, internal infrastructure testing or a combination. Share the relevant architecture at an appropriate level and identify constraints early. This helps providers propose comparable work instead of guessing from a domain name.

Use our penetration testing scoping brief to organise those inputs. Ask for a proposal that explains requirements, techniques and evidence separately. For application work, discuss web application testing and API testing boundaries explicitly when both are relevant.

Reviewed 25 October 2026. Verify the selected editions when commissioning work; framework versions and technical guidance change. Contact Atlant Security to define a test plan with clear acceptance evidence. The outcome should be a report your engineers and decision makers can act on, with methodology references that clarify the work rather than decorate it.

Published by Atlant Security. Sources, editorial policy and corrections.

PUT THE GUIDANCE TO WORK

Choose your next step.

LET’S START A CONVERSATION

Define the scope.
Take the next step.

Your systems, operating constraints and security objectives. A clear starting point for the test.

Discuss your pentest