After a red-team exercise, add a retest requirement beside the report's most consequential attack route. An August 2002 investigation found weaknesses in follow-up to FAA airport-security Red Team findings. Corrective actions generally had not produced sustained improvements in screening performance. The next decision needs evidence that the relevant route has changed before someone accepts the remaining exposure.

This article concerns findings from actual adversarial tests. The risk assessment hub considers how such evidence should inform decisions about the activity being protected.

A red-team exercise uses authorized adversarial testing to discover how an attacker could defeat defenses and achieve a specified objective.

FAA red-team findings required further security work

The FAA Red Team conducted covert airport-security penetration tests before September 2001. The Department of Transportation Inspector General's August 16, 2002 investigation found that the team submitted reports for each mission. Headquarters did not track resolution of identified deficiencies. Field units typically lacked headquarters correction letters and carrier replies.

Follow-up testing generally fell outside the team's charter, although some return tests occurred. FAA took corrective actions, including correction letters and fines. The investigation found weaknesses in coordination and feedback. It did not substantiate allegations that officials covered up findings. These were physical airport-security tests, not cybersecurity exercises.

Separate action taken from outcome demonstrated. The distinction matters when reviewing a finished red-team report. A record of work can show who changed a control. Acceptance of the remaining exposure needs evidence about what the changed control now prevents, detects, or contains. The FAA findings illustrate that distinction without establishing how a modern cyber retest should perform.

Consider the handoff in the reader's own exercise. The test team knows how the route worked. A control owner knows what was changed. The person responsible for the operation must decide whether it can continue under the remaining exposure. A useful follow-up record lets each person see the evidence relevant to that decision.

For a proposed follow-up process, retain the original route with the corrective action. Record the starting conditions, the objective reached, and the point where a control failed. Then state what observation would justify closing that finding. Keep the attack route in the closure record. A generic task description such as “improve access controls” leaves the required outcome unclear.

For example, suppose an authorized exercise reached protected records through a supplier account. This is a hypothetical route, not an FAA finding. Changing that account's permissions may be appropriate. Before accepting the remaining exposure, identify whether the same starting access still permits the tested outcome under the changed configuration.

Keep changed test conditions visible. The reviewer also needs to know what the retest did not cover. If it uses a different account, environment, or access level, record the difference. Explain whether it affects the conclusion about the original route. A successful test in a simplified environment may justify another test rather than acceptance of exposure in the real operation.

Make the acceptance decision explicit. A finding can be technically addressed while evidence remains incomplete. Record who will decide whether the evidence is sufficient, what activity that decision permits, and any temporary restriction. This builds on clear decision rights without assigning the control owner authority that belongs elsewhere.

Keep this review focused on the consequences of the route. The question is whether the evidence supports the operation that will continue after the exercise. It is not whether every item in the report has acquired a completion date. Other findings still need owners and follow-up suited to their consequences.

What to do after a red-team exercise: require route retest evidence before accepting the remaining exposure
The original attack route connects a control change to the evidence needed for acceptance.Click to expand

The Inspector General's March 18, 2003 public release recommended procedures linking covert-test results to security improvements and feedback to testers. The proposed checkpoint below applies that follow-up principle to the reader's decision. It does not claim that FAA used this process.

What must a retest demonstrate before the remaining exposure from this attack route can be accepted? Start the Walk →

What does an adversarial exercise establish?

A red-team finding provides evidence that a route worked under the recorded test conditions. That can challenge a written control description or a reassuring system diagram. It can also show how individually plausible controls fail when an adversary combines access, timing, and human responses. Preserve those connections when assigning corrective work.

The exercise has boundaries. Its objectives, permitted techniques, available time, access, and environment constrain what the team can establish. NIST's technical security testing guide describes limits on testing and recommends verification of implemented mitigation. A test report is useful evidence within its scope. It cannot establish that every possible route has been found.

Review which routes matter to the next operating decision. A threat vulnerability assessment can help relate a demonstrated route to the assets and activity being protected. Avoid ranking solely by how dramatic a demonstration looked. Identify the consequence that would make continued operation unacceptable.

Task completion accepted

A permissions change is marked complete. The operation continues without evidence about whether the supplier-account route still reaches protected records.

Route evidence reviewed

The retest checks the same starting access and objective. The decision-maker reviews its result and limits before permitting the operation to continue.

The comparison describes a hypothetical follow-up decision, not a reported test outcome. Define the required result before the retest. If the control should prevent access, test that result. If the accepted arrangement instead depends on detection and response, establish whether they operate in time to protect the stated activity.

A discussion exercise answers different questions. A crisis management exercise can examine decisions and coordination through a scenario. An adversarial route demonstrated against controls needs follow-up evidence about those controls. A team discussion about a completed fix does not reproduce the route that made the finding material.

Evidence needs to fit the consequence. If the route could expose protected information, record whether the retest attempted the relevant access. If the concern is interruption, identify what must continue operating. The reviewer should be able to explain why this evidence supports this decision, rather than borrowing confidence from an unrelated successful test.

A blocked route may support a narrower conclusion than the report summary suggests. Keep the tested objective beside the observed result. If another objective was excluded, record it as untested. This prevents the reviewer from approving broader activity on evidence that covered only one part of the operation.

Retest the route before accepting the remaining exposure

Before closing a consequential finding, agree the retest with the test team and the responsible control owner. Preserve the relevant starting conditions and objective. State the expected result, the evidence to retain, and the limits imposed for safe testing. These instructions should identify the decision the result will support.

NIST's security and privacy control catalog distinguishes control functionality from assurance about the capability provided. For this decision, that distinction means examining both the change and the evidence that it works. A description of the new configuration addresses only part of the question.

Before accepting the exposure

What retest evidence would justify accepting the exposure from this attack route?

A sufficient result depends on what continued operation requires. Set the threshold for the activity being approved. The decision-maker should name the relevant consequence and explain why the observed outcome, within the test's limits, permits proceeding. The answer may require a further test or a temporary restriction rather than immediate acceptance.

Do not treat a blocked original route as proof that all related routes are blocked. Ask whether the corrective change leaves a materially similar route available. The test team can advise which variation would challenge the claimed protection. Keep that work proportionate to the consequence and the decision, rather than demanding an unlimited search.

If the required evidence is unavailable, record the gap and its effect on the decision. Options could include restricting access, reducing the activity, introducing a temporary control, or delaying approval. These are proposed choices to assess. They are not automatic recommendations for every finding. Identify which choice the responsible person authorizes and why.

The Universal Decision-Making Method asks what sufficient certainty requires for the decision being made. Here, the concrete evidence concerns a demonstrated route and a claimed correction. Having enough information to decide means resolving the material question, rather than collecting more reports without a clear use.

Retain the reasons for acceptance. Finally, record when that acceptance must be reviewed. Changes to accounts, supplier access, staffing, or system configuration may alter the conditions tested. Monitoring those conditions gives the owner a reason to reopen the decision. Closure is justified by the evidence and its limits, with responsibility for continued review clearly assigned.

Which conditions would invalidate the evidence used to accept exposure after this red-team exercise?

Work through your decision

No sign-up. Just pick your decision and start.


Grant Purdy is the co-author, with Roger Estall, of Deciding (2020), and the architect of the Universal Decision-Making Method.