# Final Report and Codex Review Directives

Use this guidance when assembling the final Codex Security markdown report and Codex app review directives.

## Final Outputs

The final outputs are:

- primary readable report at the final HTML scan report path from `scan-artifacts.md`
- markdown report source used to generate the HTML report, at the final scan report path from `scan-artifacts.md`

Use `report.html` as the primary readable entry point. Explain report-relevant artifact paths in the report itself, especially in `Scope`, `Reviewed Surfaces`, and `Open Questions And Follow Up`.

Every scan mode uses the same final report pipeline: repository-wide scans, scoped-path scans, diff scans, commit scans, branch-diff scans, working-tree scans, and Deep Security Scan all write the same `report.md` structure and render the same `report.html` template. Do not invent a scan-mode-specific report shape.

After writing `report.md`, run the deterministic report validator:

`python3 <plugin_dir>/scripts/validate_report_format.py --report-md <scan_dir>/report.md`

If validation fails, record the validator output in `<scan_dir>/report_validation.md`, treat each error as a concrete repair checklist, repair `report.md` from existing scan artifacts, and rerun the validator before rendering HTML. Keep passing the same `report.md` path back to the validator after each repair until it passes or reaches a terminal blocker. Do not remove required fields, required subsections, or findings merely to satisfy the validator; populate missing values from the discovery, validation, attack-path, and coverage artifacts, or record the exact missing proof as the field value when the final-report format allows uncertainty. If the validation failure cannot be repaired from existing scan artifacts, record a terminal report-generation blocker in `<scan_dir>/report_validation.md` and do not silently switch to a different report format.

Generate the HTML report from the validated markdown report with:

`python3 <plugin_dir>/scripts/render_report_html.py --template <plugin_dir>/assets/report_template_inlined.html --report-md <scan_dir>/report.md --report-html <scan_dir>/report.html --title "<repo_name> Codex Security Scan"`

If the validator, renderer, or template is missing, repair the skill resource path or record a terminal report-generation blocker in `<scan_dir>/report_validation.md`. Do not silently substitute another validator, renderer, or HTML template.

When there are no reportable findings, include a short `No findings` section that explains why nothing survived discovery or the later reportability gates. For repository-wide and scoped-path scans with a coverage ledger, still include `Reviewed Surfaces` so checked, rejected, not-applicable, and follow-up-needed surfaces remain auditable.

When there are reportable findings, render them as readable markdown findings rather than raw JSON or a dumped schema object.
Order findings from highest severity to lowest severity: `critical`, then `high`, then `medium`, then `low`.

Use a separate finding entry for each independently attackable source/control/sink instance. Do not combine sibling routes, templates, query builders, parser operations, auth/object-access endpoints, or shared-helper callers into one representative finding solely for readability; if grouping helps, add a short grouped summary after the individual finding entries.

If validation or attack-path analysis provides a broad family row with multiple independently triggerable sink, parser, helper, API-mode, or protected-action lines, split it into child final findings before writing the report. Multiple affected lines inside one finding are appropriate for one inseparable proof tuple, such as a wrapper plus its shared sink, but not as a substitute for separate findings when sibling operations can be triggered independently.

Set the finding category and CWE from the primary broken control. Do not add secondary support-impact CWEs, such as data exposure or missing authentication, to an injection/RCE/path/file/parser finding merely because they make exploitation worse; mention those impacts in prose or emit a separate finding if that secondary control is independently vulnerable.

Examples that should normally become separate final findings include SQL API modes such as `execute`, `executemany`, and `executescript`; deserializer variants such as `pickle.load`, `pickle.loads`, `yaml.load`, and `yaml.load_all`; distinct path/file helper calls; SSRF modes with different destination controls; and missing-auth protected actions such as create, delete, reset, admin, and job-trigger endpoints.

Before writing the report, reconcile each final finding against its candidate-ledger path from `scan-artifacts.md`, the saved validation closure table, and the repository coverage ledger when those artifacts exist. Every final candidate finding must have discovery, validation, and attack-path receipts for the same candidate id, or an explicit follow-up-needed reason for the missing proof. Start from validated rows marked `reportable` or `survives: yes`, not only from the most polished candidate narrative. Every `reportable` seeded or root-control ledger row must become a finding with the same root-control file:line. Rows closed as `suppressed`, `not_applicable`, or `deferred` should appear in `Reviewed Surfaces` using public-facing outcomes such as `Rejected`, `Not applicable`, or `Needs follow-up`. Do not silently drop a seeded/root-control row because a same-family neighboring finding survived. If attack-path analysis omitted a reportable validation row, assemble a concise attack path from the validation evidence and threat model rather than dropping the row.

## Report Structure

Use this report structure:

`# Security Review: <repo_or_target_name>`

`## Scope`

List in-scope code, scan mode, artifacts reviewed, runtime or test status, and explicit exclusions or limitations. Include the generated-or-provided context used by the scan. If the threat model was generated during Phase 1 rather than provided by the user, say that clearly. Do not call generated threat-model material an external input.

After the scope bullets, include a compact `### Scan Summary` table when the scan has findings or repository-wide coverage. Use columns `Field` and `Value`. Include the count of reportable findings, severity mix, confidence mix, coverage, and validation mode when those values are known. Keep artifact paths below this table.

`## Threat Model`

Copy the scan threat model into this section from `<context_dir>/threat_model.md`. If it is long, keep its headings and concise bullets, and remove only duplicated prose that is already covered by `Scope`. Do not replace the threat model with separate product-context or invariants sections.

`## Findings`

Start this section with the findings summary table.

After the summary table, include a compact `### Confidence Scale` table with columns `Label` and `Meaning`:

- `high`: direct source, configuration, or runtime evidence supports the finding, with no material unresolved reachability or exploitability blocker.
- `medium`: source evidence supports a plausible issue, but runtime behavior, deployment configuration, role reachability, type constraints, or exploit reliability still need proof.
- `low`: weak or incomplete evidence; include only when the user explicitly wants follow-up candidates in the final report.

Then render each finding as:

`### [<number>] <title>`

For each finding include a compact two-column metadata table immediately below the heading. Use columns `Field` and `Value`. Include these rows:

- `Severity`: `critical|high|medium|low`
- `Confidence`: `high|medium|low` or a short calibrated confidence label
- `Confidence rationale`: one sentence explaining why the confidence label is calibrated that way, grounded in the validation method, direct evidence, and missing proof if any
- `Category`: concrete vulnerability class
- `CWE`: id and name list, or `none`
- `Affected lines`: path:line-range list

Use a concrete category such as `Authorization bypass / IDOR`, `Path traversal`, `SQL injection`, `XXE`, `Open redirect`, or `Hardcoded credentials`. Do not use generic placeholders such as `security scan finding`.

The summary table should link each finding title to its detailed finding section with an intra-document markdown anchor. Keep the link text identical to the detailed heading title, without the numeric prefix. Use the detailed heading slug generated from `[<number>] <title>`; for example, link finding 1 to `#1-example-title`.
The summary table and the detailed finding sections must use the same descending severity order: all `critical` findings first, then `high`, then `medium`, then `low`. Renumber findings after sorting so the table order, detailed headings, and anchors match.

Affected lines must include the root broken control or dangerous sink line when that line is identifiable, not only the public wrapper, route, or caller that makes it reachable. For wrapper-to-shared-helper findings, list both the reachable wrapper/entrypoint and the underlying parser, deserializer, path/archive helper, expression evaluator, or auth/authz control line. If a seeded file, class, package, or hunk shares the surviving proof tuple, keep that seed anchor in affected lines instead of replacing it with a broader sibling-only location. If the bug is caused by unsafe transformation or selection before the sink, include the split, parse, canonicalization, normalization, comparison, regex, object-selection, or object-binding line where the control fails. For parser, XML, deserialization, and object-construction findings, include the concrete codec, converter, deserializer, parser feature setup, resolver, class filter, or container handler line when that line performs recursive parsing, type resolution, object conversion, class filtering, or fail-open hardening. For central file-format object models, include low-level helper lines such as `to*Array`, `toList`, `getObject`, numeric conversion, iterator, size-based allocation, unchecked cast, or collection-to-array loops when those helpers are the broken malformed-input control. For recursive placeholder/template findings, include the helper/parser setup line that enables recursive expansion or expression evaluation, not only the later resolver or render call. For resource-serving findings, include the allowlist, path-matcher, URL decoding, canonicalization, or resource-selection line that decides whether the attacker-selected resource is allowed. For stateful authentication protocol findings, include the principal/credential/token/issuer installation, rebind/reauthentication, or validated-vs-consumed object-selection line that creates the auth bypass. For SSO/SAML/federation findings, include the response/assertion selection, signed-object lookup, cloned/returned assertion, subject, audience, recipient, destination, ACS URL, or issuer-binding line that determines which identity object is trusted. For polymorphic or request-selected handler, operation, converter, filter, validator, or strategy families, include the concrete subclass/implementation line that transforms, validates, canonicalizes, selects, or reinterprets attacker input before a shared sink/control, including specialized helper methods and branch predicates inside the concrete class when they perform or enable the unsafe transform. If a special-case branch such as append, wildcard, fallback, copy/move `from`, default-value, or type-resolution handling bypasses or narrows validation, include that branch-local root-control line even when a shared helper is also affected. If the finding text says a shared flaw affects "all", "every", or "any" concrete operation, codec, converter, handler, validator, filter, or resolver, the affected lines must include the concrete implementations identified during discovery or validation; do not rely on "and related classes" prose for independently reachable root-control lines. If equivalent resolver/filter controls are duplicated across core, server, client, remoting, plugin, or import packages, include the runtime/exported implementation that enforces the broken control. For repeated vulnerable templates, routes, query builders, parser operations, or auth/object-access endpoints, keep each independently vulnerable file and line as its own affected instance; do not hide sibling instances as extra context on one representative finding when they can be attacked independently. The Codex review directive should point at the tightest root-cause line unless the wrapper or concrete implementation line is the actual broken control.

Then render these subsections under each finding:

- `#### Summary`
  - Explain why the issue matters, what the vulnerable path is, and why the current controls are insufficient.
- `#### Validation`
  - Include method, checklist items, evidence, and remaining uncertainty.
- `#### Dataflow`
  - Show the technical source-to-sink path inside the code, such as request parameter -> controller -> service/helper -> dangerous sink -> response or side effect.
- `#### Reachability`
  - Explain who can realistically trigger the dataflow, from what boundary, under what preconditions, and what attacker outcome follows. Fold any attack-path facts into this prose or compact bullets instead of emitting a separate `Attack Path Facts` section.
- `#### Severity`
  - State the final severity and then explain the rationale.
  - Treat likelihood and impact as inputs to the final severity, not as separate report labels.
  - The rationale should fold in reachability: attacker role, exposed entry point, exploit steps, required feature flags/config, runtime/deployment assumptions, and any counterevidence or blockers.
  - The rationale should explain the concrete security consequence using repository evidence: data exposed, integrity boundary broken, credential/control-plane effect, code execution path, or why impact is narrower.
  - Include one concise sentence explaining what specific additional evidence would raise or lower the severity.
  - Avoid circular phrasing such as `this is high because it is high severity`, `maps to high`, or `high-severity issue`.
- `#### Remediation`
  - Give concrete minimal fixes, tests, and preventive controls.

For repository-wide and scoped-path scans with a coverage ledger, include a concise `## Reviewed Surfaces` section after the findings. This section summarizes what was inspected, what came out of each reviewed surface, and seeded/root-control rows that were suppressed, not applicable, or deferred so an auditor can see why they did not become findings. Use a table with `Surface`, `Risk Area`, `Outcome`, and `Notes`.

Recommended outcomes:

- `Reported`: became a final finding.
- `No issue found`: reviewed and no credible issue survived.
- `Rejected`: plausible-looking candidate was ruled out with specific counterevidence.
- `Not applicable`: the risk class does not apply to that surface.
- `Needs follow-up`: plausible but not fully closed because of a concrete blocker or proof gap.

Write the same content, or a slightly more detailed version, to `<coverage_dir>/reviewed_surfaces.md`.

For broad scans where the completed coverage is useful for triage but too large for high-precision review, include a concise `## Open Questions And Follow Up` section near the end of the report. Use concrete, copyable prompt ideas that narrow the next review to individual commits from the current scan. Do not include this section for precise scans where the requested scope was already sufficient.

Follow-up prompts should be tailored to the actual scan results:

- use exact commit SHAs, PR numbers, short titles, file paths, or component names from the report
- focus each prompt on the specific boundary that made the commit worth follow-up, such as auth, plugin/MCP exposure, artifact downloads, signed URLs, or gateway routing
- avoid generic placeholders

Each finding should make it easy for an application security engineer or software engineer to answer:

- what changed or what path is vulnerable
- what attacker-controlled input or trust boundary matters
- what direct evidence supports the claim
- what counterevidence or uncertainty remains
- why the severity landed where it did
- what the smallest safe fix is

Include both final report paths in the response so the user can find the markdown and HTML reports easily.

## Codex Review Directives

For Codex app rendering, emit one `::code-comment{...}` directive per surviving finding in the final response. The markdown report and review directives should agree on title, file, line range, and core explanation.

Map the final report severity to Codex directive priority only when emitting the directive:

- `critical` -> `P0`, `priority=0`
- `high` -> `P1`, `priority=1`
- `medium` -> `P2`, `priority=2`
- `low` -> `P3`, `priority=3`

For each reportable finding, emit a Codex review directive in this form:

`::code-comment{title="[P1] Example title" body="One-paragraph review explanation." file="/absolute/path/to/file" start=10 end=12 priority=1 confidence=0.55}`

Directive requirements:

- `title`, `body`, and `file` are required
- `title` should include the mapped Codex directive priority, formatted like `[P1] Example title`
- `file` should be an absolute path
- `start` and `end` should be tight 1-based line numbers
- `priority` should match the mapped Codex directive priority
- `confidence` should be numeric when available
- emit one directive per finding and none when there are no findings
- inline Markdown code spans are allowed and encouraged for short identifiers, flags, function names, and config keys, such as `git -c`, `--config`, and `diff.external`
- do not put double quote characters inside quoted attribute values, including escaped quotes like `\"`; rewrite quoted command examples without quotes or leave them only in the markdown report