# Scoring, verdicts and priorities

The overall score only ranks where to look; the verdict per output decides what can be used. The constants below live at the top of [../scripts/score_audit.py](../scripts/score_audit.py). Change both together and regenerate [../assets/regression-expected.json](../assets/regression-expected.json).

## Statuses

- PASS: evidence seen and correct. Full credit.
- WARN: a plausible defect that needs a decision, more evidence, or a minor deviation. Half credit.
- FAIL: evidence seen and wrong, or data unsafe for its use, unless a dated decision in `decisions.md` accepts it. No credit.
- NOT CHECKED: applicable but not verified (no access, no export, zero denominator). Lowers coverage, never the score. Needs the reason and the evidence that would close it.
- N/A: does not apply (for example no custom API jobs). Removed from score and coverage. Needs a reason.

## From rows to checks

- Copy the header of [../assets/findings-register.csv](../assets/findings-register.csv) into `findings.csv` in the run folder; never edit the skill's own asset. score_audit.py stops with exit status 2 and the line number when a value still holds template text in angle brackets.
- Record one row per sub-finding, tagged with the output IDs it affects (separated by semicolons) or `all`. Two rows of the right shape (the values are illustrations, not a template to copy):

```csv
finding_id,check_id,status,outputs,value,threshold,threshold_source,evidence,owner,due,decision_ref,notes
F01,FRS-03,FAIL,O1;O5,"3 of 84 account-days missing; platform value 6,900.00 USD (6.5%)",FAIL at 0.5% of platform value or 1% of expected slices,default,Q04; reconcile_exports.py missing_in_b,Analytics engineer,,,
F02,OPS-08,NOT CHECKED,all,,,,,Analytics engineer,,,Dataset access lists unavailable to the audit identity; an administrator's export of dataset IAM bindings would close it
```

- Columns: `value` is what was found, with the number; `threshold` the rule applied; `threshold_source` is default, metrics.md or a dated decisions.md entry; `evidence` the query ID or script output; `owner` a role, never a name; `due` a date for every WARN on a blocking check; `decision_ref` the decisions.md entry when an accepted FAIL is recorded as WARN; `notes` cross-references, hand-offs and the reason for every NOT CHECKED or N/A.
- A check's status is the worst of its rows in the order FAIL, WARN, NOT CHECKED, PASS. A check cannot PASS while part of it went unchecked.
- Rows that are all N/A make the check N/A. A check with no row is NOT CHECKED with the reason "no finding recorded".
- An N/A row tagged to one output excludes the check for that output only.

## Weights, caps, bands and coverage

- Impact weights: A 3, B 2, C 1. Credits: PASS 1, WARN 0.5, FAIL 0.
- Domain score = 100 x sum(weight x credit) / sum(weight), over checks rated PASS, WARN or FAIL in the domain.
- Any class A FAIL caps its domain at 60.
- Domain coverage = rated weight / applicable weight (every check except N/A).
- Domain weights: FRS 20, KEY 15, IDN 10, MET 15, REC 20, CHN 10, OPS 10.
- Overall score = sum of (domain weight x capped domain score) over domains with at least one rated check, divided by the sum of those weights. Computed unrounded, then rounded half up to a whole number; domain scores display to one decimal, half up.
- Overall coverage = rated weight / applicable weight across all checks.
- Coverage labels: 80% or more graded; 60% to under 80% provisional (the score carries the word); under 60% insufficient evidence (score and band withheld, findings by domain only).
- Bands: Healthy 85 to 100; Needs work 70 to 84; At risk 50 to 69; Failing below 50.

Why not one averaged score: a weighted average of completeness, consistency and timeliness lets a high average hide a failed key, currency or reconciliation check. Blocking checks per output make the condition for use explicit.

## Output types and blocking checks

- paid_media_performance (spend, clicks, CPC, CPA or CAC, ROAS by platform, campaign or channel): FRS-02, FRS-03, FRS-05, KEY-01, KEY-08, MET-01, MET-07, REC-02, REC-03, REC-04, CHN-06, CHN-07.
- pipeline_and_lead_source (leads, MQL or SQL counts, pipeline and deals by source or campaign): FRS-02, FRS-03, KEY-01, KEY-03, KEY-04, KEY-06, IDN-05, MET-01, MET-02, REC-05, CHN-08.
- revenue_attribution (billed or collected revenue, new MRR or ROAS on business revenue by channel): FRS-02, KEY-01, KEY-04, KEY-05, KEY-08, IDN-03, IDN-04, MET-01, MET-05, REC-06, CHN-07, CHN-08.
- activation_audience_or_upload (warehouse audiences synced to ad platforms or the CRM, offline or enhanced conversion uploads): FRS-02, FRS-04, KEY-01, KEY-02, IDN-01, IDN-04, IDN-05, REC-08 (what the destination accepted, and consent), OPS-04, OPS-08.
- executive_kpi (blended CAC, total conversions, revenue and similar headline numbers): KEY-08, MET-01, MET-02, MET-03, MET-04, REC-07, REC-08.

`extra_blocking` in the config adds checks to one output, with a reason. Nothing can remove a blocking check.

## Verdict rules

For each output, take the worst row status per blocking check among rows tagged with the output or `all`.

1. Any blocking FAIL: "Do not use until fixed", listing the FAIL checks. Blocking NOT CHECKED items are listed as "also not assessed".
2. Else any blocking NOT CHECKED, or a blocking check with no row for the output: "Not assessable", listing what is missing.
3. Else any blocking WARN: "Use with caveats", listing the caveats.
4. Else "Decision-grade".

Non-blocking FAIL and WARN rows tagged to an output are listed under it and never change the verdict. A FAIL becomes WARN only with a dated decision from `decisions.md`, cited in the row's `decision_ref`.

## One root cause, one check

When one root cause shows in two checks, score it in the check named first below, reference it from the other, and evaluate the other on what remains.

- Missing slices score in FRS-03, not REC-02 or REC-03. REC checks compare matched keys only.
- Stale restatements beyond the reload window score in FRS-05; REC-03 compares only the days inside the reload window at the last successful sync, against a platform export pulled after that sync.
- Currency and unit errors score in KEY-08; REC-02 notes them and compares values on matched keys.
- Duplicates at the declared grain score in KEY-01; fan-out introduced by a join scores in KEY-04.
- Two implementations of one metric that differ score in MET-02, including CAC built on different numerators under one name; REC-08 covers cache, filter scope and blends against the tile's declared source, and what an upload's destination accepted; the cost-to-finance residual scores in REC-07.
- A wrong platform column or overlapping action types score in MET-07, even when REC-03 shows the gap.
- Click ID loss scores in IDN-01, not REC-07.
- Unknown coalesced to Direct in warehouse logic scores in CHN-07; overwritten CRM source fields score in CHN-08.
- A weak freshness signal scores in FRS-04; a check schedule that is too sparse scores in OPS-03.
- Confirmed collection defects in analytics outcomes score in REC-04; REC-07 never fails on by-design differences.

## Priorities

- P1: a FAIL row on a blocking check of an output in use (`in_use` true).
- P2: every other FAIL row.
- P3: a WARN row on a blocking check of an output it applies to.
- P4: every other WARN row.
- Within a priority, order the fix list by value at stake (spend or revenue affected), then the number of outputs affected, then impact class. The script sorts by finding ID; the ordering by value is the agent's job in the fix list.
- An activation or upload output with a P1 item gets a "pause until fixed" recommendation for its owner. The agent never pauses anything.

## Overrides

- `threshold_overrides` in the config: check ID, the threshold used and a reason. They are printed in the report. Agreed values in `metrics.md` or `decisions.md` count as overrides too; cite them in `threshold_source`.
- `domain_weights` with `domain_weights_reason`: printed as a warning, so readers know the score is not comparable with a default run.
- Overrides cannot remove a blocking check, change a status without evidence or turn NOT CHECKED into PASS.

## Worked computation (Acme Cloud, 2026-11-01)

Check statuses from 62 rows: 11 PASS, 25 WARN, 14 FAIL, 1 N/A, 1 NOT CHECKED.

- FRS: weights 16; earned 1 (FRS-01 WARN, B) + 3 (FRS-02 PASS, A) + 0 + 0 + 0 + 1 (FRS-06 WARN, B) + 0.5 (FRS-07 WARN, C) = 5.5; 34.375, shown 34.4. The class A FAILs cap it at 60, which does not bind.
- KEY: weights 21; earned 3 + 1.5 + 1 + 0 + 2 + 0 + 1 + 0 = 8.5; 40.476, shown 40.5.
- IDN: weights 15; earned 1 + 2 + 1.5 + 3 + 0 + 2 = 9.5; 63.333, capped at 60 by the IDN-05 class A FAIL.
- MET: weights 17; earned 1 + 0 + 3 + 0 + 1 + 0.5 + 0 = 5.5; 32.353, shown 32.4.
- REC: weights 20; earned 1 + 1.5 + 3 + 1.5 + 2 + 1.5 + 1 + 1 = 12.5; 62.5.
- CHN: weights 16; earned 1 + 1 + 2 + 0.5 + 0 + 1.5 + 0 + 1 = 7; 43.75, shown 43.8.
- OPS: rated weights 13 (OPS-07 N/A, OPS-08 NOT CHECKED); earned 0 + 1 + 2 + 0 + 1 + 1 = 5; 38.462, shown 38.5; coverage 13 of 16 = 81.3%.
- Overall: (20 x 34.375 + 15 x 40.476 + 10 x 60 + 15 x 32.353 + 20 x 62.5 + 10 x 43.75 + 10 x 38.462) / 100 = 44.52, rounded to 45, band Failing. Coverage 118 of 121 = 97.5%, graded.
- Priorities: P1 11, P2 4, P3 12, P4 15. The REC-08 WARN on the qualified-demo upload (F62) is a P3 because REC-08 blocks activation outputs; the REC-08 WARN on Paid media daily (F43) is a P4 because it does not block that output type.