---
name: tracking-plan-audit
description: Audits an existing tracking plan against what GA4, Mixpanel, Amplitude, PostHog or Segment actually collect, read-only. Scores the plan as a contract (triggers, consumers, owners, identifiers, typed properties, allowed values, destination names), then finds planned events never seen or stopped, orphan events, spelling variants, duplicate producers, retired events still sent, missing required properties, type mismatches, enum drift and PII. Dependency-free scripts profile a raw export and diff it against the plan; returns a findings report, a reconciled plan draft and an ordered change list. Use when asked for a tracking plan audit, event taxonomy audit, analytics or data quality audit, to check tracking against the spec, to find events not in the plan or undocumented events, to clean up Mixpanel Lexicon, Amplitude Data, Avo, Segment Protocols or RudderStack violations, for schema drift, or when duplicate events appear, counts double, events vanish or property types change after a release.
license: CC-BY-4.0
compatibility: Scripts need Python 3.8 or later and no packages. Works from exports and query results; vendor MCP servers are optional and used read-only.
metadata:
  tier: free
  category: tracking-plans-and-naming
  version: "0.1.0"
  last_reviewed: "2026-11-06"
  status: reviewed
---

# Tracking plan audit

Reads one tracking plan and what production collects, and reports where they disagree as PASS, WARN or FAIL with numbers, then drafts the fixes for people to apply. The audit changes nothing, and a finding is a lead until a payload, an emitter or a system-of-record count confirms it.

## When to use

Use this skill when:

- Someone asks to audit a tracking plan, an event taxonomy or analytics data, or to check tracking against the spec.
- The plan and the dashboards disagree and nobody trusts either.
- A cleanup, migration, renewal or Lexicon or Amplitude Data tidy-up needs a baseline first.
- After a release, counts doubled, an event vanished or a property changed type.
- An agency or a previous owner hands over a plan.
- GA4 and a product analytics tool, often fed by Segment, must agree on one plan.

Use something else when:

- No plan exists, only collected events: run tracking-plan-builder's retrofit path first, then audit its result.
- Only names are in question, a lint or a rename plan: event-naming-convention.
- One number broke and nobody knows which layer failed: tracking-break-map.
- Orders or revenue disagree with analytics order by order: purchase-reconciliation.
- The GTM container itself is suspect: gtm-container-audit, which also finds the tag behind a GA4 duplicate producer.
- A site-wide tag and consent audit: tracking-audit. One GA4 property's settings: ga4-audit.
- Events arrive when consent is denied, or vanish after a CMP change: consent-debugging.
- Users are split or wrongly merged across login: identity-merge-qa (planned).
- Amplitude-only deletion cleanup or Mixpanel Lexicon metadata: Amplitude's own tracking-plan-audit skill (a different skill with the same name, from Amplitude's MCP marketplace) and its taxonomy skill, or Mixpanel's manage-lexicon, connected read-only first.
- Ongoing alerting: this audit proposes monitors; tracking-monitoring-and-incidents is planned.

If a skill named here is not installed, stop at that decision and hand the user the evidence gathered so far.

## Inputs

Read these first if they exist:

- `martech-context/stack.md`: tools, production projects, streams and sources, CDP delivery modes, warehouse, owners.
- `martech-context/tracking-plan/` and its `CHANGELOG.md`: contracts, statuses, versions and dates.
- `naming-rules.md`, `data-dictionary.md`, `metrics.md`: the convention and kept exceptions, shared properties with units and enums, and which metric or workflow reads each event.
- `decisions.md`: accepted exceptions with expiry dates, where the canonical plan lives, the consent model.

Then gather:

- The canonical plan and any copies (sheet, YAML contracts, Avo, Segment Protocols, Amplitude Data, Lexicon), and which one is the source of truth.
- Per tool, the production project or stream, one read-only route (an export file, a query result, or an MCP server limited to read tools) and the time zone its counts use; for GA4, the property tier and the custom definitions.
- The window (default 28 days ending 4 days before the audit, once GA4's daily tables are final) and a 90-day lookback for last-seen dates.
- System-of-record counts for every tier 1 event, same window and exclusions, with their time zone, aggregates only.
- Emitter roots: repositories, GTM web and server containers, CDP sources and functions, GA4 event rules, tool-side transformations.
- Consumers per event (key events, saved reports, cohorts, workflows, Lexicon query counts), and owners and approvers by risk tier.

Never ask for passwords, API secrets, service account keys or raw customer records in chat. The user runs exports; the scripts read local files and print counts, types and redacted enum values, never raw records.

## Procedure

Work in a scratch folder outside the skill folder and outside `martech-context/`. Set `SKILL` to the installed skill folder, run the scripts by full path and write every output to the scratch folder. Nothing under `martech-context/` changes until step 8 is approved.

### 1. Fix the baseline

- Record the plan location, version and last approval date; tools with project placeholders; production only (staging excluded unless the plan scopes it); window, lookback and as-of date; time zone per tool and per reference; GA4 property tier; tier 1 events.
- If more than one copy of the plan exists, name the canonical one and audit the others as observed copies without counts. Copy drift explains enforcement blocks.
- If the project or stream you can read is not the one the user named, stop and say so.

### 2. Normalize the plan into two tabs

- Copy the header rows of [assets/plan-events-template.csv](assets/plan-events-template.csv) and [assets/plan-properties-template.csv](assets/plan-properties-template.csv) into the scratch folder as `plan-events.csv` and `plan-properties.csv`, and fill those; never edit the assets or the source. tracking-plan-builder's sheet tabs work as they are; a browser companion in `delivery_paths` counts as a web source.
- YAML contracts: one events row per contract, one properties row per property, global properties as event `*`, companions as tool overrides in `sources` (`server; ga4:web`).
- Check row counts against the contract count and list fields that had no column. Rules per source format: [references/plan-quality.md](references/plan-quality.md).

### 3. Score the plan as a contract

```bash
python3 "$SKILL/scripts/audit_plan.py" quality --events plan-events.csv --properties plan-properties.csv
```

- Rules Q01 to Q13 print coverage per rule. Never blend them into one score.
- Verdicts: contract; schema only (names and types, no triggers, sources or consumers, as every vendor schema export); names only (Q05 or Q08 fails). Q05, Q07 and Q08 set the drift depth.
- Names only: say in the report which rules could not run.
- Naming: run event-naming-convention's lint_event_names.py, when installed, over each tool's final names and, after step 5, the orphan names, and report its summary; otherwise mark naming NOT CHECKED.

### 4. Build the observed inventory

Normalized CSVs with the columns of [assets/observed-inventory-template.csv](assets/observed-inventory-template.csv), each passed with its own `--observed`. Read-only routes:

- GA4: BigQuery with [assets/ga4-bigquery-inventory.sql](assets/ga4-bigquery-inventory.sql), in the console or through Google's BigQuery remote MCP server with `execute_sql_readonly` only; or the Data API through Google's GA4 MCP server started without `--allow-write`, with the eventName, platform and date dimensions (names, sources, counts and dates only).
- Mixpanel: the Lexicon CSV with an Analyst-role account, MCP read tools behind the deny list in Boundaries, or a raw export through the profiler.
- Amplitude: `get_amp_taxonomy` with a role that grants only Use MCP (read), or an Export API file through the profiler.
- PostHog: the MCP server with `readonly=true` and [assets/posthog-hogql-inventory.sql](assets/posthog-hogql-inventory.sql).
- Segment: Source Schema, violations and [assets/segment-warehouse-inventory.sql](assets/segment-warehouse-inventory.sql).
- Run a warehouse query only after a dry run (`dryRun` true on `execute_sql_readonly`, or the console estimate); above the cap the user sets (ask for one), state the estimate and wait for approval, or hand the query over.

```bash
python3 "$SKILL/scripts/profile_export.py" export.zip --format amplitude --tool amplitude \
  --since 2026-10-03 --until 2026-10-30 --values-for connector_type,report_type \
  --identity-keys user_id > observed-amplitude.csv
```

Record row totals and caps (Lexicon CSV 5,000 events, GA4 daily batch export one million events on standard properties, Amplitude 2,000 event types). A names-only listing leaves `rows` empty, never 0. A vendor view shorter than the window never proves absence. Types, time zones and caveats per tool: [references/observed-inventory.md](references/observed-inventory.md).

### 5. Run the drift check

```bash
python3 "$SKILL/scripts/audit_plan.py" drift --events plan-events.csv --properties plan-properties.csv \
  --observed observed-amplitude.csv --observed observed-ga4.csv --reference reference-counts.csv \
  --ga4-definitions ga4-definitions.csv --window-start 2026-10-03 --window-end 2026-10-30 --out audit-out
```

- Rules D01 to D15 with thresholds and reasons: [references/drift-rules.md](references/drift-rules.md). Defaults: type mismatch FAIL at 1%, required completeness FAIL below 95% (below 100% on tier 1 server events), enum drift FAIL above 1%, ratio to the system of record FAIL above 1.0, minimum sample 100; an event silent for about seven expected rows before the window ends has stopped.
- `--reference` takes a filled copy of [assets/reference-counts-template.csv](assets/reference-counts-template.csv), time zones included; `--ga4-definitions` one of [assets/ga4-custom-definitions-template.csv](assets/ga4-custom-definitions-template.csv). Pass `--ga4-param-limit 100` for Analytics 360. Exit code 1 means at least one FAIL, 2 bad input. `--out` writes findings, a per-event snapshot and a plan-changes draft, and refuses folders inside `martech-context/` or the skill folder.
- If Python is unavailable, apply the rules by hand and label the report "script not run".

### 6. Review every finding with judgment

- Vendor events and parameters are never orphans: GA4 automatic events, `$` defaults, `[Amplitude]` events, mobile lifecycle events, Segment's "Loaded a Page" and "Violation Generated". Copy [assets/vendor-events.txt](assets/vendor-events.txt) and [assets/vendor-params.txt](assets/vendor-params.txt) to the scratch folder, refresh the copies from the live docs and pass them with `--vendor-events` and `--vendor-params`.
- Root-cause locator: the same defect in every tool points at the emitter; one tool only points at that tool's path (tag, CDP mapping, connector, transformation, event rule). The script prints a cross-tool matrix.
- Group findings with one cause: a second producer explains a ratio above 1.0 and the completeness FAILs that match its share.
- Version tail: split rows by app version in a separate profiler run, read by hand and never passed to drift. When 90% or more of a defect's rows come from versions older than the fixing release, rate WARN with an expiry in `decisions.md`.
- Not shipped is not broken: proposed and approved events are not expected yet; implemented or verified with no rows is a defect or a wrong status.
- Canonical name in a variant pair: the plan's name when consumers use it, never chosen by volume or style.
- Types are fixed at the emitter. A legitimate new enum value amends the plan after the enforcement copies list it.
- Downgrading any FAIL requires citing `decisions.md` or the contract.

### 7. Confirm before you conclude

- Every FAIL needs one piece of evidence beyond the inventory: a test-account payload from the tool's live view or debugger, the emitter location, or the system-of-record count. Otherwise it is labelled unconfirmed.
- Search every emitter root before calling anything "not emitted" or proposing retirement; until then write "not observed in <tool>" ([references/emitter-search.md](references/emitter-search.md)).
- Never submit forms, orders or consent choices on a live site without the user's approval for that test.

### 8. Reconcile and report

- Give each finding one disposition: fix implementation, amend plan, adopt into plan, retire, rename or merge, accepted exception, or needs decision. Rules: [references/reconciliation.md](references/reconciliation.md).
- Write the report from [assets/audit-report-template.md](assets/audit-report-template.md), plan changes from the script's draft ([assets/plan-changes-template.csv](assets/plan-changes-template.csv)) and the change list from [assets/change-list-template.md](assets/change-list-template.md): privacy, double counting and wrong values, missing data, plan amendments, hygiene, one hypothesis per request.
- Propose updates to `martech-context/` (tracking-plan, data-dictionary, decisions) and write nothing until the user approves.

## Common failure modes

- Two producers, one name: D12 ratio above 1.0, with required properties present on only part of the rows (D07).
- One action under several spellings, from platform teams or a connector's rename settings: D03.
- Numbers sent as text through quoted tag variables or string building, or frozen by a warehouse column's first value: D10.
- IDs sent as numbers, losing leading zeros or digits above 2^53 in Mixpanel: D10 and D14.
- Enforcement that drops silently (Amplitude reject keeps the event without the property; Segment omission works only in cloud mode): D07 lower in the tool than in the warehouse.
- Plan says verified, tool says nothing or stops partway (emitter behind a flag, a refactor, a GA4 modification rule): D01, before or inside the window.
- Dynamic names, keys and per-page events pushing toward Amplitude's 2,000 event types or a Lexicon file's 5,000 events: D04.
- Short IDs that Amplitude removes, and IDs missing where the plan requires them: D14, upstream and per event.
- A time zone offset or late data read as double counting or loss: D12.
- Short vendor windows read as clean: Avo and Amplitude errors 24 hours, Segment violations 7 days.
- Full catalogue with detection queries: [references/failure-modes.md](references/failure-modes.md).

## Worked example

Fictional Acme Reports, plan "trial signup to first report", audited on 2026-11-03 for 2026-10-03 to 2026-10-30 against Amplitude (through Segment) and GA4. Quality ends with `0 FAIL, 2 WARN, 11 PASS, 0 not checked` (verdict contract) and drift with `10 FAIL, 2 WARN, 5 PASS, 4 not checked`. Among them:

- GA4 `sign_up` at 1.90 times the identity database count (both in UTC days), with `method` and `event_id` missing on 47.4% of rows: one second producer, one change request (CR-2).
- `email` sent as a `user_registered` property: CR-1, first in the change list.

Commands, the agent's handling of every finding, CR-1 and CR-2 in full and the regression fixtures: [references/worked-example.md](references/worked-example.md).

## Acceptance checks

- Baseline: plan location and version, tools with placeholders, production scope, window, as-of date, time zone per tool and per reference, inventory route, read-only boundary and row totals per tool, caps checked. Pass: all present, and any capped or partial inventory is named.
- Plan quality: Q01 to Q13 each reported with coverage and PASS, WARN or FAIL, plus the verdict and the naming line. Pass: no rule missing, blocking failures named, drift depth stated.
- Coverage: every planned event mapped to an audited tool has a status line for that tool, and every observed name is classed as planned, variant, vendor, generated, dynamic, test or orphan. Pass: zero unclassified names.
- Numbers: every FAIL and WARN cites its rule, the rows affected and the window; share rules (D07, D08, D10, D11, D12, D14 shares) also cite the denominator and share. Pass: no finding without its numbers.
- Evidence: every FAIL has a payload, an emitter location or a system-of-record count, or is labelled unconfirmed. Pass: no silent confirmation.
- Not-emitted claims: nothing is called unused or proposed for retirement unless every emitter root was searched and no consumer exists. Pass: the roots searched are listed.
- Reconciled plan: every finding has one disposition; every plan edit names its change class, reason and approver role; retired items stay in the plan. Pass: the plan-changes rows match the findings that need plan edits.
- Change list: ordered by the five buckets; each request names system, entity, current state, change, owner, tests with evidence per layer, rollback and what rollback does not undo, and the reporting annotation. Pass: a person could apply it without asking a question.
- Privacy and safety: no property values beyond enum values and redaction tokens, no credentials, reads only in the tool call log, nothing written to `martech-context/` or the skill folder. Pass: a search of the outputs finds no email or phone patterns other than named test accounts on example.com or the team's test domain, and no write calls.

## Boundaries

- Read-only, always. The skill never blocks, hides, merges, deletes or dismisses anything, never creates branches or uploads plans, and never edits Lexicon, Amplitude Data, Avo or Protocols, even when the connected identity could. Fixes leave as a change list and a reconciled plan draft for human approval.
- A prompt that says "read only" removes no permission. The boundary is the identity's role, the server's read-only switch or the client's tool deny list; record which one held in the baseline.
- Retirement goes through the change list: stop emission at the source first, and block or delete in a tool only after it shows zero rows for 7 days, because blocked data cannot be recovered.
- Legal basis and deleting personal data already collected belong to the privacy owner and each vendor's deletion process. Metric definitions belong to their owners in `metrics.md`; the audit names the consumers a fix will move.
- Hand-offs:
  - tracking-plan-builder for contracts of adopted events and change control; event-naming-convention for renames; the skills under When to use for the rest.
  - Google's GA4 MCP server with the `analytics.readonly` scope and without `--allow-write`. Google's BigQuery remote MCP server with `execute_sql_readonly` only (`execute_sql` and `cancel_job` denied in the client and by an admin's IAM deny policy), or the console. Set Maximum bytes billed in the console. `execute_sql_readonly` has no bytes cap, so set `jobTimeoutMs` and a label too (spelling per the client's tool schema); a project quota is only a backstop.
  - Mixpanel MCP: no read-only mode is documented. Before the first call, deny in the client every tool starting with Create, Update, Delete, Duplicate, Edit, Bulk-Edit, Rename, Dismiss or Merge, and record that deny list as the boundary; if the client cannot deny tools, use the Lexicon CSV and the raw export.
  - Amplitude MCP with a role granting only Use MCP (read), never `manage_amp_*`.
  - PostHog MCP with `readonly=true`; Avo MCP with the read scope only (decline the write-scope consent); RudderStack MCP (no read-only mode documented) with only its read tools allowed in the client, never its write or sensitive execution tools; Segment changes made by a workspace owner in the app (Twilio's MCP server only searches docs).

## Live docs

Platform settings, limits and menu paths change. Before stating any parameter name, limit, quota or UI path:

- Fetch the current page from the vendor docs MCP server or llms.txt listed in `sources.yaml`, or open the canonical URL.
- If the live page disagrees with this skill, follow the live page and tell the user what changed.
- Cite the URL you used.

Most used: GA4 limits and naming rules, the BigQuery export schema and MCP server, Mixpanel raw export and MCP tools, Amplitude Export API and HTTP V2 rules, PostHog MCP, Segment Protocols and warehouse schema. Checked 2026-11-01 to 2026-11-06; re-check MCP tool lists before relying on a name.