# Failure modes seen in production

Each entry: what happens, how to detect it (with check IDs), and what to do next.

## Inventory and money

- Inventory from memory or from one channel.
  - What happens: SSO lists miss tools without single sign-on; AP misses free apps with data access; contracts miss card purchases.
  - Detect: run all four channels; read "Seen in one channel only"; INV01 and INV02.
  - Next: register every unregistered tool or mark it out_of_scope with a reason.
- Monthly equivalent hides annual cash.
  - What happens: an annually prepaid tool shows a small monthly figure; cancelling saves nothing until the term ends.
  - Detect: keep payment dates; split savings into cash through a date, run-rate and capacity; check the next invoice.
  - Next: date every saving from its effective date.
- Media or agency fees counted as tool spend.
  - What happens: ad spend dominates the cost view, hides tool waste and pulls the top-80% contract scope towards the wrong rows.
  - Detect: `spend_type` media and services, reported apart by the script.
  - Next: set spend_type on every register row before reading the top share.
- Currencies added together.
  - What happens: EUR and USD totals summed into a meaningless figure.
  - Detect: the script reports spend per currency only.
  - Next: convert only in the report, with the rate and date stated, if leadership asks.
- A vendor-level match hides a second product.
  - What happens: an SMS charge from the vendor behind the CDP is booked to the CDP, or an office suite charge lands on a free analytics row from the same vendor, so an unregistered subscription stays hidden.
  - Detect: the descriptors listed per tool in the channel coverage section; INV01 lines that name a vendor's register rows.
  - Next: add the descriptor as an alias on the right row, or register the second product.
- A sparse register read as clean.
  - What happens: blank columns turn duplicate-send, renewal and DPA checks into silent passes on the first run.
  - Detect: NOT CHECKED lines naming the blank columns; PASS lines naming their population ("2 of 20 tools record system_of_record_for").
  - Next: fill the columns and re-run after steps 4 and 5; report from the final run.
- Industry benchmark used as evidence.
  - What happens: a survey utilisation figure or a vendor savings claim becomes the business case for a cut.
  - Detect: a finding with no company evidence attached.
  - Next: quote industry figures only as context, with source and edition.

## Contracts and renewals

- The renewal fires mid-audit.
  - What happens: auto-renew locks another term while the audit is still scoring tools.
  - Detect: build the calendar first and sort work by review-by; CON03, CON04.
  - Next: decide overdue items in the first week; recertify locked ones 120 days before the next term end.
- Seat removal the contract does not allow.
  - What happens: the order blocks in-term reductions, or reprices a smaller quantity at renewal.
  - Detect: reduction_rule from the executed order; USE02 cash timing.
  - Next: plan the reduction for the allowed point; price the smaller renewal before committing.
- Discount cliff at renewal.
  - What happens: an introductory or startup discount ends and the renewal rises sharply even with fewer seats.
  - Detect: CON05; decompose the renewal into discount, unit price and quantity.
  - Next: put the decomposition in front of the decision owner before the decision target.

## Usage and shelfware

- Background tool retired on login data.
  - What happens: a one-seat dedupe, enrichment or sync tool looks idle; removing it corrupts routing or reporting within weeks.
  - Detect: jobs, syncs, API calls and workflow references in 90 days (HubSpot Connected Apps insights show API call usage, record insights and automation insights).
  - Next: no retire disposition without background-work evidence or the owner's written confirmation.
- Short log retention read as dormancy.
  - What happens: Entra Free keeps sign-ins 7 days and P1 or P2 30 days, so everyone older looks inactive.
  - Detect: USE03 compares each source's retention with the 45-day threshold.
  - Next: use the tool's own last login.
- A retirement that did not finish.
  - What happens: the row says retired, but the card is still charged or the renewal fired after notice, and people still sign in.
  - Detect: INV08 on the monthly re-run; retired spend reported apart.
  - Next: finish the retirement checklist; the contract owner checks the notice and the invoice.

## Overlap and replacement

- Both systems live during a replacement test.
  - What happens: customers get two onboarding emails; the CRM gets duplicate tasks.
  - Detect: the candidate's send and write logs in the test window; OVL02.
  - Next: rerun the test with sends and writes off in the candidate, comparing eligibility and state.
- Consolidation premised on a roadmap.
  - What happens: a suite's announced feature is counted as the replacement; it ships late or underperforms.
  - Detect: a replacement test on what ships today; the five-field rule for consolidate.
  - Next: keep the disposition at investigate until the test passes.
- Suppression state lost in migration.
  - What happens: hashed suppression exports (Customer.io's GDPR suppression export uses SHA-256) are imported as plain lists, or not at all.
  - Detect: the retirement checklist asks for suppression and preference state mapped and tested before cutover.
  - Next: plan the matching method; test that a suppressed person stays suppressed in the destination.

## Flows and identity

- Two writers on one field.
  - What happens: a warehouse reverse ETL and a CRM workflow both set lifecycle stage, so it oscillates and re-triggers workflows.
  - Detect: FLW01; property history alternating between sources.
  - Next: name one authority, or split the field, by the two-writer route in [handoffs.md](handoffs.md), and record it in `decisions.md`.
- Green connector taken as proof.
  - What happens: status shows for some apps only, user-level and multi-account apps show no last activity, and rejected records stay hidden.
  - Detect: the four-case test with a read-back at the destination; FLW06, FLW07.
  - Next: mark the connection configured until dated evidence exists.
- Suppression flow slower than the send.
  - What happens: unsubscribes reach the sending tool after the next send, so opted-out people get mail.
  - Detect: FLW02 on flows that feed customer actions; required against observed freshness.
  - Next: class 1, days 0 to 30. Contain first (suppress in both tools), then fix the flow.
- Automation acts on stale state.
  - What happens: a delayed step sends after the person unsubscribed or converted, or a duplicate source event enrols the same person twice.
  - Detect: the point-of-action checks; enrolment history for records that changed during a delay.
  - Next: the send or task step re-checks suppression, consent and goal state when it runs; class 1 when people were messaged after opting out.
- An audit test changes production.
  - What happens: a four-case test creates a real contact or unsubscribes a real person, or a consent-banner click sends data to ad platforms.
  - Detect: the call log; the Safety acceptance check.
  - Next: read records that already went through each case; run anything that writes only with the owner's approval and agreed test data.
- Email as the only durable key.
  - What happens: shared domains merge accounts, changed addresses split people, billing and CRM disagree.
  - Detect: FLW03; KEY02 duplicates; one CRM company holding two billing accounts.
  - Next: add an internal ID as the match key; no new segments on the email-only key meanwhile.
- Integration tied to a departed person.
  - What happens: an OAuth grant or private app created by someone who left breaks when the user is removed (HubSpot documents association calls failing after a legacy private app's creator is removed).
  - Detect: installer and owner columns in connected-app lists; Salesforce OAuth usage per user, plus External Client App usage from the Connect REST API; FLW05.
  - Next: rebuild under a service identity with a named human owner before anyone removes the user.

## Access and governance

- Agency-owned accounts and admins.
  - What happens: the GA4 property, GTM account or ad accounts sit under the agency; the company cannot recover access when the agency leaves.
  - Detect: INV05, ACC01, ACC02; the account owner field in each admin console.
  - Next: two company admins first, then an ownership transfer the agency performs.
- Admins without enforced MFA.
  - What happens: a CRM or warehouse admin signs in with a password alone; one phished password changes everything.
  - Detect: ACC04 on the seat `mfa` column and each tool's security settings.
  - Next: replace shared admin logins with named users, then require MFA or SSO for every admin.
- A leaver keeps tool accounts outside SSO.
  - What happens: the identity provider user is deactivated, but local passwords, API keys and personal integrations in tools outside SSO keep working.
  - Detect: ACC05 on the seat `idp_status` column.
  - Next: list what each account owns, move it to a named owner, then the tool admin disables the account.
- Dangling DNS after a retirement.
  - What happens: a CNAME still points at a cancelled landing-page, help-centre or link-tracking service and someone else claims it; an old SPF include keeps the domain near SPF's 10-lookup limit.
  - Detect: the DNS line in the retirement checklist; the DNS owner's zone export.
  - Next: the DNS owner removes the records after cutover; sender authentication goes to lifecycle-marketing-audit.
- Unknown page senders.
  - What happens: theme snippets, CMS apps and hard-coded pixels send data outside the tag manager and outside consent.
  - Detect: a fresh-profile page load per consent state, compared with the register.
  - Next: hand off to consent-debugging and gtm-container-audit.
- Edited admin export uploaded back.
  - What happens: Google Workspace's bulk app upload replaces Type, Id, Access and Org Unit settings in the console.
  - Detect: the boundary itself: download only, never upload.
  - Next: changes leave as a change list for the super administrator.
- Personal data pasted into the audit report.
  - What happens: user lists, emails or HAR contents end up in a document shared with vendors or agencies.
  - Detect: the scripts print counts and row numbers; search the report for "@" before sharing.
  - Next: replace any address with a role and a row number.

## AI

- Shadow AI with mailbox access.
  - What happens: meeting or writing assistants connected through Google or Microsoft sign-in read mail, calendars and files with no owner or DPA.
  - Detect: INV02 FAIL on restricted Google scopes in the accessed apps export; ACC06 for registered apps that hold them; the Entra enterprise app grants on Microsoft 365, recorded by hand; the AI connection inventory.
  - Next: IT security decides the same day; the audit drafts the revocation list, it does not revoke.
- AI connector on a personal admin login.
  - What happens: an assistant inherits Super Admin rights and write tools; nobody can switch it off centrally.
  - Detect: the AI connection inventory (identity, scopes, enforcing layer, off switch); excessive agency FAIL.
  - Next: the connection owner disconnects it in days 0 to 30, or reconnects it under a least-privilege user with only read tools allowed in the client; widen nothing until the access check is ready.
- An averaged score hides a blocking control.
  - What happens: a 3.7 out of 5 maturity or a 70 out of 100 stack score masks broken suppression or unknown ownership.
  - Detect: any composite number in the report.
  - Next: remove it; list blocking gaps separately from levels and ratings.