# Enrichment and cost per accepted record

Depth for procedure steps 4 and 9, and the ledger format `scripts/enrichment_cost.py` reads. The planned waterfall-enrichment-design and data-provider-evaluation skills go further on provider choice; until they are installed, use this file.

## Buy only what something consumes

- A field is bought only when a gate, score, route, message claim or owned CRM property reads it. Write the consumer next to the field in the spec.
- Typical cuts: phone numbers with no calling step, biographies nobody reads, ten AI summaries when routing needs region and size.
- Personal emails and mobile numbers only with a recorded privacy decision per region (W05). Business contact data is the default.

## Reuse before re-buying

- Freshness defaults: title and employer 90 days, firmographics 180 days, email validation 30 days before an enrollment.
- A job change invalidates employer, title and work email at once. Re-run identity before reusing anything.
- Re-check suppression and eligibility immediately before the downstream action, whatever the freshness says.
- Record reuse as a stage 0 `reused` row in the ledger so the cost report shows it.

## States, retries and the stopping rule

- States per record and data point: `not_attempted`, `pending`, `no_result`, `candidate_held`, `candidate_rejected`, `accepted`, `error`.
- Transitions: not_attempted to pending or a result; pending to a result (Apollo delivers waterfall results later by webhook, so pending is not failed); error to a retry at the same stage.
- Retries: up to 3 with backoff, honouring the provider's Retry-After. An error never falls through to the next paid provider; that buys the same data twice and hides an outage.
- Stop at the first candidate that passes identity match and the validation policy. Held and rejected candidates let the waterfall continue.
- Stop paying a provider that repeats the same invalid candidate: in Clay set the duplicate-result threshold on email waterfalls to 2 or more (the default 0 disables it).

## Acceptance policy for work email

- Identity mismatch (different person or employer): reject, whatever the validator says.
- Valid with identity matched: accept.
- Accept-all or catch-all: hold. Allowed only for a human-reviewed task, never an automated sequence, unless `decisions.md` records otherwise.
- Override for catch-all, only with a `decisions.md` entry: run a second-pass catch-all verification and keep its raw result beside the status. Accepted catch-alls go to their own campaign with a lower per-mailbox daily limit, and that campaign pauses at 2% hard bounces measured on its own sends. Report the held catch-all share of eligible records each run, so the owner sees what the hold costs.
- Unknown: hold. Invalid or disposable: reject.
- Keep the provider's raw status beside the normalized decision (`enr_email_status` and the accepted flag).
- Validation is not permission to contact. Suppression is checked again just before enrollment.

## Ordering providers

- Measure each provider on the records earlier stages left unresolved. Its standalone hit rate on the whole list is not its marginal value.
- Use matched or randomized samples with the same acceptance policy, segmented by country, company size and role.
- Sample size: 200 records entering the stage under test, per segment, or the whole population if smaller. For a 50% acceptance rate the 95% margin is 1.96 x sqrt(0.25 / 200) = 0.069, about plus or minus 7 points.
- Later stages see only what earlier stages left: in the Acme pilot, 30 eligible records left 9 for stage 3. Test a later provider head to head on 200 records that the earlier stages left unresolved.
- Ten configuration test rows are not a benchmark.
- Providers can resell the same underlying data. More providers is not more independent evidence.
- Start with a low-cost, high-acceptance source; add sources only for distinct coverage; revisit when the audience or terms change.

## The ceiling

- Ceiling per accepted record = what the owner will pay for one more accepted record of this kind. Propose it for `decisions.md` with its basis.
- Optional derivation: accepted-to-meeting rate x meeting-to-opportunity rate x value per opportunity x the share of value the owner allows for acquisition.
- Keep a stage only at or below the ceiling. Without a ceiling, `enrichment_cost.py` warns when a stage costs more than 3 times the blended cost of earlier stages (`--warn-ratio`).

## Spend controls and traps (checked 2026-11-01; verify live)

- A budget written in chat is not a cap. Spend controls live in the tools: run conditions, auto-run state, filtered views of approved rows, approved record counts and per-user credit limits. A per-record or batch figure that no tool enforces is a pilot-job limit: check it in the usage Run view and the ledger after each run.
- Clay tables:
  - Table auto-run is on by default, and column auto-run only works while table auto-run is on. Turn table auto-run off while building.
  - "Only run if" formulas gate a paid column (for example: stage 2 runs only when stage 1 did not accept).
  - Keep existing results limits automatic runs to errored, empty or new cells.
  - A scheduled source with Update existing rows set to All re-runs every row on each run.
  - Filtered views enrich only the rows in view, so a pilot runs on a view of the approved rows.
  - Spend limits (checked 2026-11-06): on the Enterprise plan, admins can set fixed credit limits on workbooks and on the tables, signal tables and campaigns inside them. At the limit, credit-consuming processes stop and recurring spends are cancelled. Credit budgets, also Enterprise only, are shared pools that track Data Credits only, not Actions, and blocked runs must be re-run by hand. Clay's limits page contradicts itself on legacy tables outside a workbook, so check the table. No per-record cap is documented on any plan, so on other plans use the controls above.
  - A run cost estimate appears before a column run that triggers dependent columns. The expensive-run warning fires only for a run of at least 5% of the monthly credit budget and more than 500 credits, so smaller mistakes run without a prompt.
  - Website tracking connections have their own "Cap monthly credit spend" setting.
- Clay Work Email waterfall: Require validation success; strategies Conservative (verified only), Balanced (includes catch-alls), Aggressive and Advanced. Use Conservative with validation required for automated sequences. Infer Email runs first at no cost and still needs validation.
- Clay MCP for reps: per-user credit limits; "No limit" means no cap; a limit of 0 hard-blocks further actions; limits reset on the 1st at midnight UTC.
- Apollo:
  - An admin configures the waterfall sources and validator.
  - The API waterfall needs `run_waterfall_email` or `run_waterfall_phone` and a `webhook_url`. The synchronous response says accepted or failed (`partial_accepted` for bulk); final results arrive by webhook with the sources attempted, their statuses and credit usage.
  - Some sources charge for lookups that return no data. The waterfall data request log shows credits per record.
  - Rate limits apply per team, per endpoint, in minute, hour and day windows.
- ZoomInfo: per-user data and AI credit limits in the Admin Portal; enrich takes at most 10 records per call; a redeeming search returns at most 25 per page; Upsert Audience with a search query loads every match up to the plan maximum. There is no cost preview.

## Meters (checked 2026-11-01; verify live)

- Clay: Actions measure platform work, including CRM exports and syncs; Data Credits buy marketplace data and AI. Your own provider API key removes the Data Credit cost, but an Action is still consumed. No-result enrichments and API errors consume neither; marketplace validations returning valid or invalid are charged, but select validator providers are free under current pricing (checked 2026-11-06), so read the validator's actual Data Credit cost in the usage Column view (spend per column, not per provider) before modeling validation. Actions expire each cycle; monthly Data Credits accumulate up to 2 times the monthly limit; top-ups carry a 30% premium. The usage page's Time, Column and Run views export to CSV.
- Apollo: People API Search costs no credits and returns no emails or phone numbers; enrichment credits depend on data type, plan and waterfall configuration. Check the API pricing page live.
- ZoomInfo: 1 data credit per new record; re-enrichment within 12 months is free under Records Under Management (organization-wide per the GTM AI docs; confirm in the contract, older plans differed). MCP draws only pooled bulk data credits, not recurring monthly credits. Viewing a record in ZoomInfo Copilot is free; enriching through MCP counts as an export.
- HubSpot: native enrichment and buyer intent consume HubSpot credits. Check amounts live.

## The cost formula

- Accepted record: passed the acceptance policy and reached the destination field or task. In this skill, cost per enriched record always means cost per accepted record.
- Cost per accepted record = (provider data + validation + platform actions + allocated subscription share + review minutes x rate) / accepted records.
- Report four numbers: cost per returned candidate, marginal cost per accepted record by stage, blended cost per accepted record, and cost per accepted downstream outcome once the window matures.
- Tasks beyond weekly capacity have zero value in the model.

Worked math from the fictional Acme ledger (`assets/example-acme-ledger.csv`, placeholder providers and prices):

- Stage 1 provider_a: 20 charged candidates x 0.05 USD + 20 validations x 0.005 USD = 1.10 USD for 15 accepted: 0.0733 USD each.
- Stage 2 provider_b: 10 x 0.12 + 10 x 0.005 = 1.25 USD for 6 accepted: 0.2083 USD each.
- Stage 3 provider_c: 9 lookups x 0.40 (charged on misses) + 3 x 0.005 = 3.615 USD for 1 accepted: 3.615 USD.
- Blended: 5.965 USD over 22 accepted = 0.2711 USD. Stage 3 costs about 32 times the 0.1119 USD blended cost of stages 1 and 2, and breaks the 1.00 USD ceiling.

## Reconciling with billed usage

- After every pilot, compare modeled units with each vendor's own usage: Clay usage CSV, Apollo waterfall data request log, ZoomInfo Admin Portal usage, invoices.
- Variance above 10% for any provider blocks scaling until explained (`--billed`, W09).
- Usual causes: reruns, charged misses, retries, auto-run firing on edits, Update existing rows re-running every row, a second table pointed at the same source.

## Ledger format for enrichment_cost.py

One row per attempt. Columns (template: `assets/enrichment-ledger-template.csv`):

- `run_id`, `record_id`, `segment`, `data_point` (work_email, phone, company_domain).
- `stage`: 0 for exclusion or reuse before any paid step; 1, 2, 3 for waterfall stages in order.
- `provider`, `attempt` (1, 2, 3 for retries at the same stage).
- `outcome`: `excluded`, `reused`, `no_result`, `candidate_accepted`, `candidate_rejected`, `candidate_held`, `error`, `pending`.
- `units`, `unit_type`, `unit_price_usd`: what the attempt was charged (0 units for a free miss).
- `validation_units`, `validation_price_usd`, `validation_status` (the validator's raw status).
- `excluded_reason` on stage 0 rows; `note`.

Build it from the Clay table export and usage CSV, the Apollo waterfall log or webhook payloads, or your orchestration's run log. Keep record IDs, not personal data.

Usage: `python3 scripts/enrichment_cost.py LEDGER.csv --ceiling <USD from decisions.md> --billed BILLED.csv`. Billed file columns: `provider`, `unit_type`, `billed_units`, with an optional `source` column naming where each figure came from. Put validator usage on a row with provider `validation`, whatever the vendor, as in `assets/example-acme-billed.csv`. Exit 1 means at least one FAIL. Exit 2 means bad input, for example a file that is not a UTF-8 CSV (save Excel files as CSV UTF-8) or a row with units and a blank price (enter 0 when the unit is free). A ledger with no stage 0 excluded rows prints W04 as not checked, never PASS, so add the exclusion rows; W06, W07 and W08 do the same when the ledger gives them nothing to read.