# Worked example: Acme Analytics (fictional) Acme Analytics is a fictional B2B reporting SaaS on HubSpot Sales Hub Professional, with Clay for orchestration, three placeholder enrichment providers (provider_a, provider_b, provider_c, with made-up prices that are not vendor pricing) and Instantly for sequences. Version 1 (demo and pricing requests, product milestones, job changes of known contacts) passed four weekly reviews. Version 2 adds web intent and hiring, and the RevOps lead asked for a review of the draft before its pilot. Decision sentence: when a target-fit account shows a qualifying signal, the account owner, or an SDR for unowned new business, gets one task or one approved sequence enrollment within the response target, and the outcome returns to the weekly signal review. Capacity: 2 AE reviewers x 90 minutes / 8 = 22 tasks a week; SDR sequence capacity 40 new contacts a week. ## Files - [../assets/example-acme-stack.md](../assets/example-acme-stack.md): the `stack.md` and decisions the agent reads first. - [../assets/example-acme-episodes.csv](../assets/example-acme-episodes.csv): 52 rows from the last 30 days, 48 episodes plus 4 repeat deliveries; 6 direct requests, 14 first-party, 16 inferred, 8 job change, 4 hiring. 48 episodes, kept small for the example; a real dry run extends the window to 200 episodes. - [../assets/example-acme-routing-policy-v1.json](../assets/example-acme-routing-policy-v1.json): the draft policy. [../assets/example-acme-routing-policy-v2.json](../assets/example-acme-routing-policy-v2.json): the corrected one. - [../assets/example-acme-ledger.csv](../assets/example-acme-ledger.csv) and [../assets/example-acme-billed.csv](../assets/example-acme-billed.csv): the work email pilot on 31 records and the billed units. - [../assets/example-acme-contract.csv](../assets/example-acme-contract.csv), [../assets/example-acme-crm-contacts.csv](../assets/example-acme-crm-contacts.csv) and [../assets/example-acme-writeback.csv](../assets/example-acme-writeback.csv): the field ownership contract, a 20-contact HubSpot export and the proposed writeback from the Clay table. - [../assets/example-acme-workflow-spec.md](../assets/example-acme-workflow-spec.md): the finished spec after the fixes. ## 1. Routing dry run ```bash python3 scripts/route_dry_run.py --policy assets/example-acme-routing-policy-v1.json --records assets/example-acme-episodes.csv ``` Expected, exit 1: ```text Routes: replay 4, no_action 3, service 1, cs_task 4, hold 3, owner_task 9, attach_existing 2, pool_task 7 (ae 5, sdr 2), sequence 10, research 5, nurture 3, unrouted 1 FAIL I3 open-opportunity: episode E-014 (row 16) has an open opportunity but routed to sequence (rule R10); expected opp_owner_task to AE-2 FAIL I3 open-opportunity: episode E-035 (row 38) has an open opportunity but routed to sequence (rule R10); expected opp_owner_task to AE-1 FAIL I4 existing-owner: episode E-003 (row 4) assigned AE-1 but account owner is AE-2 (rule R09) FAIL I4 existing-owner: episode E-007 (row 8) assigned AE-2 but account owner is AE-1 (rule R09) FAIL I6 coverage: episode E-022 (row 24) matched no rule; add a monitored fallback rule at the end FAIL I8 inferred-signal: episode E-004 (row 5) is an inferred signal routed to sequence without corroboration (rule R10) FAIL I8 inferred-signal: episode E-028 (row 30) is an inferred signal routed to sequence without corroboration (rule R10) FAIL I8 inferred-signal: episode E-041 (row 45) is an inferred signal routed to sequence without corroboration (rule R10) Reconciliation: 52 rows = 4 replay + 47 routed + 1 unrouted 8 FAIL, 0 WARN. Failed invariants: I3, I4, I6, I8. Nothing was changed. ``` The route summary is condensed here; the script prints one route per line. With policy v2 the same command ends `0 FAIL, 0 WARN. Nothing was changed.` with exit 0 and routes replay 4, no_action 3, service 1, cs_task 4, opp_owner_task 5, hold 3, owner_task 8, attach_existing 2, pool_task 5 (ae 3, sdr 2), sequence 5, research 8, nurture 3, fallback 1. Rules R02a, R06a, R06c and R99 never fire in this sample, so they print as INFO; keep all four and prove each with a test row: a job change 120 days after its start date with a 90-day expiry for R02a (expected no_action), a demo request from a personal email address for R06a (fallback), a job change with no start date on an unowned account for R06c (research), and a row no other rule matches for R99 (fallback). The file covers about four weeks, so the period capacity is `--capacity ae=88,sdr=160`; even the weekly figures `--capacity ae=22,sdr=40` pass, with ae holding 16 items and sdr 7. 48 episodes is below the 200 the procedure requires; extend the window to 200 episodes before approving the pilot. What the agent does with each finding: - I4 x2: the direct-request pool rule R09 sits above the named-owner rule R07, so owned accounts rotate to the other AE. Move R07 above every pool rule. - I3 x2: the open-opportunity rule R05 sits below the sequence rule; two band contacts with open opportunities and no account owner were sequenced. Move R05 directly under the customer rule. - I8 x3: R10 has no corroboration condition, so web-intent-only contacts enter the sequence with a visit claim. Add the `when_any` corroboration condition. R11 also needs `signal_class ne inferred`, or the same three rows become SDR pool tasks; with both, uncorroborated web intent falls through to R12 research. - I6 x1: a first-party band row with no territory matches nothing. Add R09a (territory missing to fallback) and R99 (fallback to duty-rep with a 60-minute escalation). - No finding, still a defect: v1 has no expiry rules, so a job change past 90 days or one with no start date would become an owner or SDR task. No row in this sample is expired or undated, so the dry run cannot show it. Add R02a (expired to no_action) and R06c (undated to research) and prove them with the test rows above; I12 fails a policy that lacks them on such rows. ## 2. Enrichment cost ```bash python3 scripts/enrichment_cost.py assets/example-acme-ledger.csv --ceiling 1.00 --billed assets/example-acme-billed.csv ``` Expected, exit 1 (excerpt): ```text Stage 1 provider_a: entered 30; candidates 20 (1 on excluded records); accepted 15, held 2, rejected 2, no result 10, error 1, pending 0 cost 1.1000 USD (0.0550 per candidate); marginal 0.0733 USD per accepted; coverage 50.0% Stage 2 provider_b: entered 15; candidates 10 (1 after acceptance); accepted 6, held 2, rejected 1, no result 6, error 0, pending 0 cost 1.2500 USD (0.1250 per candidate); marginal 0.2083 USD per accepted (2.8x earlier stages); coverage 70.0% Stage 3 provider_c: entered 9; candidates 3; accepted 1, held 2, rejected 0, no result 6, error 0, pending 0 cost 3.6150 USD (1.2050 per candidate); marginal 3.6150 USD per accepted (32.3x earlier stages); coverage 73.3% Final: accepted 22 of 30 (73.3%); reused 0; unresolved 8 (held 3, rejected 1, no result 4, pending 0, error 0, not attempted 0) Cost: variable 5.9650 USD; blended 0.2711 USD per accepted Waste: 0.0550 USD on excluded records (c-031); 0.1250 USD after acceptance (c-012) provider_c lookup: modeled 9, billed 12 (+33.3%) FAIL W04 exclusion-before-spend: c-031 work_email was excluded at stage 0 (customer) but has 1 paid attempt at stage 1 costing 0.0550 USD FAIL W06 waterfall-stop-and-states: c-012 work_email was attempted again at stage 2 (provider_b) after it was accepted at stage 1, and has 2 accepted candidates FAIL W08 marginal-cost: work_email stage 3 (provider_c) costs 3.6150 USD per accepted record, above the 1.00 USD ceiling WARN W06 waterfall-stop-and-states: c-030 work_email ended stage 1 (provider_a) in error after 1 attempt and moved to stage 2 without a successful retry WARN W08 marginal-cost: segment na-midmarket (work_email): stage 1 entered 30, stage 2 entered 15, stage 3 entered 9, below the 200 records a provider ordering decision needs at each stage WARN W09 caps-and-reconciliation: provider_c lookup billed 12 vs modeled 9 (+33.3%), above the 10% variance; find the reruns, charged misses or retries before scaling 3 FAIL, 3 WARN. Nothing was changed. ``` Without `--billed` the run ends `3 FAIL, 2 WARN` and prints an INFO line that billed usage was not checked. Without `--ceiling` the stage 3 FAIL becomes a WARN at 32.3 times the 0.1119 USD blended cost of stages 1 and 2. What the agent does: - W04: the customer exclusion runs after stage 1 in the Clay table. Move it before the first paid column (CR-2). - W06 FAIL: stage 2 has no run condition, so an accepted record was bought again. Add "Only run if" stage 1 did not accept (CR-2). - W06 WARN: c-030 timed out and fell through. Retry at the same stage up to 3 times before moving on. - W08 FAIL and W09: provider_c charges misses, so one accepted email cost 3.615 USD. Limit it to tier-1 accounts or remove it until a 200-record sample proves it under the ceiling, and explain the 3 extra billed lookups (CR-3). ## 3. Writeback check ```bash python3 scripts/check_writeback.py --contract assets/example-acme-contract.csv --current assets/example-acme-crm-contacts.csv --proposed assets/example-acme-writeback.csv --blank-mode send ``` `--blank-mode send` models the Clay HubSpot action as configured today: Ignore blank values off. Row 20's email belongs to contact07, which the export holds as the update target of row 8. A create whose match is not an update target is caught only when the export also holds a lookup on that create's email. Expected, exit 1 (excerpt, order as printed): ```text FAIL W18 field-ownership: mapping: writer clay_writeback maps lifecyclestage (never written by a writeback); with blank mode send its 14 blank cells would clear current values (rows 2, 3, 4, 5, 6 and 9 more); remove it from the mapping FAIL W18 field-ownership: mapping: writer clay_writeback maps hubspot_owner_id (never written by a writeback); with blank mode send its 14 blank cells would clear current values (rows 2, 3, 4, 5, 6 and 9 more); remove it from the mapping FAIL W18 field-ownership: row 5 update 10234500104 enr_company_size: clear not allowed (blank_input is ignore) FAIL W18 field-ownership: row 9 update 10234500108 lifecyclestage: never written by a writeback; proposed a new value FAIL W18 field-ownership: row 11 update 10234500110 linkedin_bio: not in the contract, so no writer owns it; proposed a new value FAIL W18 field-ownership: row 12 update 10234500111 enr_company_size: writer zap_import is not allowed; proposed a new value FAIL W18 field-ownership: rows 16 and 17 update record 10234500115 with different values for enr_company_size, enr_source; hold both until one value is chosen FAIL W19 create-update-identity: row 18: update target 10234500199 is not in the export; never fall through to create FAIL W19 create-update-identity: row 19: record ID 1.02345E+10 looks damaged by a spreadsheet; re-export with IDs as text FAIL W19 create-update-identity: row 20: duplicate create; email matches existing record 10234500107 (case and spaces ignored) FAIL W19 create-update-identity: row 21: create without identity evidence (email empty) WARN W18 field-ownership: row 4 update 10234500103 jobtitle: differs from the current value; write to enr_jobtitle_candidate for review WARN W18 field-ownership: row 8 update 10234500107 enr_email_status: clears the current value; allowed only with approval INFO W19 create-update-identity: 4 create rows checked for duplicates only against the 20 records in the export, on email (export column email); it must hold a lookup on each create row's email or domain Decisions: fill 57, overwrite 2, create 12, candidate 1, no_change 12, needs_approval 1, blocked 7 Blocked through a mapping finding: 28 blank cells. Rows blocked whole: 6. 14 FAIL, 4 WARN. Nothing was changed. ``` The INFO line says which export columns the duplicate check compared. On a real HubSpot export, add `hs_additional_emails` to the export and pass `--identity "email=email|hs_additional_emails"`, quoted because `|` is a shell pipe, so a create whose email is another contact's additional email is caught. Findings carry no cell values. The `--out` change list holds current and proposed values for the approver; `--show-values` prints them in findings, which is fine on this fictional data and never on a real export. The full run also blocks blank clears of phone (row 6) and jobtitle (row 7), the owner change on row 10, and prints WARN mapping lines for linkedin_bio and for zap_import. With `--blank-mode ignore` (Ignore blank values back on) the same file ends `9 FAIL, 5 WARN`: the three clears and the approval disappear, and the two mapping FAILs become WARNs. The file still cannot go for approval. What the agent does: - Mapping FAILs: lifecycle stage and owner are mapped in the Clay action. Remove both from the mapping and turn Ignore blank values back on (CR-2). Lifecycle changes stay with HubSpot workflows; owner changes stay with routing. - Clears: with Ignore blank values on they disappear; the enrichment status clear on row 8 needs a named approver or goes. - Candidate: the rep's "RevOps Lead" stays; the provider's title goes to `enr_jobtitle_candidate` for review. - Row FAILs: re-export with IDs as text, drop the duplicate create (contact07 already exists), hold the identity-less create, resolve rows 16 and 17 to one value, and find why row 18's target is missing (merged or deleted) instead of creating it. ## Change requests ```markdown ### CR-1 Routing precedence, evidence age and fallback (W12 FAIL, W13 FAIL, W02 FAIL) - System: HubSpot workflow "Signal routing" and the policy file, v1 to v2. - Current: R09 above R07; R05 below R10 and R11; R08 also catches direct requests; R10 lacks the corroboration, suppression and direct-request conditions; R11 takes inferred and undated rows; no expiry rules; no fallback rule. - Change, rule by rule (v1 to v2, compare the two policy files): - R02a, new, under R02: direct_request ne true and evidence_age_days gt expiry_days: no_action, reason expired. - R05: move directly under R04. - R06c, new, under R06b: direct_request ne true and evidence_age_days or expiry_days empty: research, reason unknown_source_date. - R07: move directly under R06c, above R08 and every pool rule. - R08: add direct_request ne true. - R09a, new, under R08: territory empty: fallback, reason territory_missing. - R09: move under R09a. - R10: add suppressed eq false and direct_request eq false; add when_any signal_class ne inferred or corroborated eq true; remove territory not_empty (R09a catches it now). - R11: add signal_class ne inferred and evidence_age_days not_empty; remove territory not_empty. - R99, new, last: fallback to duty-rep, 60-minute escalation. - Test: AC06, AC07, AC21, AC22, AC28, the four test rows for R02a, R06a, R06c and R99, and route_dry_run.py on at least 200 episodes (the last 30 days hold 48, so extend the window): zero FAIL. - Rollback: restore policy v1 and the previous workflow version. Does not undo tasks already created. ### CR-2 Clay table run conditions and HubSpot action settings (W04, W06, W18 FAIL) - System: Clay table "Acme signals - work email", HubSpot Update object action. - Current: customer exclusion after stage 1; stage 2 runs on every row; Ignore blank values off; lifecyclestage and hubspot_owner_id mapped; jobtitle written over rep values. - Change: move the exclusion column before stage 1; stage 2 "Only run if" stage 1 did not accept; Ignore blank values on; remove lifecyclestage and hubspot_owner_id from the mapping; map jobtitle conflicts to enr_jobtitle_candidate. - Test: AC11 to AC14 on 10 test records, then check_writeback.py on the next proposed file: zero blocked cells. - Rollback: previous column settings. Does not undo credits spent or values already cleared (restore those from the before-values log). ### CR-3 provider_c scope (W08 FAIL, W09 WARN) - System: Clay waterfall stage 3. - Change: limit stage 3 to tier-1 accounts, or remove it until a 200-record sample shows marginal cost at or below 1.00 USD; reconcile the 3 billed lookups the ledger does not explain. - Test: enrichment_cost.py on the next pilot with --billed: no W08 FAIL, variance at most 10%. ``` ## Regression baseline After changing any script, rerun from the skill folder and compare the last line and exit code: - Routing v1: `8 FAIL, 0 WARN. Failed invariants: I3, I4, I6, I8. Nothing was changed.` exit 1. - Routing v2: `0 FAIL, 0 WARN. Nothing was changed.` exit 0, with INFO lines for R02a, R06a, R06c and R99. The template policy gives the same routes and result on these episodes. - Cost with `--ceiling 1.00 --billed`: `3 FAIL, 3 WARN. Nothing was changed.` exit 1. - Writeback with `--blank-mode send`: `14 FAIL, 4 WARN. Nothing was changed.` exit 1; with the default blank mode: `9 FAIL, 5 WARN.` exit 1. The example files never reach these cases, so build a few test rows in a scratch folder: - Fallback above a protective rule: in a copy of policy v2 with R09a moved directly under R01, rows with a blank territory fail like this. A suppressed row that is not a direct request: `FAIL I2`. An open opportunity owned by AE-2: `FAIL I3`. An account owned by AE-1: `FAIL I4`. A customer owned by CSM-1: `FAIL I4` and `FAIL I5`. The same opportunity, account and customer rows pass when the owner is away with no covering rep, and so does a suppressed direct request. - Same news from two sources: two rows with one `episode_id` and different `source_event_key` on an owned account: `WARN I1`, `0 repeat deliveries`, 0 FAIL. - Expired and undated evidence: the R02a and R06c test rows from section 1 fail `I12` on policy v1 and pass on policy v2. - Pending: a ledger record pending at stage 1 and bought at stage 2: `WARN W06`. - Duplicate forms: with `--identity "email=email|hs_additional_emails,domain"`, a create whose email sits in another record's `hs_additional_emails` list, and a create with `https://www.example.com/` against an export holding `example.com`, each `FAIL W19`. - Owned account in a sequence: in a copy of policy v2 with the sequence rule above R07 and its pool removed, owned rows print `FAIL I4 ... routed to a sequence that assigns nobody`. - Thin ledgers: with no stage 0 rows, `INFO W04 ... not checked`, never PASS. A row with units and a blank `unit_price_usd` exits 2, because a blank would make a charged attempt free and hide a stage above the ceiling. - Empty writeback: a header-only file exits 2. A create-only file against an export with no records: `WARN W19 ... no duplicate check`, no PASS. - Bad input: an `.xlsx`, a Windows-1252 CSV or a file with NUL bytes exits 2 on all three scripts, and so does `--out` into a folder that does not exist yet. Exit 1 always means a FAIL finding.