Martech integration engineering

Designs or reviews one flow between martech tools so the receiving system holds the right current state after duplicates, retries, outages, removals and backfills. It outputs a spec, field ownership register, API sizing, test plan, runbook and change list. Nothing in a live system changes. A green sync status proves transport, not a correct customer record.

When to use

Use this skill when:

• A flow between CRM, billing, product, lifecycle, ads or warehouse tools needs a method and a spec.
• Someone asks native integration versus Zapier, Make or n8n versus reverse ETL versus custom code.
• Two systems overwrite each other, a two-way sync loops or a field keeps changing back.
• Webhooks go missing or arrive twice, tasks or orders are created twice, or old updates overwrite new ones.
• An API limit was hit, or a backfill, resync or replay is planned.
• An agency or vendor hands over an integration that needs review before anyone changes it.

Use something else when:

• Analytics event contracts: tracking-plan-builder.
• Browser tags: gtm-container-audit.
• Orders against GA4 purchases: purchase-reconciliation. Warehouse models and numbers you can trust: marketing-data-audit.
• A whole-stack audit: martech-stack-audit; bring single flows from its findings here.
• Go-to-market signal, enrichment and outbound workflows into the CRM: gtm-workflow-design.
• CRM duplicates and hygiene: crm-data-hygiene-audit. CRM configuration: crm-audit.
• Lifecycle stage, lead status or deal stage definitions and who may set each: revops-workflow-design.
• HubSpot portal setup, migration or property design: hubspot-implementation-plan.
• Braze SDK, Canvas entry and send QA: braze-implementation-qa.
• Choosing or securing an MCP connection: marketing-mcp-setup, mcp-security-review.
• An agent will apply the approved change: agent-write-safety.
• Building inside one iPaaS: the vendor's own skills (see Boundaries).
• Planned deep dives (crm-sync-and-field-ownership, reverse-etl-and-audiences, pipeline-connector-qa, event-pipeline-qa): this skill covers one flow until they ship.

Inputs

Read these first if they exist:

• martech-context/stack.md: tools, accounts, environments, owners and integration identities (names only).
• martech-context/data-dictionary.md and decisions.md: field meanings, ownership, identity and consent decisions, accepted exceptions.
• martech-context/tracking-plan/ when the flow carries events; metrics.md when it feeds a metric.

Then gather:

• The decision the flow serves, who acts on it, and how stale the data may be before that decision goes wrong.
• Source and destination objects, accounts, and volumes: records now, changes per day, peak per minute.
• The current setup, read-only: connector mappings, n8n workflow JSON or Make blueprint, sync settings, webhook subscriptions, API usage readings. Routes per tool: references/handoffs.md.
• Field history for the step 2 sample.
• Sandbox payloads, and source and destination exports holding IDs and the flow's fields only.
• Owners: business, technical and on-call.

Never ask for API keys, tokens, passwords, signing secrets or customer records. Ask for exports with header and query token values replaced by a placeholder; strip personal data from samples.

Procedure

Work in a folder outside the skill folder and martech-context/ (for example integration-drafts/<flow_id>/), with SKILL_DIR set to this skill's folder; list the drafts at hand-over. Without Python, apply the rules in the references' linter sections and the script headers by hand and label the report "script not run".

1. Frame the flow

• One flow is one source fact producing one destination state or action. Write the decision sentence, the mode and the tier. More than one fact or action: split it.
• Tier 1 can contact a customer, change consent, move money or bidding values, grant access or create sales work. Tier 2 changes fields people decide on. Tier 3 is an informational copy. Defaults per tier: references/operations.md.
• No owner or no decision: stop and write the question for the owner.
• Design mode: steps 2 to 10.
• Review mode: steps 1 and 2, write the current design into the inventory and field-map templates and lint it, test its method against step 4, size the daily and replay load with step 6 if a rate-limited API takes the writes, check n8n or Make exports, read back 50 records, report with assets/review-report-template.md. Export check: python3 -I "$SKILL_DIR/scripts/check_ipaas_export.py" workflow.json --tier <tier>, plus --ordered yes when changes must apply in order (state syncs, per-account serialisation).
• Recover mode: write the recovery plan from assets/runbook-template.md for a person to run after approval; the agent only reads, sizes the replay and runs the read-back.

2. Find every writer

• Read history for the flow's fields on at least one record per lifecycle path it touches (created, changed, cancelled or churned, merged, manually corrected) and on the ten most recently modified records per destination object.
• Where history comes in bulk (HubSpot batch read with propertiesWithHistory; Salesforce history on tracked fields), tally change sources over the last 90 days.
• No history (Salesforce tracks only chosen fields; many tools keep none): mark the check NOT CHECKED with the reason and use the fallback evidence in the contract rules. Never PASS without history.
• A writer in history but not in the inventory is a FAIL until someone owns it.

3. Settle the contract

Rules: references/contract-rules.md.

• Identity:
  • A stable source ID in a destination field that enforces uniqueness (HubSpot unique value property, Salesforce External ID plus Unique).
  • Email only for people; never name or domain.
  • Two matches: no write, exception.
  • No match: create only if this flow owns creation.
  • Keep a crosswalk.
• Keys: never lowercase opaque IDs, keep leading zeros as text, test the destination's case rules.
• Ownership: one owner and one direction per field; conflicts by version or effective time; last write wins only when arrival order equals authority. Write an allowlist of owned fields; skip writes that change nothing.
• Blanks: absent means no change; clearing needs an explicit clear from the owner; a null from a failed join is skipped and flagged. Vendor semantics differ (Salesforce Bulk clears on #N/A; HubSpot on an empty string; Hightouch sends nulls by default).
• Removals: unsubscribe, exit, delete and merge each get a behaviour. Add-only cannot meet a current-membership contract. Stop any run removing more than 10% of the population or 1,000 records until approved.

4. Choose the method

Take the first method that passes every contract clause; record the clause each simpler one failed. Tests and nine pattern cards: references/method-selection.md.

• Native connector: exact objects and associations, per-field direction and blank handling, a match key you control, record-level errors, the removals you need and a sync cycle inside the freshness target. A global "this system wins" setting fails decision-driving fields.
• iPaaS: 10 actions or fewer, 2 error branches or fewer, durable idempotency (data store or upsert on a unique key), safe retries, readable run history, monthly runs x billable steps x 1.3 within the plan, trigger interval inside the freshness target.
• Event pipeline: one event to two or more destinations; not current-state CRM fields.
• Warehouse plus reverse ETL: joins, modelling or one field to several tools; freshness must allow load, build and sync interval (15 minutes is the floor); never send-time suppression.
• Custom code: version checks, creates with uncertain outcomes, per-account serialisation, or failing iPaaS economics. Needs a managed runtime, secrets manager and on-call.
• File only when a system only exports files. MCP only for investigation and reviewed one-off operations, never as the scheduler.

Then fill assets/integration-spec-template.md, assets/flow-inventory-template.csv and assets/field-map-template.csv.

5. Design idempotency, ordering and retries

Vendor contracts with their check dates: references/delivery-reliability.md.

• Two keys: delivery (event ID) and effect (flow, source object, business version).
  • Claim the effect with a unique constraint before the side effect.
  • HubSpot's eventId is not unique.
  • Stripe can send two Event objects for one change: the effect key makes the second a no-op (refetch, compare the version, skip unchanged writes).
  • Never put a unique constraint on data.object.id plus type; .updated and invoice.payment_failed events repeat for one object.
• Ledger retention covers the longest retry, resend, replay or overlap window (IP03).
• A timeout after sending is an unknown outcome: read the destination by key before retrying.
• Arrival order is not business order: refetch current state or compare versions, and serialise writes per account.
• Classify errors by status and body: Marketo errors arrive in HTTP 200 bodies, Salesforce quota is a 403, GA4 Measurement Protocol accepts malformed events silently.
• Backoff is exponential with full jitter, a cap and a maximum, then an exception queue. Retry at one layer: nested attempts multiply to 10 or fewer.
• Honour Retry-After in the vendor's unit (RT05).
• Receivers verify the signature on raw bytes with the pinned scheme, store durably, then return 2xx, with p99 under half the sender's timeout (WH01). Never return 2xx to silence retries or dodge a ban.

6. Size the load

python3 -I "$SKILL_DIR/scripts/size_sync.py" --records 48000 --batch 100 --readback-calls-per-batch 1 \
  --preset hubspot-private-pro --remaining 600000 --reserve 0.2 --other-calls 50000

• Size the initial backfill, a normal day, and a replay after the longest outage you would accept.
• Stay inside remaining minus reserve minus other consumers, at 70% or less of the tightest shared window. Resets differ by vendor (RL03).
• Presets carry their check date; confirm live. Formulas and limits: references/rate-limits-and-backfills.md.

7. Plan backfill and recovery

Write these into the spec and runbook for a person to run after approval:

• Pilot 100 records (or 1%, whichever is smaller) covering every edge case and read back; then batches inside budget, parents before children, checkpoints only after durable writes.
• Pause or filter workflows, sequences and journeys that fire on create or update; a replay re-checks eligibility at the action (Zapier history replay never re-runs filters).
• Recovery sources and windows: references/operations.md.

8. Lint and test

python3 -I "$SKILL_DIR/scripts/lint_integration.py" flow-inventory.csv field-map.csv test-plan.csv

• FAIL means the design can reach the wrong customer, cause an unintended action, lose or duplicate an effect, write without authority or leave no recovery path. Fix it, or propose an exception with its owner for decisions.md, added only after the user approves. Columns and rule IDs: references/csv-formats.md.
• FO01 sees only the flows you pass: lint a redesign with every flow still writing the same fields, or list those writers in other_writers.
• Build the plan from assets/test-plan-template.csv: ten core cases for every flow, more by trigger and tier, each with evidence at four layers (references/test-plan.md).

9. Read back

python3 -I "$SKILL_DIR/scripts/reconcile_readback.py" --source expected.csv --dest destination.csv \
  --key account_id --map field-map.csv --flow billing_to_crm --baseline before.csv --dest-id hs_object_id

• Every owned field must be a column in both exports under the field map's internal names (HubSpot UI exports use labels); a missing one exits 2.
• Person flows: pii yes columns and email-like values print as hidden; add --hide-values for other personal data in a --baseline column. An email key exits 2.
• Pass: no missing, duplicate, mismatched owned field, null overwrite or mangled key; orphans and non-owned changes explained.

10. Operate and hand over

• Alerts:
  • Oldest unprocessed item older than the freshness target.
  • Exceptions: tier 1 any, tier 2 over 1% of a run.
  • Volume under 50% or over 200% of the median of the same weekday over the previous 4 weeks.
  • Quota over 80%.
  • A subscription disabled or deleted.
  • Credentials expiring within 14 days.
  • Read-back drift: tier 1 over 0.5%, tier 2 over 2%.
• Runbook from assets/runbook-template.md: contain, scope, reconcile, repair, bounded replay, verify, close. Service-owned credentials with a rotation date; pinned API versions.
• Changes leave as assets/change-list-template.md items. Propose martech-context/ updates; write nothing until approved.

Common failure modes

• Acknowledged before durable receipt. Detect: sender delivery count above stored receipts.
• Delivery ID used as the business key. Detect: two tasks or orders per episode key.
• Unknown outcome re-created. Detect: duplicates created seconds apart after a timeout.
• Global winner on a two-way sync. Detect: field history alternating between two sources.
• Silent subscription loss (Pipedrive deletes after three failing days; Klaviyo disables after 48 hours). Detect: daily status check and the zero-volume alert.
• Search recovery cut at 10,000 results. Detect: a 400 past 10,000, or a window stopping at exactly 10,000.

All 32, with mechanisms and the test that exposes each: references/failure-modes.md.

Worked example

Acme Analytics (fictional) hands over six flows (assets/example-acme-stack.md). From the skill folder:

python3 scripts/lint_integration.py assets/example-acme-flow-inventory.csv \
  assets/example-acme-field-map.csv assets/example-acme-test-plan.csv

• The lint ends 26 FAIL, 17 WARN, 25 PASS. The Stripe to HubSpot billing flow matches companies by domain and keys effects on arrival time (ID01, ID02, IP02, IP03 FAIL); the renewal date has two writers (FO01 FAIL), a decision for the owners.
• The n8n check ends 5 FAIL, 5 WARN, 1 PASS, 1 INFO: blind retries on Salesforce creates (IX01) and Continue after a failed lookup (IX02).
• The read-back ends 7 FAIL, 3 WARN, 2 PASS, 1 NOT CHECKED: a duplicate company, stale writes, a blanked renewal date and a key stripped of its leading zeros.
• The redesign, assets/example-acme-integration-spec.md, lints to 0 FAIL, 2 WARN on its own: WH01, and FO02 because the connector still writes the renewal date. Linted with lifecycle_sync it still shows FO01 until decision D-1 and CR-3 land. Commands, outputs, handling and change requests: references/worked-example.md.

Acceptance checks

• Owners: decision, tier, business owner, technical owner and (tier 1) on-call are filled. Pass: no blanks or placeholder text.
• Method (Design): the chosen method passes and each simpler one has its failed clause. Pass: method_rationale filled.
• Writers: the step 2 sample and bulk pull, with sample and window recorded in the review. Pass: no unlisted writer, or NOT CHECKED with the reason and the fallback evidence.
• Identity, ownership and blanks: ID, FO and RM rules pass, or each exception is recorded with an owner. Pass: lint shows no unexplained FAIL.
• Delivery: business keys, ledger retention, unknown-outcome handling, error classes, nested attempts and acknowledgement time are stated. Pass: IP, RT and WH rules pass.
• Budget (Design): initial, daily and replay loads fit the available budget. Pass: no size_sync.py FAIL; limits cited with URL and date; reset rule stated.
• Method and Budget (Review): each is in the review report as PASS, FAIL with evidence, or NOT CHECKED with the reason.
• Tests (Design): the test plan. Pass: every mandatory case for the triggers and tier, with expected evidence at source, transport, destination and downstream; no TP01; sandbox runs and the pilot read-back listed under Open decisions as owed before go-live.
• Tests and read-back (Review, Recover): sandbox evidence and the read-back output. Pass: mandatory cases ran in a sandbox with evidence; the read-back exits 0 with every owned field under Owned fields compared, or each FAIL is accepted by the owner; drift under the tier threshold.
• Operations: alerts with owners, a runbook with replay steps, service-owned credentials, pinned versions. Pass: OP rules pass.
• Safety: no write, replay, resend, subscription change or deploy in any vendor system; no credential in chat. Pass: the tool log shows vendor reads only, local writes only in the drafts folder, and test actions only for approved cases.

Boundaries

• Read-only. The skill never changes connectors, scenarios, syncs, webhook subscriptions, CRM properties, workflows or code, even when the connected identity could. A person applies approved changes, or an agent under agent-write-safety.
• Tests that write, in a sandbox or in production, run only after the user approves each case; production cases also need named test records and a cleanup list. Sandboxes differ: Stripe's sandbox retries webhooks three times over a few hours, not up to three days, and allows 25 requests per second, not 100.
• Consent, legal basis and retention belong to the privacy owner; the spec records them and checks removals reach every destination.
• A design is not proof. Runtime evidence comes from sandbox tests, vendor delivery logs and read-backs.
• Hand-offs, read-only where possible (references/handoffs.md):
  • HubSpot remote MCP with every write tool on approval; Salesforce SObject Reads server; Stripe MCP with a read-only agent API key; Klaviyo MCP with read-only=true; Customer.io MCP read scope; Fivetran MCP with FIVETRAN_SCOPE=read.
  • Review iPaaS from exports: n8n's execute_workflow runs production and Zapier MCP calls execute actions. Build after approval with n8n's, Make's and Zapier's own skills.

Live docs

Platform settings, limits and menu paths change. Before stating any parameter name, limit, quota or UI path:

• Fetch the current page from the vendor docs MCP server or llms.txt listed in sources.yaml, or open the canonical URL.
• If the live page disagrees with this skill, follow the live page and tell the user what changed.
• Cite the URL you used.

Most used here: the HubSpot, Stripe, Salesforce, Pipedrive, Klaviyo, Segment, Hightouch, Make, Zapier and n8n entries.