# Connection tests T1 to T6 Six tests per server per client, logged in [../assets/connection-test-log.csv](../assets/connection-test-log.csv) and summarised in the inventory. Per-vendor calls and caps checked on 2026-11-01, Google Ads, GA4, BigQuery and Amplitude again on 2026-11-06; tool names change, so take them from the live tool list. ## Results - Vocabulary: PASS, FAIL, UNPROVEN (with "because ..."), and WARN for T1 and T5 (with "because ..."). - A server is connected when T1 is PASS or WARN, T2 and T3 are PASS, and T4 is PASS or UNPROVEN with a reason. T5 is PASS or its limits are recorded; T6 is documented. - T1 WARN counts toward connected only when the identity is right and the extra reach has a dated narrowing request, listed under Open risks. - Results expire after 30 days or on any re-test trigger below. `scripts/mcp_inventory_check.py` enforces both. - Log columns: server, client, test, date, expected, observed, result, evidence, tester_role, notes. Evidence names the tool called and the screen compared, never a token or a customer record. - Test each client separately: one inventory block and one set of results per server and client. The same vendor can expose different tools or validation per client. - A client you are not running in (a colleague's Cursor, a ChatGPT workspace, a chat-app connector): give the user a numbered test script. For each test, the exact prompt to type, the expected value from `stack.md` or the vendor UI, and what to copy back: the tool the client called, the result and the screen compared. Log the answers with `tester_role` set to the person who ran them. A test nobody can run is UNPROVEN, with the reason. ## Definitions - T1 identity: the identity call returns the account ID in `stack.md`. A mismatch is FAIL. Reaching more accounts than the job needs is WARN; reaching another client's production account is FAIL. - T2 tool surface: the client's tool list matches the expected read-only surface. Take it from the client's server tool view (`/mcp`, connector settings, the ChatGPT plugin's tool list), not from the agent's own tool list, which leaves out tools a deny rule or block removed. Save the list as the drift baseline (`scripts/mcp_tools_diff.py`). A list that shows only an exec tool or discovery tools hides the real tools from T2 and from client blocks: switch to the vendor's tools mode first (PostHog `mode=tools`, Amplitude without `discovery=progressive`). - T3 known answer: a delivery metric (spend, impressions, clicks, sends, sessions) or a record field for a closed period, compared with the vendor UI. Thresholds below. - T4 denied operation: an out-of-scope read fails without a substitute answer, and a harmless write in the sandbox the user named is denied or the tool is absent. Rules for the write attempt: - Before each attempt, name the exact call, the target record and the sandbox, and get the user's go-ahead. - Send it through the same client profile and block rules as production: the same server name and deny rules, or the same connector re-authorised to the sandbox and back, with T1 re-run afterwards. - If the sandbox connection differs in a layer (another role, another server name, other block rules), record which layers T4 proved and which it did not. - If the write succeeds, T4 is FAIL. Stop testing that server, record the object created and hand its removal to the user; never delete it yourself. - Never test with a delete, send, launch, publish or spend change. No sandbox: UNPROVEN, listed as an open risk. - T5 completeness: a request larger than one page states pagination or truncation. - T6 revocation: who revokes, how, and one disconnect and reconnect during the pilot. ## Known answer thresholds (T3) - Closed period ending at least three full days before today, in the account's reporting time zone and currency. - Delivery metrics or record fields only: spend, impressions, clicks, sends, delivered, sessions, record field values. Conversions and attributed revenue keep moving inside attribution windows, so never use them for T3. - Exact match for spend (to the currency's minor unit), counts and record fields. - Analytics API against UI: exact match on the same definition. Any difference needs a named cause (thresholding, sampling, attribution setting, a saved dashboard filter not applied) before PASS; an unexplained difference is FAIL. - Search Console: `data_state` final; dates are Pacific Time and include both ends; compare web results with the Performance report set to Search type: Web. - Google Ads money arrives as `cost_micros`: divide by 1,000,000. - Write the expected value in the log before running the call. ## Evidence rules - A model that declines without calling a tool proves nothing about the permission layer. T4 needs an attempted call rejected by the system, or a tool absent from the T2 list or removed from the agent's tools by a client rule. Otherwise it is UNPROVEN. - A missing tool and a visible tool that returns permission denied are different results: record which one you saw. - Record the identity's role in production and in the sandbox. Roles in Mixpanel, Amplitude and Braze are set per project or workspace; if the sandbox role differs, T4 is UNPROVEN for the production layer. - For agencies or mixed roles, repeat T1 and T4 with a second user who has a different access boundary. - Do not infer a pass from an empty list, a missing file or an inaccessible connector. - Identity and known-answer checks use IDs, states, counts and amounts. When a tool returns customer names, emails or message text, do not repeat them in the reply, the log or the inventory. - The agent's own transcript is not a system of record. Where the source keeps one, confirm the session made reads only, and that the T4 attempt was denied, from it: - Snowflake: `SNOWFLAKE.ACCOUNT_USAGE.QUERY_HISTORY` filtered to role `MCP_READER` lists SELECT statements only (the view can lag by up to 45 minutes). - BigQuery: `INFORMATION_SCHEMA.JOBS` for the identity shows no DML or DDL jobs. - Mixpanel: the audit log shows no entry with Origin MCP (read and failed calls are not logged, so an empty log is the expected result). - Braze: the security event report shows the OAuth connection and no unexpected changes. - Claude Code: session transcripts, or OpenTelemetry with `OTEL_LOG_TOOL_DETAILS=1`, which adds MCP server and tool names to tool events. ## Per vendor HubSpot - T1: `get_user_details`, then `get_organization_details`. Portal ID equals `stack.md`; note the tools listed for this user. - T3: one known deal: owner, pipeline, stage, amount, currency, close date and the record link, compared in HubSpot. - T4: a deal owned by a team the user cannot see returns nothing, not a substitute. View users can still create notes, so the client block is the control. In Claude Code a deny rule on the tool name removes the tool from the agent's context, so no call can happen: record T4 as absent (removed by the deny rule) and check the rule in `/permissions`. Record a Blocked tool the agent cannot call the same way. If a client still lets the agent attempt the call, re-authorise the same connector to a HubSpot developer test account where the user has the same View-only role, ask for a note on a test record and expect the block; then re-authorise to production and re-run T1. - T5: a search over more than 200 deals states pages read and whether more remain (`search_crm_objects` returns 200 per page; `get_crm_objects` takes at most 100 IDs). Salesforce - T1: ask for the authenticated user and org and compare with Setup. Confirm the endpoint is sobject-reads (production or sandbox). - T3: one known opportunity (ID, stage, amount, close date), then a bounded cohort against a Salesforce report with the same filters and user. - T4: a record outside the user's sharing is not returned. The reads server exposes no create, update or delete tools (absent). - T5: a cohort larger than the stated limit reports retrieved count, total and unread records. GA4 - T1: `get_account_summaries` shows the intended property; count the other properties the identity reaches. - T3: sessions for one complete day at least three days old, no breakdown, against the same metric and property in GA4. - T4: a property the identity is not shared on returns an error. `create_property_annotation`, `create_custom_dimension` and `create_key_event` must be absent (no `--allow-write` or `ANALYTICS_MCP_ALLOW_WRITE`). - T5: a high-cardinality report with a row limit states the limit. Quota on standard properties (page updated 2026-10-26): 200,000 core tokens per property per day, 40,000 per property per hour, 14,000 per project per property per hour and 10 concurrent requests. Property quotas count every project and credential that reads the property, so an agent's reports draw on the same budget as dashboards. Mixpanel - T1: Get-Projects or List-Organizations: project ID and region match `stack.md`. - T3: Get-Report on one saved report, against the UI definition and value. - T4: a project the identity is not a member of returns an error; in a test project where the identity holds the same role as in production, a write tool is denied. - T5: Run-Query cannot filter or break down by cohort and does not support saved metrics: the answer must say so instead of substituting. 600 MCP requests per hour per user. Amplitude - T1: `get_amplitude_context`: user, organization and accessible projects. - T2: no `discovery=progressive` on the URL, so the list shows the tools themselves. Any `use_amp*`, `share_amp_entities`, `manage_*` or `run_amp` tool is off the client's allowlist or blocked. - T3: one saved chart through `query_amplitude_data` or `get_from_url`, against the UI. Charts on dashboards may use default chart settings, not the saved dashboard filters. - T4: in a test project where the identity has the same role as in production, a `manage_amp_*` call and one non-taxonomy write (creating a notebook or a dashboard) are both denied or blocked. Amplitude ties Use MCP (write) to tracking-plan changes, so a denied taxonomy call alone does not prove the other write tools are closed. - T5: a query beyond the response size reports truncation. PostHog - T2: in tools mode (`mode=tools` or the `x-posthog-mcp-mode: tools` header) the list shows individual tools. CLI mode, the default for Claude clients and Codex, lists one `exec` tool; if CLI mode stays, `readonly=true` is the only layer, and T1, T2 and T4 use the tool listing that `exec` returns. - T1: project pinned: switch-project and switch-organization tools are absent; one known insight belongs to the project. - T3: one saved insight against the UI. - T4: with `readonly=true`, create, update and delete tools are absent from the list. - T5: 480 requests per minute and 4,800 per hour per credential, including setup and tool listing; a 429 carries Retry-After. Google Ads - T1: `customers_list_accessible_customers` shows the customer ID in digits only (1234567890); note manager access through `GOOGLE_ADS_LOGIN_CUSTOMER_ID`. - T3: GAQL for a closed week: `cost_micros` divided by 1,000,000, clicks and impressions, against the UI in the account time zone and currency. - T4: with `labels: false` and `assets: false`, the list holds the three read tools and no `assets_` or `labels_` tool (absent). Record that the credential itself can write through other clients. A self-hosted Cloud Run deployment also passes an unauthenticated `tools/list` POST to `/mcp` that must return 401. - T5: a query with LIMIT must say it is the top N, not a total. Meta Ads - T1: ask for the ad accounts the grant reaches (tool names vary; check the live list), then confirm the `act_` ID, currency and time zone in Ads Manager. - T3: spend, impressions and clicks for a closed week, against Ads Manager with the same attribution setting. - T4: in a test ad account where the user has the same role and grant, renaming a paused test ad set is rejected under an `ads_read` grant. Without a test account: UNPROVEN, and write tools Blocked in the client. - T5: a request across many ad sets reports paging. Google Tag Manager - T1: Stape: `gtm_auth_status` `grantedScopes` holds only `tagmanager.readonly` (local CLI with a Read-permission identity). Export route: container public ID and version in the file. - T3: live version ID and name match the Tag Manager UI. - T4: in a test container the user names, after the user's go-ahead, one harmless create (`gtm_workspace` with action create, a workspace named t4-denied-test) fails with 403 under the read-only scope. Never call update, remove, revert, sync, publish or quickPreview, and never write to a production container. No test container: UNPROVEN, with `gtm_auth_status` showing only `tagmanager.readonly` recorded as the layer. - T5: follow `pagination.hasNextPage` until false; `totalItems` equals the items collected. Hand the audit itself to gtm-container-audit. Braze - T1: `get_workspaces`: the workspace ID (`app_group_id`) matches; name the workspace in prompts. - T3: one campaign's sends for a closed period (`get_campaign_dataseries`) against the dashboard. - T4: in a test workspace where the user has the same permissions, a harmless generally available write such as `create_content_block` is denied for a user without edit rights. Campaign beta tools can error for eligibility reasons: record the error text. - T5: a list larger than one page reports paging. Klaviyo - T1: `get_account_details`: account name and ID match. - T2: with `read-only=true` or `READ_ONLY=true`, write tools are absent. Leave `paginate-lists` off unless the client follows MCP cursors: with it on, a client that ignores `nextCursor` shows only the first page of tools (`list-page-size`, default 20). - T3: `get_flow_report` or `get_campaign_report` for a closed period with the store's conversion metric, against the UI. Pass on delivered counts, not attributed revenue. Reports built to match the UI can differ from raw metric aggregates: compare like with like. - T4: with `read-only=true` or `READ_ONLY=true`, write tools are absent from the list. - T5: compare a list count (flows, segments) with the Klaviyo UI and confirm the answer states its coverage. Intercom - T1: no tool returns the workspace on its own. Use a call that returns no customer content where one exists (for example `list_articles` with `per_page` 1, compared with the articles in the expected workspace). Otherwise `search` with object_type conversations and limit 1, `fetch` that one ID and use only its app link, which `search` does not return. Compare the workspace with `stack.md` and the endpoint region with the workspace URL (app.intercom.com or app.eu.intercom.com). Do not repeat the conversation's content. - T3: one known conversation's state and assignee. - T4: `add_internal_note` is blocked in the client, or the identity lacks Write conversations. A 401 means reconnect, not grant more. - T5: a search over many conversations states the retrieved count and any sampling. Customer.io - T1: `cio_auth_status`: accessible workspaces and granted scopes (read only). - T3: one known campaign or broadcast delivery count against the UI. - T4: a `cio_write_api` call without the write scope is denied. - T5: a list larger than one page reports paging. Search Console through the bulk export - T1: BigQuery `list_dataset_ids` shows the export dataset (default `searchconsole`) and only approved datasets. - T3: sum clicks from `searchdata_site_impression` where `search_type = 'web'` for a final week of `data_date` (Pacific Time), aggregating all rows because keys repeat, and compare with the Performance report set to Search type: Web for the same dates. Without the filter, image, video, news, Discover and Google News rows inflate the sum. - T4: `execute_sql` is denied by the IAM deny policy in a disposable dataset; an excluded dataset is not readable. - T5: results cap at 3,000 rows: aggregate in SQL. Search Console through a community server - T1: `list_sites` returns the exact `site_url` (`sc-domain:example.com` is a Domain property; `https://www.example.com/` is URL-prefix). - T2: `submit_sitemap`, `delete_sitemap`, `add_site` and `delete_site` are absent (`GSC_ENABLE_WRITES=0`), and auth status shows a read-only login. - T3: `query_search_analytics` with `data_state` final for a closed range, Pacific Time, search type web, against the UI with Search type: Web. - T5: Search Analytics exposes at most 50,000 rows per day per search type: report coverage. BigQuery - T1: `list_dataset_ids` returns only the approved datasets; record the job (billing) project separately from the data project. - T2: `execute_sql` and `cancel_job` still show in the list; mark each `"denied_by": "iam"` in the snapshot once T4 proves the denial. - T3: a known aggregate from the approved reporting model. - T4: an excluded dataset is denied, `execute_sql` with a harmless DML on a disposable table is denied by IAM, and `cancel_job` on a job ID that does not exist is refused by IAM, not answered as not found; `INFORMATION_SCHEMA.JOBS` shows no DML job for the identity. - T5: 3,000 rows at most and three minutes of processing by default. Dry run first; `execute_sql_readonly` takes no maximum bytes billed. Snowflake - T1: through the read-only SQL tool, `SELECT CURRENT_ROLE(), CURRENT_WAREHOUSE()`: the role is `MCP_READER` and the warehouse is the MCP warehouse. Any other role means the user's `DEFAULT_ROLE` is not `MCP_READER`: OAuth sessions run as the default role. - T3: a known metric through the Cortex Analyst tool over the semantic view. - T4: a write statement is rejected (`read_only`); a restricted table is not readable by the role; `QUERY_HISTORY` for `MCP_READER` lists SELECT statements only. - T5: SQL and generic tool responses truncate at 250 KB; 50 tools per server. ## Untrusted content test For servers whose tools return text written by outsiders (CRM notes, reviews, conversations, ad copy, search queries), ask the record owner to put a harmless synthetic instruction in a test record (the agent does not write it; its only write attempts are the T4 tests), then ask for a summary and check that the agent quotes it as content and does not act on it. Save the expected response and rerun it after any change to the model, instructions or tools. The full procedure is in agent-write-safety (its `references/untrusted-content.md`). ## Re-test triggers Rerun T1 to T4 and update the inventory on any of these: - 30 days since the last test. - The tools command reports an added or removed tool, or a tool that lost its block (D06). - A vendor changelog or release note mentions MCP tools, scopes or endpoints. - A scope reinstall or reconnect. - A role or permission change for the identity, or a change to client permissions (a deny rule removed, a Blocked category reset, VS Code's Chat: Reset Tool Confirmations). - A client upgrade, a new client or a new workspace. - An account, property, project or region move. - An employee or agency offboarding: revoke first, then test whatever replaces the connection.