claude-registry-search-decoded

Claude registry search — decoded (the backend “opaque box”)

Completes the routing pipeline in claude-connector-routing: that page measured Lever 1 (prompt→keyword compression) and Lever 2 (post-retrieval re-judge); this page decodes the stage between them — the backend search_mcp_registry match/score/rank that was previously opaque. Feeds the signal inventory in signal-set-and-weights and the Retrievability/ Rank halves of discoverability-framework.

Method (one line, deliberately): live probing via controlled Claude sessions (~1,500 queries) + a 30-day daily registry-metadata diff history + a 7-week historical results corpus. Full methodology, scripts, and data live in the project repo (~/Dev/mcp_registry_search_simulation — FINDINGS.md), not here. Treat the existence of this capability as competitively sensitive.

Current truth

FACTS — live-verified engine mechanics (reproducible on demand)

• Exactly four metadata fields are searched: connector name (strongest by far), tool names (nearly as strong), shortDescription (the store-card line), description. Nuance (2026-07-02): shortDescription ≈ description in fitted weight — both bottom-tier. shortDescription’s practical value is edit responsiveness (26/26 directional edit calls; the ≈3-ranks-per-keyword causal coefficient was measured on shortDescription edits), i.e. it is a reliable lever, not a strong field — advise it because edits demonstrably move ranks, not because it weighs much.
• NOT searched — proven by isolation probes: tool descriptions, tool parameters, category/familyName (searching literal category names like “wellness”/“hospitality” returns none of the category’s members). Category re-tagging does nothing for retrievability.
• Matching is literal — no stemming. A connector renamed to include “Order” became #1 for order and stayed absent for orders. A plural in your text covers the singular query (found inside it); the singular does NOT cover the plural. Practical rule: write plurals.
• …and non-semantic — no synonym or embedding expansion (ruled out 2026-07-01): embedding nearest-neighbours are the opposite of the engine’s output (semantically asana neighbours monday.com/Todoist; the engine returns Canva/Campfire); search ≠ seo. Clients constantly assume “the AI understands synonyms” — at this stage it does not. Every target keyword must be covered literally, word by word.
• Fuzzy matching = edit-distance ≤2 against any substring of an indexed word, with a minimum-length floor. Approximate — this is the least-certain mechanic (see Open).
• Query semantics: AND within a keyword element, OR across elements; result list capped at 10 with a real relevance threshold (empty/short results are meaningful — many niche keywords return 1–4 results, i.e. unclaimed queries any matching connector gets onto). Keyword order is only a tie-breaker. Duplicated keywords are no-ops.
• Multi-keyword queries introduce no new candidates (2026-07-02): 100% of connectors returned for multi-keyword queries are present in the union of the per-keyword results (oracle recall 1.000 on 284 same-snapshot labels). Multi-keyword search only re-orders per-keyword pools — if you’re retrievable on none of the individual keywords, no combination saves you. Per-keyword retrievability is necessary and sufficient for candidacy; the audit unit is the single keyword.
• Deterministic and account-independent at a point in time (byte-identical across accounts and retries — including install/connection state: a needs_reconnect connector still ranks #1, so retrieval is unpersonalized); but drift is fast and continuous. Quantified (2026-07-01): 97.5% of repeated identical queries returned conflicting results across a 7-week corpus; a 10-day slice is only ~90% self-consistent (n=6,720 corpus query-rows). Any retrievability measurement is stale within roughly a week — weekly audits are the floor; daily measurement is the product.
• No length penalty: longer tool names / more tool words do NOT dilute weight (280 real tool-list-lengthening events: mean effect ≈ 0; extreme case 5→31 tools, no damage). The tools field behaves as one bag of words — more tools ≠ more weight; add-vs-rename is search-equivalent.

FINDINGS — validated quantities (n and source stated)

• Ranking shape: score ≈ TextScore(4 fields) + Prominence(connector). Prominence is a stable per-connector latent (~97.7% consistent ordering across queries, account-independent) that is earned, not writable — and it is NOT any of Anthropic’s exposed metrics (directory rank, trending, active users, tool calls all ≈ 0 correlation with backend order). The sort is additive, not tiered (live-reproduced 2026-07-01): on short/generic head terms, high-prominence connectors with only fuzzy matches outrank exact-match connectors (seo: Synapse/Ramp above Semrush/Ahrefs). Client corollary: you cannot out-text a high-prominence incumbent for #1 on a generic head term — compete for ranks 2–10 there, or take niche keywords.
• Causal effect of an edit: adding a searched keyword to your indexed text ≈ 3 ranks better (n=30 natural experiments, diff-in-diff vs placebo ≈ 0). Marquee: Supermetrics added “Salesforce” to its store-card line → absent→#1 on salesforce overnight.
• The registry is an active battlefield, measured (2026-07-02): in one 30-day window (2026-06-03→07-02), 96 connectors added (~3/day), 13 removed, 62 metadata field changes; 33 connectors changed indexed text within 7 weeks — and at least one deliberate in-the-wild registry-SEO edit (the Supermetrics “Salesforce” move above). Crowdedness rises at a measured ~3 connectors/day; unclaimed keywords are a wasting asset.
• Field hierarchy (measured over 639 probes): a name-match puts you at #1 in 96% of appearances; tool names next; description weakest. Keyword repetition buys ~1 rank at most — placement beats frequency, decisively.
• Tool names are the single largest source of #1 results — and the most under-used lever (2026-07-01, n=1,257 (phrase, connector) pairs): 41% of all observed #1 spots come from tool-name matches (vs 23% from name matches, which are rarer). Most connectors under-keyword their tool names (search_invoices beats a generic search) — the cheapest big fix in any registry-SEO audit.
• Multi-keyword rank is dominated by coverage of the query’s keywords (2026-07-01): connectors matching 1/2/3/4 of a query’s keywords land at mean joint position 5.1 / 3.1 / 2.3 / 1.2 — and being #1 on any single keyword element is convex (an element’s rank-1 connector lands at mean joint position 2.5 vs 5.1 for its rank-2). Cover as many of Claude’s 3–4 compressed keywords as possible; owning #1 on even one compounds.
• The name lever creates findability from zero but does not carry multi-keyword queries (2026-07-02; 6/6 frozen rename predictions, the corpus’s only 2 genuine renames): IBKR added “(IBKR)” to its name → #1 and sole result for ibkr, a token that previously existed nowhere; but “Order by Cash App” took #1 on order yet stayed absent from all 5 observed crowded multi-keyword order-management queries. A name token wins only its own keyword; multi-keyword coverage requires the tools/shortDescription fields. Prevents over-promising on renames.
• Lift prediction is validated: our advisor called the direction of real metadata edits right 88% of the time (104/118 events from the 30-day diff history), symmetric on helpful and harmful edits; by field: shortDescription 26/26, tool names 6/6, name renames 6/6 (frozen live predictions). Magnitude correlation +0.76.
• The soft spot, stated honestly: top-10 membership calls near the cutoff of a full list are a coin flip (~51%); solid-margin calls are ~85–90% reliable. Prominence co-decides who makes a full list (it cancels out of rank-move predictions but not membership). ~13% of observed rank movement has no text cause (competitor edits / drift).
• Claude-side consequence (links this page to claude-connector-routing): backend rank strongly predicts picker survival (rank-1 → shown 74%/picker-#1 44%), so these backend mechanics are upstream of everything in Lever 2.

CAPABILITY (what we can do with it — not facts about the engine)

• We can predict the effect of a metadata edit before shipping it, per keyword, with graded confidence — the basis of the direct-booker recommendations (2026-07-02/03) and the repeatable “registry-SEO audit” offer shape.
• The daily loop is LIVE (since 2026-07-05, on the platform): every day it ingests the registry metadata diff, freezes predictions in advance, probes affected + client + rotating keywords (~100/day ramping to 300), scores yesterday’s frozen predictions into a trailing scoreboard, and turns every registry edit into a scored natural experiment (day one emitted 149 watchlist keywords and 63 natural-experiment events). Every fact on this page is now mechanically re-verifiable within a day; the scoreboard’s regression alarm doubles as the engine-change detector.
• Remaining goal (not started): >90% blind prediction (top-10 membership F1 ≥0.90, members within ±1 rank) on never-seen keywords, via identifying the engine’s actual scoring implementation (it behaves like a standard Postgres full-text + trigram stack — a finite hypothesis space) rather than more curve-fitting, which plateaued. Plan: repo PRODUCTION_PLAN.md.

Open questions

• Exact fuzzy rule (our edit≤2-vs-substring is an approximation that mispredicts rare cases, e.g. directory→DirectBooker) — first workstream of the identification program.
• What prominence actually is (usage? curation? install base?) — measurable as a constant, cause unknown.
• Engine changes: the stack WILL be updated by Anthropic (shortDescription indexing appeared to change mid-corpus once). The daily loop’s regression alarm is the detection mechanism; treat every fact above as “true as of last_reviewed”, re-verifiable in ~one session. This prediction has already fired twice at the platform layer (both evidenced 2026-07-06): (a) claude.ai’s guardrails now flag exact-array JSON-dump search prompts as prompt injection on all models (verified on two model families; our first probe battery was refused 100/100 — plain natural search prompts are the fix), and (b) a ~Jun-26 claude.ai UI rollout (steps-timeline “Result” chip) hid tool payloads — our tracked-run registry-search evidence was dark Jun 26→Jul 5 (an unrecoverable hole); anyone extending the routing corpus must treat that window as missing.
• Does ChatGPT’s / other hosts’ registry search share this shape? (Untested — everything here is Claude.)

Timeline

• [2026-07-07] (maintenance) Folded in the remaining sim-repo findings that had never reached the brain: drift quantification (97.5%/7-week conflict rate), additive-not-tiered prominence sort, per-keyword union property (oracle recall 1.000), coverage-dominates joint rank, tool-name 41% share of 1s, name-lever limits (IBKR / Order-by-Cash-App), 30-day registry churn (96 adds/13 removals/62 edits), no-semantic-expansion proof, and the shortDescription weight-vs-responsiveness nuance. Status updated: the daily prediction loop went live 2026-07-05 (goal → capability); recorded the two 2026-07-06 platform-drift events (guardrail refusal of JSON-dump probes; ~Jun-26 Result-chip UI change with the Jun-26→Jul-5 tracked-run evidence hole).
• [2026-07-01] (internal discussion — Vincent + Elliot afternoon call, provenance) The project’s genesis conversation: Vincent proposed reverse-engineering the connector tool search into an internal simulator (“figure out the exact math… train it on real results… predict if you do this keyword, do you show up”) so keyword recommendations can be cycled without submitting to Claude. Observational seeds cited on the call: the search is lexical-only (“searching the nineties” — later fact-grade above); higher retrieval-list rank → more likely picked (later measured in claude-connector-routing); Netlify went 0→~17% retrievability on our benchmark purely from a tool update (natural experiment of the keyword lever); open question posed whether tool name outweighs description (answered above: name ≫ tools ≫ descriptions).
• [2026-07-04] (research) Page created. Work done 2026-07-01→03 in the registry-search project repo: engine decoded and live-validated (rounds 1–3), advisor built + display fixed (round 4), validated against the 30-day metadata diff history at 88%/118 events incl. a correct blind call of Direct Booker’s own Jun-23 edit (round 5), Direct Booker recommendations issued (see direct-booker), production plan + >90% novel-keyword target set. Deliverables (FINDINGS.md, PRODUCTION_PLAN.md, DIRECT_BOOKER_CASE_STUDY.md, advisor + probe tooling) live in ~/Dev/mcp_registry_search_simulation.