Decision: public score = the better of benchmark vs custom/live
SUPERSEDED 2026-06-24 by 2026-06-24-self-contained-custom-arena. Building the dual score showed “public = better of two” mixes two non-comparable arenas on the public surface (a private-prompt number next to competitors’ benchmark numbers). Public is now benchmark-only; the custom score is a private, self-contained arena, never public. Framing below kept for history.
Set in the 2026-06-23 Vincent + Elliot call. Reverses the prior stance (decision 132 / the locked-score honesty work) that a brand’s custom/private score never becomes a public number. Has major product implications — the whole customer experience is built around it.
Current truth
- Two scores exist. Benchmark = authoritative, apples-to-apples across all apps in a category, comparative vs competitors, public. Custom / live = computed on the brand’s own defined prompts/topics — “winning the prompts that matter to you.”
- Publicly, we signal the BETTER of the two. On signup the public score = benchmark; once the custom/live score beats the benchmark, we publish the live one.
- In-product, a paying user’s score is computed on their own prompts. UX labeling: the score reads “Benchmark” until the brand adds prompts, then becomes “Live.”
- No public signal of who is a paying customer — the public number looks the same either way; it’s “just a number” that shows you’re winning discoverability (which is public information anyway).
- Still measured, never fabricated/estimated, shown only when fully scored. The honesty pin (no proxy/estimated score) is intact — this only relaxes which measured score can be public, not whether it’s measured.
Why / alternatives
- Why: avoid the bad signal of a brand scoring ~92 on its niche prompts but showing ~58 publicly (“your app sucks”). The score is the flagship marketing asset; it should signal “winning.”
- Rejected: showing both public + private scores side by side (confusing; would signal who’s a customer); benchmark-only public (penalises niche winners who legitimately dominate their sub-niche).
Consequences / risks
- Edge case: a niche brand games 2 prompts to 100 and “beats the category.” Accepted for now — needs category discipline; revisit if it bites.
- Requires the custom-score + add-prompts machinery (built) and the public profile to take
max(benchmark, live). - Supersedes the “custom never public” part of decision 132 / the locked-score stance; the public-benchmark-only rule is relaxed to better-of-two (still measured).
Timeline
- [2026-06-23] (call — Vincent + Elliot, recorded by Vincent) Decided. Mechanics in ado-score-model; framework + UI in discoverability-framework.