The KES Score, anomaly detection, and what we don't do.
Every number on this site comes from public Shopify App Store data, updated daily, plus a parallel daily ChatGPT analysis. This page documents how the KES Score and anomaly detection work so writers, the Shopify team, and our users can check them.
1. What we monitor
Only public data from the Shopify App Store. We never use Shopify Partners data, paid signals, or your Shopify account. Two kinds of public data:
- Search-result rankings for every tracked keyword. Updated daily.
- Public listing pages, for every tracked app: title, subtitle, description, screenshots, pricing, Built for Shopify badge, rating, and review count. Updated daily.
2. The KES Score, in full
KES (Keyword Efficiency Score) is a per-keyword, per-app 0–100 score. Two scored signals are blended, then scaled by a search-volume multiplier and clamped to 0–100:
KES = (0.65 × rank_position + 0.35 × rank_trend) × volume_multiplier| Signal | Weight | What it measures |
|---|---|---|
rank_position | 0.65 | Non-linear function of current rank: a #11 (just off page 1) scores higher than a #28. Page boundaries matter more than absolute rank. |
rank_trend | 0.35 | Directional movement over the last 7 days, bounded so a single day can't dominate. Sustained upward motion scores higher than a one-day spike; downward motion scores lower. |
volume_multiplier | 0.7–1.3× | Bucketed estimated search volume (0.7× low · 1× medium · 1.3× high) so the same rank on a head term outscores it on a long-tail one. An estimated bucket, not a precise volume number. |
Bands surfaced in the dashboard: 80–100 excellent, 60–79 strong, 40–59 watch, 0–39 weak. The default sort surfaces gap opportunities first, keywords ranking page 3+ with page-1 trend velocity.
3. How anomaly detection works
Every morning at 06:00 UTC, after the overnight update, the detector runs across the keyword × app rank matrix. The logic:
- For each (keyword, app) pair, compute the rank delta from yesterday vs a 7-day rolling baseline.
- Count the share of tracked apps that moved by ≥3 positions on the same day.
- If the share exceeds 20%, flag the day as a possible Shopify App Store algorithm event and record it in our permanent anomaly log.
- Surface the same flag in every affected user's dashboard so a rank drop they didn't cause stops looking like one they did.
We log days that fall below threshold too — quiet days are a useful baseline. Severity bands: major ≥20%, minor 8–20%, info <8% (logged for record).
4. AI Visibility
Separate process, same cadence. Each morning we send a per-category prompt set to ChatGPT. We deliberately sample several times rather than pinning the model to a single deterministic answer — natural variance is the point. Each response is parsed for Shopify app mentions and matched against our index of known app handles.
Metrics: 7-day rolling mention count per app, share of voice within each category prompt, and the “framing” (whether your app appears as the recommended choice, a runner-up, or in a list).
5. What we explicitly do not do
- No paid signals. No Shopify Ads data, no install counts purchased from a third party, no Partners API.
- No private data. We read only what any visitor to the public App Store can see.
- No user PII in the public surfaces. The Market Intelligence page and anomaly log never reveal which app a specific user tracks.
- No silent edits. If we update a dated anomaly page based on later evidence, we append a dated note. The original body stays intact.
6. How we check our own claims
Every load-bearing claim about how the public store behaves is graded internally by how strong the evidence is, and each is re-checked whenever we change anything that could affect it. If you spot a claim on this site that contradicts a measurement you can reproduce, email us with the probe and we'll publish the update.
Cite this page directly: asomify.com/methodology. Last reviewed 2026-05-18.
See it in your dashboard
Start a free trial and see updated data in your dashboard each morning.