typeahead and race38

on sept 4, bsky38.com opened voting for "influential" bsky poasters

View post on Bluesky

its login and its account picker uses typeahead.waow.tech and since everyone has 10 votes to cast, that became a lot of actor searches!

note: the overwhelming reaction to scale: "oh boy!! this is fun!"

per the previous post from waow.tech, we did set out for scale

it seems we held up pretty well! search volume went from 163 to 5,423 requests per 15 minutes, while the engine's median stayed around 3–4 ms, even while traffic grew to >half of all time traffic1.

the first 12 hours

some stats on how the typeahead service fared during this time

this animation was created with astra, using leaflet html embeds

we recorded engine searches, alongside public vote counts.2

in recent months, our index has grown from 5.9M to 11.63M actors. the search backend still runs on 4 shared CPUs & 2GB RAM on fly.io

what changed since may

search now uses a prepared index plus recent updates, without a full database replica. turso keeps the actor records; my personal laptop builds the index at home, orchestrated by my Prefect server, and R2 delivers it to fly.

the index maps beginnings of names/handles to short lists of matching accounts, computed ahead of time. each search looks up those lists and merges in recent updates.

what broke during increased load

empty search results still triggered a fallback query to turso. during the race, those queries saturated the database, stalled ingest writes, and caused the backlog to drop events :( (only a lil bit)

after disabling that fallback, a second one triggered by engine restarts. search no longer falls back to turso; uncached requests during a restart can briefly return nothing.

the hosting footprint

removing the replica let me shrink the search volume from 100GB to 60GB. the index itself is 14GB; builds now run at home.

but honestly, my may proclamation was too optimistic: storage and cache memory still grow with the index.

lastly

race to 38 now exists to follow votes (uses typeahead.waow.tech)

https://bsky38.waow.tech/

thanks to @pds.dad and @dame.is for making this idea real!

resources

typeahead's stats · API docs · source

or at least, attributed traffic, since i started asking users to set X-Client in the docs

Saved recording: September 4, 17:00 to September 5, 05:00 UTC, 2026. Sources: Logfire engine spans and public race38 vote history. Public trace examples: race search (3.03 ms), later search (2.25 ms), and slow search (565 ms). Five-minute buckets; all 144 counts checked against a separate aggregate query. After 20:08 UTC, one-in-ten sampling makes counts estimates (≈); P95 uses recorded spans. Timing excludes the network and cache hits. Zero matches means an empty engine response, not a failed request; fallback may have found results. Portrait sizes emphasize votes with a readable minimum. Reconstructed history may omit withdrawn votes. Shared timestamps do not connect individual searches to votes.