Lobsters benchmark results

1. Roundhouse vs Rails — runtime × JIT

5 lanes, three of them the claim. Rails YJIT is the baseline; Roundhouse YJIT is the like-for-like comparison against it — same runtime, same JIT, so the only variable is the code; Roundhouse AOT is the same emit through a different runtime, and is not at full content parity. The 2 no-JIT rows size the JIT's contribution and are not the headline.

3.65× faster than Rails — 124.5 ms/iter (avg) vs Rails' 454.1, both stacks CRuby with YJIT. The full field:

These ratios depend on run length. Roundhouse no JIT slowed 9.0% across its own timed run, so its ms/iter — and every multiplier drawn from it — would read differently at a different --time. Upward drift on the AOT lane is live-heap growth raising mark cost; downward drift on a JIT lane is warmup that has not finished.

Every lane replays the same frozen sequence in-process, in-memory, single-threaded — no HTTP server, no load generator — so the only variable across rows is the interpreter × JIT under test (the ruby-bench lobsters-for-YJIT shape). Lower is better; whiskers span each lane's min–max.

• Rails gains 1.66× from YJIT (754.6 → 454.1 ms/iter)
• Roundhouse gains 1.34× from YJIT (167.2 → 124.5 ms/iter)
• Roundhouse is 3.65× faster than Rails with YJIT (454.1 → 124.5 ms/iter)
• Roundhouse is 4.51× faster than Rails without a JIT (754.6 → 167.2 ms/iter)

Every lane queried SQLite 3.50.3 at runtime.

The AOT lane marks generationally — spinel's default collector since d3b1400d, attested by the binary's own counters rather than by what the harness exported. Roundhouse AOT: 6224 collections in the timed loop, 741 of them whole-heap.

Two ratios, because one lane is not rendering what Rails renders. Roundhouse AOT is at 22/26 content parity; over the 21 routes it renders equivalently it is 4.75× (88.9 ms/iter vs 421.9), against 4.84× over the whole sequence. Both sides of the restricted division cover the same routes. The unrestricted number is not wrong, but part of it is earned by rendering less.

Drifting lane: Roundhouse no JIT slowed 9.0% between the first and last quarter of its own timed iterations. A drifting lane's median depends on how long the run was, so its ratio is not a stable number — read it as a range, not a point. Lanes that hold steady drift under 1%.

“vs Rails” is a speed ratio — Rails' ms/iter ÷ this row's, at the same JIT level — so 3× means three times faster. Memory is a separate section below. CV = per-lane standard deviation ÷ mean across the timed iterations.

Lanes measured 2026-08-26T07:01:28Z · host showcase.party — one process per lane, JIT pinned per lane.

2. Per-endpoint breakdown

Where the overall 3.65× win comes from — each endpoint's median latency over every timed visit of the frozen sequence, per lane. Bars are log-scaled; the ratio at the right is Rails ÷ roundhouse (both CRuby +YJIT) (green = roundhouse faster). The chart draws rails, roundhouse, roundhouse AOT; the no-JIT lanes appear as extra columns in the table below.

Both headline ratios include the transpile. Isolating the two: transpiling accounts for the bulk of the win, and AOT compilation adds a further 1.33× over the same emit running on CRuby+YJIT — in aggregate only, since AOT is faster on the heavy routes and slower on the cheap ones, where per-request fixed costs dominate.

A diagnostic breakdown, not a re-derivation: per-endpoint medians don't sum to the sequence median (GC and ordering don't decompose linearly), and the sub-millisecond rows sit near clock resolution — trust the ranking, not the third decimal.

Read the AOT bars against 22/26 content parity, not in isolation. 4 of its routes render something other than what Rails renders (worst by differing tokens: hottest, u_michell_wiegand, recent, active), and a route that renders less is faster for a reason that is not speed. The lanes drawn beside it are at full parity. Per-route detail in the content-parity section below.

3. Memory footprint

Peak resident memory of each lane's replay process — lower is better. Because the lanes replay in-process, each one measures itself (VmHWM from /proc/self/status), so this is the whole stack: interpreter, framework, the in-memory fixture DB, and whatever the timed run retains on top.

The bar splits at the point the sequence starts. The solid segment is the baseline — everything loaded and seeded, before a single visit. The translucent segment is growth across the timed run. The two say different things: baseline is what the stack costs to stand up, growth is what serving retains.

Roundhouse AOT is at 22/26 content parity. Peak memory is measured over the same sequence as the timings, so a lane that renders less also allocates and retains less — read its bar the way you read its ms/iter, not as a like-for-like footprint.

• Rails pays 37 MB (+12%) in peak memory for YJIT, buying 1.66× the speed
• Roundhouse pays 20 MB (+17%) in peak memory for YJIT, buying 1.34× the speed
• Roundhouse peaks 2.55× lower than Rails with YJIT (346 → 136 MB), and starts 1.93× lower before serving anything (184 → 95 MB)
• Roundhouse peaks 2.67× lower than Rails without a JIT (309 → 116 MB), and starts 1.86× lower before serving anything (177 → 95 MB)
• Across the timed run Rails retains 4.0× more than Roundhouse (161 MB of growth against 40 MB) — a separate fact from the baseline gap, and the one a peak-only number hides

“vs Rails (memory)” is peak RSS ÷ peak RSS at the same JIT level — not the speed ratio in the matrix table above; the two differ. Baseline is sampled after the in-memory fixture DB is seeded, and it is not a common floor across lanes — each stack loads its own framework, so subtracting one lane's baseline from another's peak is meaningless.

4. Coverage — HTTP 200 (26/26 · spinel 23/26)

Incomplete coverage: 3 route(s) fail on the spinel AOT lane — the failing routes' visits short-circuit cheaply, so treat the affected lane's timing rows above as optimistic. Details in the table.

5. Content parity vs Rails — Roundhouse YJIT 26/26, Roundhouse no JIT 26/26, Roundhouse AOT 22/24

Every parity route is fetched from real Rails and from the emit, and both bodies are parsed into a canonical token stream and diffed there. That makes the comparison blind to four things that are not differences — indentation and formatting whitespace between block elements, attribute order, void-element spelling (<img> vs <img />), and character-reference spelling (' vs ') — and blind to nothing else.

Roundhouse AOT — 22/24

Compared against the same Rails oracle, by the same token-stream rules. Ordered by size of difference.

- <!doctype html>
- <html class="color-scheme-system" lang="en">
- <head>
+ "BENCHERR TypeError: can't convert nil into Integer"

- <!doctype html>
- <html class="color-scheme-system" lang="en">
- <head>
+ "BENCHERR TypeError: can't convert nil into Integer"

- is Rails, + is the emit. Full hunks in spinel/content-parity.md.

6. Environment & provenance

Generated 2026-08-26T07:01:29Z from summary.json.