Lobsters benchmark results

1. Roundhouse vs Rails — runtime × JIT

5 lanes, three of them the claim. Rails YJIT is the baseline; Roundhouse YJIT is the like-for-like comparison against it — same runtime, same JIT, so the only variable is the code; the AOT lane did not run. The 2 no-JIT rows size the JIT's contribution and are not the headline.

3.47× faster than Rails — 149.4 ms/iter (avg) vs Rails' 519.1, both stacks CRuby with YJIT. The full field:

Every lane replays the same frozen sequence in-process, in-memory, single-threaded — no HTTP server, no load generator — so the only variable across rows is the interpreter × JIT under test (the ruby-bench lobsters-for-YJIT shape). Lower is better; whiskers span each lane's min–max.

• Rails gains 1.69× from YJIT (876.4 → 519.1 ms/iter)
• Roundhouse gains 1.35× from YJIT (202.0 → 149.4 ms/iter)
• Roundhouse is 3.47× faster than Rails with YJIT (519.1 → 149.4 ms/iter)
• Roundhouse is 4.34× faster than Rails without a JIT (876.4 → 202.0 ms/iter)

Every lane queried SQLite 3.50.3 at runtime.

Two ratios, because one lane is not rendering what Rails renders. Roundhouse YJIT is at 25/26 content parity; over the 22 routes it renders equivalently it is 4.24× (108.0 ms/iter vs 458.1), against 3.47× over the whole sequence; Roundhouse no JIT is at 25/26 content parity; over the 22 routes it renders equivalently it is 4.90× (161.0 ms/iter vs 789.5), against 4.34× over the whole sequence. Both sides of the restricted division cover the same routes. The unrestricted number is not wrong, but part of it is earned by rendering less.

“vs Rails” is a speed ratio — Rails' ms/iter ÷ this row's, at the same JIT level — so 3× means three times faster. Memory is a separate section below. CV = per-lane standard deviation ÷ mean across the timed iterations.

Lanes measured 2026-10-12T09:12:28Z · host showcase.party — one process per lane, JIT pinned per lane.

2. Per-endpoint breakdown

Where the overall 3.47× win comes from — each endpoint's median latency over every timed visit of the frozen sequence, Rails vs roundhouse. Bars are log-scaled; the ratio at the right is Rails ÷ roundhouse (green = roundhouse faster).

A diagnostic breakdown, not a re-derivation: per-endpoint medians don't sum to the sequence median (GC and ordering don't decompose linearly), and the sub-millisecond rows sit near clock resolution — trust the ranking, not the third decimal.

3. Memory footprint

Peak resident memory of each lane's replay process — lower is better. Because the lanes replay in-process, each one measures itself (VmHWM from /proc/self/status), so this is the whole stack: interpreter, framework, the in-memory fixture DB, and whatever the timed run retains on top.

The bar splits at the point the sequence starts. The solid segment is the baseline — everything loaded and seeded, before a single visit. The translucent segment is growth across the timed run. The two say different things: baseline is what the stack costs to stand up, growth is what serving retains.

Roundhouse no JIT is at 25/26 content parity; Roundhouse YJIT is at 25/26 content parity. Peak memory is measured over the same sequence as the timings, so a lane that renders less also allocates and retains less — read its bar the way you read its ms/iter, not as a like-for-like footprint.

• Rails pays 35 MB (+11%) in peak memory for YJIT, buying 1.69× the speed
• Roundhouse pays 21 MB (+17%) in peak memory for YJIT, buying 1.35× the speed
• Roundhouse peaks 2.43× lower than Rails with YJIT (344 → 141 MB), and starts 1.77× lower before serving anything (183 → 104 MB)
• Roundhouse peaks 2.56× lower than Rails without a JIT (309 → 121 MB), and starts 1.72× lower before serving anything (177 → 103 MB)
• Across the timed run Rails retains 4.2× more than Roundhouse (160 MB of growth against 38 MB) — a separate fact from the baseline gap, and the one a peak-only number hides

“vs Rails (memory)” is peak RSS ÷ peak RSS at the same JIT level — not the speed ratio in the matrix table above; the two differ. Baseline is sampled after the in-memory fixture DB is seeded, and it is not a common floor across lanes — each stack loads its own framework, so subtracting one lane's baseline from another's peak is meaningless.

4. Coverage — HTTP 200 (26/26)

What the numbers above are numbers of: every distinct route the benchmark exercises returns 200 on the CRuby emit target — the timing sections measure a complete application, not a working subset. A failing cell here would invalidate that lane's rows above.

Status is a weak oracle, though — a 200 that renders half a page is still a 200 — so the bodies are also compared against Rails' as parsed documents. 25/26 routes render content equivalent to Rails. See the section below for what "equivalent" is defined to mean and what the remainder are.

5. Content parity vs Rails — Roundhouse YJIT 25/26, Roundhouse no JIT 25/26, Futamura YJIT 26/26

Every parity route is fetched from real Rails and from the emit, and both bodies are parsed into a canonical token stream and diffed there. That makes the comparison blind to four things that are not differences — indentation and formatting whitespace between block elements, attribute order, void-element spelling (<img> vs <img />), and character-reference spelling (' vs ') — and blind to nothing else.

- "Tue, 18 Jul 2023 05:29:46 -0500"
+ "Tue, 18 Jul 2023 10:29:46 +0000"

Each route's first differing hunk; - is Rails, + is the emit. Full hunks in ruby/content-parity.md.

6. Environment & provenance

Generated 2026-10-12T09:12:29Z from summary.json.