Benchmarks
Measured, not claimed.
Corpus parse, golden replay, and fixed-input transliteration throughput for the four Interscript runtimes, generated by checked-in scripts — never hand-edited. The parity audit-day table is kept frozen below as the baseline; this page tracks current HEAD.
Generated 2026-11-16T15:00:31.525Z by scripts/benchmarks/generate.mjs:
3 runs per leg, median aggregation, strictly serial (one runtime at a
time, nothing else executing). The TS leg reads a built dist, the rs leg reads
the release iscrs binary, and the ruby/py legs run in-process against sibling
checkouts of the four runtime repos and the map corpus — every tree's exact commit is
recorded below.
| Runtime | Repo / ref | Commit | Version |
|---|---|---|---|
| ts | interscript/interscript-ts @main | b74cfdaf56 | 5.10.0 |
| ruby | interscript/interscript-ruby @main | 4c47eb52db | 3.7.2 |
| py | interscript/interscript-ruby (py ISC engine tree) @wo17/py-override-placement | 5781a996fb | 0.1.0 |
| rs | interscript/interscript-rs @main | 74b4b8d071 | 0.3.2 |
the py ISC engine ships on this unmerged branch of the interscript-ruby repo; interscript-py main still carries the legacy .imp engine and cannot run the ISC corpus
Machine: Apple M1 Max, 10 cores, 64 GB, Darwin 23.1.0 (arm64); node v24.14.0, ruby 3.4.8, python 3.10.5. Corpus: 428 maps (maps@c55bc5c212); golden capture 12839 rows; fixed input set 514 rows.
Corpus parse
Parse every corpus map, median of 3
| Runtime | Median | Per-map mean | Parsed | Runs |
|---|---|---|---|---|
| ts | 15,268 ms (15.3 s) | ~35.67 ms | 428/428 | 16,106,15,027,15,268 |
| ruby | 167,428 ms (167.4 s) | ~391.19 ms | 428/428 | 167,428,178,408,149,517 |
| py | 4,678 ms (4.68 s) | ~10.93 ms | 428/428 | 5,077,4,678,4,504 |
| rs | 26,366 ms (26.4 s) | ~61.6 ms | 428/428 | 35,012,26,366,25,293 |
wall clock of the iscrs process (includes ~20 ms startup)
Golden replay
Replay the golden capture end to end
The cross-runtime contract is the RS golden capture (
interscript-rs tests/golden/isc-wf0.jsonl, 12839 rows). TS,
ruby, and py replay it in-process (1 unmeasured warmup pass, then 3
measured passes); rs runs its native single-process capture over the corpus' embedded
tests, which includes the parse.
| Runtime | Median | Byte-exact | Note |
|---|---|---|---|
| ts | 7,105 ms (7.11 s) | 12,839/12,839 | in-process jsonl replay |
| ruby | 7,605 ms (7.60 s) | 12,839/12,839 | in-process jsonl replay |
| py | 61,843 ms (61.8 s) | 12,836/12,839 (3 diverge) | in-process jsonl replay |
| rs | 66,568 ms (66.6 s) | 12839 rows captured | iscrs capture wall; translit-only ≈ 40,202 ms (40.2 s) after
subtracting the parse median
|
Divergent rows are golden output mismatches (deterministic across passes; an execution
error would fail the run outright rather than be reported). The three current py
divergences are the Ў rows of the three thin ALA-LC Belarusian maps (
gki-bel, mvd-bel, odni-bel) — the newest corpus
additions, whose exhaustive per-row tests ts and ruby replay byte-exact.
The byte-exact columns are a sanity signal, not the parity gate — conformance is enforced
in each runtime's own CI (rs golden_capture, ts test/golden-isc-wf0.test.ts, py test_golden_capture.py).
Transliteration throughput
Fixed input set (514 rows), median of 3
The input set is committed at scripts/benchmarks/fixtures/input-set.jsonl —
every 25th row of the golden capture, sampled once and then frozen so the series stays
comparable as the golden evolves.
| Runtime | rows/s | ms/row | Source |
|---|---|---|---|
| ts | 2,236 | 0.45 | in-process, fixture replay |
| ruby | 2,191 | 0.46 | in-process, fixture replay |
| py | 74 | 13.48 | in-process, fixture replay |
| rs | 193 | 5.18 | derived from the capture leg — rs has no batch CLI |
Audit-day baseline (frozen)
428-map corpus parse, 2026-11-13 audit
Quoted from the ISC runtime feature-parity audit (§A.8, macOS 14, apple-silicon, clean serial runs). This table is the audit-day baseline and is kept unchanged; the tables above track current HEAD and are regenerated by script.
| Runtime | Runs (ms) | Median | Per-map mean |
|---|---|---|---|
| ts (parseIsc) | 16,742.8 / 15,826.8 / 15,986.5 | 15,986 ms (16.0 s) | ~37 ms |
| py (parse_isc + resolver) | 77,839.3 / 80,709.3 / 81,024.7 | 80,709 ms (80.7 s) | ~189 ms |
| rs (iscrs parse, release, wall) | 81,410 / 79,220 / 77,490 | 79,220 ms (79.2 s) | ~185 ms |
| ruby (Interpreter.parse_isc) | 186,029.0 / 156,782.2 / 161,449.9 | 161,450 ms (161.4 s) | ~377 ms |
Audit-day postscript (WO18): py's generated parser was later closure-compiled — controlled before/after showed roughly 78 s down to about 22 s per corpus pass, byte-identical output. The py row above reflects the pre-closure audit run.
Reproduce
Exact commands
The scripts require sibling checkouts of the benchmarked trees — the map corpus (
maps), interscript-rs (with a release build: cargo build --release), interscript-ts (with a built dist/), interscript-ruby, and the py ISC engine tree (the wo17/py-override-placement branch of the interscript-ruby repo, whose Python
package lives at interscript-py/ inside it). Override any location with the INTERSCRIPT_MAPS, INTERSCRIPT_RS, INTERSCRIPT_TS, INTERSCRIPT_RUBY, and INTERSCRIPT_PY environment variables.
# one-time: freeze the fixed input set from the rs golden capture
node scripts/benchmarks/make-fixture.mjs
# regenerate every number on this page (clean serial runs, ~30-60 min)
node scripts/benchmarks/generate.mjs
# the output lands in src/data/benchmarks.json — never hand-edit it;
# commit the regeneration with the runtime changes it measures
Methodology follows the parity audit's timing protocol (
maps/docs/parity/README.md §A.8): median-of-3, in-process timing for the
interpreted runtimes, wall clock for the rs CLI legs, strictly serial.