Interscript
GitHub ↗

The neural layer

Neural models for the
conversions maps cannot cover.

The 289 authority-backed maps cover romanization systems that a committee has published. Some conversions have no committee: restoring the haraqat a scribe left out, turning unwritten Thai into phonemes, reading Urdu aloud. Those are learned — and they ship under the same discipline as the maps: one artifact format, checksums verified on every load, byte-identical output from every runtime.

The catalogue

Every model, with its measured score.

The index resolves 23 models across five languages. No model is published without a measured metric next to its teacher's, on the same harness, in the open. Students are distilled from frozen teachers and gated at a pre-agreed error budget. Each entry resolves from the models.yaml index; artifacts above GitHub's 2 GiB cap ship as sha256-verified parts that the runtimes reassemble transparently.

Server tier fp32, full quality

  • diacritization ara-diac-2.0 Arabic → haraqat DER 2.29 · OOD WER 17.38 fp32 · 2.7 GiB · parts
  • translit khm-latn-1.0 Khmer → Latin CER 27.42 · EM 59.66 fp32 · 1.3 GiB
  • g2p urd-g2p-1.0 Urdu → IPA CER 14.77 · EM 33.6 fp32 · 1.3 GiB
  • diacritization urd-diac-1.0 Urdu → haraqat CER 3.74 fp32 · 1.3 GiB
  • diacritization heb-diac-1.0 Hebrew → niqqud DER 29.0 greedy · 17.46 beam-4 fp32 · 2.6 GiB · parts
  • g2p tha-g2p-base-1.0 Thai → IPA PER 9.19 (teacher 4.43) fp32 · 2.6 GiB · parts

Client tier distilled or quantized, for the browser and the edge

  • diacritization ara-diac-small-2.1 Arabic → haraqat DER 4.57 [1.91, 2.35] vs teacher (1.0 was 8.26) int8 · 264 MiB · fp16 · 674 MiB
  • diacritization ara-diac-layerdrop-1.0 Arabic → haraqat DER 5.78 · 63% of the parameters int4 · 95 MiB · browser-native
  • g2p tha-g2p-small-1.0 Thai → IPA PER 2.85 greedy (12.06 beam-4) int8 · 246 MiB
  • g2p tha-g2p-small-1.0-int4 Thai → IPA same student · 4-bit int4 · 193 MiB
  • diacritization heb-diac-small-1.0 Hebrew → niqqud DER 30.37 (teacher 24.79) fp32 · 1.3 GiB
  • g2p fas-g2p-1.0 Persian → IPA CER ≈1.6 · homograph 77.34% fp32 · 2.6 GiB · parts

“Releasing” = passing its parity gate now, entering the release pipeline. Distillation budgets and per-model provenance: interscript-ml/docs/RESULTS.md.

Frontier context, same instrument (SadeedDiac-25, the same 1,200 paragraphs under the same windowed evaluator): GLM-5.2 diacritizes Arabic at 2.51 DER with plain completion; the rest of the family does not — GLM-5.3-Flash at 8.57, GLM-5.3 at 9.98, glm-4.7-flash at 13.00 raw DER — and the loss is wrong vowels, not writing convention. Newer general-purpose models focus on other tasks and score worse here, so a small dedicated model remains the reliable way to do this work. Every row with its decode protocol and bootstrap CIs: rababa/docs/RESULTS.md.

Measurements

Our results against the frontier.

SadeedDiac-25 — all 1,200 paragraphs, Misraj's own evaluator, windowed zero-skip protocol, one instrument for every row. Our dedicated 580M teacher is the best non-vendor system measured on it, 3.2× better than the 1.5B model the benchmark was built around. And the frontier is moving the wrong way: every GLM successor to 5.2 is 3–5× worse, losing classical haraqat (wrong vowels on 10.05% of marked positions for 5.3-Flash, vs 5.2's 2.64% — which matches our 580M teacher's 2.62% to 0.02pp).

System Params DER (CE) DER (w/o CE) Protocol note
Claude-3.7-Sonnet — 1.3941 0.7693 vendor-published · protocol undisclosed · not reproduced by us
Ours · ara-diac-2.0 (r7) 580M 2.2864 1.3343 best dedicated model measured
GLM-5.2 — 2.5060 1.5537 our reproduction · raw
Gemini-Flash-2.0 — 3.1926 2.3783 same protocol
GPT-4 — 3.8645 3.8645 same protocol
Sadeed 1.5B 7.2915 5.2625 prior dedicated state of the art
GLM-5.3-Flash — 8.7978 6.6368 zero-skip · reasoning_effort=low
GLM-5.3 — 9.8971 7.8219 zero-skip · reasoning_effort=low
glm-4.7-flash — 13.2256 10.3206 zero-skip · thinking-disabled

WER columns, decode protocols, and paired bootstrap CIs for every row live in rababa/docs/RESULTS.md. The GLM-5.3 family cannot disable thinking; low effort is its plainest protocol.

The client ladder 300M student, identical architecture and artifact size

  • rung 1.0 AdamW · r6 labels · 3 ep 8.26 the shipped baseline
  • + Muon optimizer swap 5.29 −2.96pp
  • + r7 labels fresher teacher labels 5.08 −0.18pp (corrected 2026-09-05)
  • + 6 epochs ara-diac-small-2.1 4.57 −0.25pp · released

What didn't move it — every rung pre-registered, measured, and kept in the log: multi-token-prediction auxiliary 5.09 (+0.26pp as measured vs the then-published control — level with the corrected 5.08); classical-register swap at constant budget 5.81 (+0.98pp as measured; +0.72pp over the corrected control); register add at matched epochs 4.82 (statistically flat against news-only 4.57, intervals overlapping — it sits between the corrected control and the news-only rung); on-policy GKD 6.00 (+1.18pp as measured; +1.43pp over the rung it was meant to improve — the worst measured); depth-halved 5.78 (63% of the parameters — shipped anyway as the browser tier). At SFT convergence, supervision quality dominates. The ladder is closed. The control's originally published 4.8218 did not reproduce and was withdrawn on 2026-09-05; two independent re-measurements, one at the artifact level on the exact released bytes, agree on 5.08.

Hebrew, same discipline: 16.43% DER on Biblical Hebrew, where the modern-Hebrew state of the art degrades to 35.6% on the same test.

The instrument

How we measure.

Each of these rules exists because the naive version produced a wrong number that nearly shipped. Together they are why a number on this page means the same thing next month, on another machine.

  • Full evaluation sets, only. Five measured instances where evaluating on a domain-neighborhood subset overstated student quality — worst case 3.2× (a claimed 0.72pp gap that was 2.28pp on the full set).
  • The decode protocol is part of the number. Beam-4 with length normalization inflated a student's PER 4.2× over greedy on the same artifact (12.06 vs 2.85) — every published row names its decode.
  • Identical numbers must mean identical data. Label provenance is content-hashed into every run record; a reproduced figure is only evidence of reproducibility when the input pipeline is versioned.
  • CIs, not point estimates. Paired sentence-level bootstrap on every verdict, deterministic seed — deltas come with their intervals.
  • Checksummed end to end. The index ships with a sha256 sidecar, every zip member is verified on load, and a margins gate watches what the CER gate cannot see — silent quantization flips (keeping the output head in fp32 cut them 36× at +0.4% size).

The contract

One artifact. Any runtime.

A model is a zip — the Interscript Model Format, IMF v1. Anything that can read a zip, hash a file, and run two ONNX sessions can serve it; no Interscript training code required. Every member is sha256-verified against metadata.yaml on load; tampering raises loudly. The tokenizer is raw UTF-8 bytes — no vocabulary to download, no sentencepiece to drift.

  • Byte tokenizer — token id = byte + 3, trailing EOS
  • Decoder emits KV-cache graphs for streaming decode
  • opset 14, the floor set by the Ruby onnxruntime gem
  • Optional int8 / fp16 zips, precision-aware parity gates
tha-g2p-small-1.0-int8.zip
├── metadata.yaml     # id, metrics, parity, sha256 block
├── encoder.onnx
├── decoder.onnx
├── decoder-kv.onnx   # streaming KV-cache decoder
└── README.md         # model card

The crystals

Three runtimes, same bytes out.

The neural layer is served by secryst — the same cross-runtime contract as the map layer. Ruby, Python, and TypeScript resolve a model id against the index, fetch (or reuse the cache), verify every checksum, and decode with the KV-cache graph. Golden sets pin the three implementations to each other. 

# Ruby
gem install secryst

# Python
pip install secryst

# TypeScript / JavaScript
npm install secryst
require "secryst"
model = Secryst::Model.load("tha-g2p-small-1.0")
model.translate("สวัสดี")
# => "sa˨˩.wat̚˨˩.diː˧"

Point deployments at a mirror with SECRYST_INDEX; pin the cache with SECRYST_CACHE. Both are read at call time in all three runtimes.

Without installing anything: the same models answer the REST endpoint POST https://api.interscript.org/v1/infer (open CORS, no key), and run in the browser at the demo page. In a script with no build step, the map layer loads from esm.sh — full recipe in the package's docs/CDN.md.

Provenance

Every number is measured.

Teachers are frozen before distillation and never LLM-generated — language-model teachers hallucinate diacritics. Every number in the ledger links to a protocol in the open results log. BSD-3-Clause, code and weights. 

github.com/interscript/interscript-ml   # contract + index
github.com/secryst                     # runtimes + training