EPR-0007: TypeScript and WebAssembly
enhancement proposal: proposed
on this record: read, decide, confirm
Read
Decide 0 of 8 settled
- 1. Where does the Rust side of the binding live?
- 2. What does a token built through the builder say in a column the parser left out?
- 3. What does `analyzeAll` return?
- 4. Which exception handling does the parser build use, and which browsers does that admit?
- 5. What does a caller see when matra's own code panics?
- 6. What is the package called on npm?
- 7. What does `with_reader` do for a format that already has a reader?
- 8. Is text detected as bytes are?
Confirm 2 to confirm
- Started
- 2026-11-18
- Pinned at
4cac941, the commit every code citation below opens at- Decides
- the owner, who alone sets the status to
accepted - Proposal PR
- #190
- Tracking issue
- #193
- Feature name
typescript_wasm
Summary
matra gains a TypeScript package for the browser and Node.js. It says what Rust and Python say: load an Engine, analyze an Input, then summarize or take keyphrases of the Document that comes back, typed from the published JSON Schema, with every failure carrying the library's kind and details. matra's core and its parser both run as WebAssembly, in a worker, so parsing never blocks a page. A spike found that this works: UDPipe's C++ compiled to WebAssembly parses matra's twelve example texts byte for byte as the native build does, in Node and in three browser engines, at about half the native speed, and the core compiled unchanged around it.
The spike also found what a package must settle that a spike could skip. The parser's identity was whatever the calling script wrote. Three pieces of the UDPipe adapter had to be copied. A corrupted model can crash the loader, natively too, and one corrupted model loaded without complaint. A panic in the core reaches the caller as an unexplained trap. And the parsing model is licensed for non-commercial use and served by a host that refuses requests from other sites, so the package cannot carry it and a page cannot fetch it from where it lives.
A design review of the crate at e07c790 found five more, in the surface a third language would copy. The reader and parser ports withhold what matra's own adapters use, so an outside reader cannot point a paragraph's spans back through markup and an outside parser copies the rule its spans are checked with. A document does not say which reader read it, and a second reader for a format replaces the first without a word. The method names are spelled again by hand on every surface. And text skips the format detection its documentation promises. One milestone before the core settles them.
This proposal answers each finding, the spike's and the review's, asks the owner eight questions, and lays out one more without a recommendation: who serves the model.
Nothing here is built. The roadmap entry that said UDPipe could not run in a WebAssembly sandbox is corrected in the same pull request.
src/extraction/mod.rs line 50, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
python/matra/types.py line 8, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
What changed in my understanding, 2026-11-19, since the text of 2026-11-18
- was
The public surface was nearly enough for a third language: the spike built a binding from outside the crate and needed three private pieces of the UDPipe adapter, which one crate-private function would replace.nowThe ports withhold from any outside reader or parser what matra's own adapters use, so a binding built inside the crate would hide the gap, not close it. A milestone before the core publishes a paragraph built from pieces and the span rule, and records which reader ran. A design review of the crate at e07c790 found it; the code cited under What a review of the ports found is the evidence.
- was
The TypeScript types were to be generated from the schema, and Python's would stay as they are.nowPython's
types.pyis generated from the same schema by the same step, and the method names come from one list in the core, because both are copied by hand today and a new method compiles while no binding can name it extraction/mod.rs:50 types.py:8. - was
Text and bytes followed Python's defaults, with detection for bytes only.nowWhether text is detected as bytes are is decision 8: the code skips detection for text while the documentation promises it, and a string in JavaScript is weaker evidence of text than a string in Rust.
Motivation
The owner wants TypeScript support, and then a playground on the docs site where a visitor pastes text and sees its structure with nothing installed.
The roadmap said the parser could not run there
The roadmap tells readers that "C FFI cannot run in a WASM sandbox", and that a WebAssembly parser would need a different parser or a server observed ROADMAP.md:63. EPR-0006 planned a spike to find out, ending in a go or a no-go. It ran between 2026-11-14 and 2026-11-18, in three stages, and the answer is a go:
ROADMAP.md line 63, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
- UDPipe 1's C++, compiled with Emscripten, gives output byte-identical to the native build over the twelve texts in
site/inputs/: 57 paragraphs, 6,529 words observed the spike. - matra's core compiled for
wasm32-unknown-unknownwith no change to its source, with that parser handed in through the constructor every outside parser uses observed lib.rs:365 the spike. Three documents came out token for token as the native command line writes them. - In the Chromium, Firefox and WebKit builds Playwright ships, on the main thread and in a worker, all 18 runs matched the native output observed the spike.
src/lib.rs line 365, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
What the spike had to work around
A spike may copy code and write its own labels. A package may not. Each item below is something the spike did that a published package cannot.
- It wrote the parser's identity by hand. The parser's name, model, digest and licence in a document's provenance were whatever the calling script passed: with nothing passed they were
null, and a script could have passed anything observed the spike. Natively the identity is derived from the model's bytes udpipe.rs:678. - It copied three pieces of the adapter. The rule that places a token's span, the function that calls it, and the mapping from one of UDPipe's words to a token are private to the crate observed domain.rs:805 udpipe.rs:768 udpipe.rs:778 grounds for decision 1, so the spike, written as an outside crate, carried its own copy of each observed the spike grounds for decision 1. Two copies of a rule drift.
- It patched a default. A token built through the public builder has an empty
depscolumn observed domain.rs:759 grounds for decision 2, where the adapter writes_observed udpipe.rs:803 grounds for decision 2. The spike set_by hand to match. - It found three calls that fail only when run.
Input::file,Inputs::dirandConfig::resolvecompile for WebAssembly and fail when called, the first two with "operation not supported on this platform" and the third because no home directory exists observed lib.rs:125 lib.rs:211 config.rs:152 the spike. - It found a model that crashes the loader, and one that does not. A model whose first byte is changed to
0xffthrows a C++ exception that nothing in UDPipe catches observed the spike. The native build does the same, and the process ends with exit code 134 observed the spike. Of 41 corrupted models, 39 failed cleanly, one crashed as above, and one, with a single byte changed at offset 40, loaded and parsed observed the spike. - It saw a panic arrive with no explanation. A panic in the core reaches JavaScript as
RuntimeError: unreachable, with no message, and the instance it left behind later failed in a different way observed the spike grounds for decision 5. - It served the model from its own machine. The model is CC BY-NC-SA 4.0, a separate work from matra with its own terms observed udpipe.rs:44 udpipe.rs:41, and its host answered 403 to a request carrying an
Originheader and 200 to the same request without one, measured withcurlon 2026-11-18 observed udpipe.rs:36. A browser always sends that header to another site, so a page cannot fetch the model from where it is published.
src/domain.rs line 805, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
src/nlp/udpipe.rs line 768, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
src/nlp/udpipe.rs line 778, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
src/domain.rs line 759, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
src/nlp/udpipe.rs line 803, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
src/lib.rs line 125, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
src/lib.rs line 211, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
src/config.rs line 152, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
src/nlp/udpipe.rs line 44, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
src/nlp/udpipe.rs line 41, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
src/nlp/udpipe.rs line 36, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
What a review of the ports found
A design review of the crate at e07c790 read the ports as an outside implementer meets them. Five of its findings bear on a third language, because a binding copies whatever the surface it binds gets wrong. Each claim below rests on the code it cites, at the commit this record is pinned to, whose src/ and python/ are the reviewed commit's.
- An outside reader cannot do what matra's markdown reader does. A reader outside the crate can emit only a paragraph whose text is the analysed text byte for byte. The constructor for a paragraph whose text maps back through dropped markup, and the map it takes, are crate-private, "which only matra's own markdown reader builds" observed decompose/mod.rs:179 decompose/mod.rs:192 decompose/mod.rs:280 decompose/mod.rs:138. The contract tells a reader that drops markup to name the text it extracted as the analysed text instead, and the digest in provenance is then the digest of that text, not of what the caller passed observed decompose/mod.rs:41-43 lib.rs:567. An HTML reader's spans then point into a text nobody else holds.
- An outside parser copies the rule its spans are checked with. The function that says whether a token's span lines up with its form is crate-private, and it is both what the UDPipe adapter places spans with and what the engine checks every parser's spans with observed domain.rs:805 udpipe.rs:771 conform.rs:224 grounds for decision 1. The spike was the first outside parser, and it carried a copy.
- A reader leaves no trace. The reader trait has one method, and provenance records the parser, the format and the source but not the reader observed decompose/mod.rs:73-76 domain.rs:1904 grounds for decision 7.
with_readerreplaces whatever reader the format had, the one matra ships or one the caller registered a line earlier, and says nothing observed lib.rs:421-422 decompose/mod.rs:251-253 grounds for decision 7. - The method names are spelled again on every surface. Each method's name is crate-private, with no public way back from a name to a method, and the names are written out again in the configuration, the Python binding, the command line and Python's types observed extraction/mod.rs:50 config.rs:52 outcomes.rs:33 cli/mod.rs:1068 types.py:523. The selectors are
#[non_exhaustive], and each string copy matches with a fallback arm, so a method added in the core compiles everywhere while no binding can name it until someone edits every copy inferred. Python's types are declared by hand, and its tests hold the top-level keys of a few of them observed types.py:8 test_corpus_conformance.py:40 test_extension_points.py:406; a nested field that drifts passes them. - Text skips the detection its documentation promises. The
Inputdocumentation says a PDF or DOCX signature is the strongest evidence of a format, before the format the caller names observed lib.rs:67-68 grounds for decision 8, but text goes straight to the reader for the format it was given, while bytes and files are detected first observed lib.rs:544-546 lib.rs:603 grounds for decision 8. No law compares text with bytes of the same content.
src/decompose/mod.rs line 179, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
src/decompose/mod.rs line 192, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
src/decompose/mod.rs line 280, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
src/decompose/mod.rs line 138, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
src/decompose/mod.rs lines 41 to 43, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
src/lib.rs line 567, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
src/domain.rs line 805, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
src/nlp/udpipe.rs line 771, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
src/conform.rs line 224, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
src/decompose/mod.rs lines 73 to 76, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
src/domain.rs line 1904, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
src/lib.rs lines 421 to 422, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
src/decompose/mod.rs lines 251 to 253, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
src/extraction/mod.rs line 50, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
src/config.rs line 52, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
src/python/outcomes.rs line 33, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
src/cli/mod.rs line 1068, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
python/matra/types.py line 523, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
python/matra/types.py line 8, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
python/tests/test_corpus_conformance.py line 40, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
python/tests/test_extension_points.py line 406, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
src/lib.rs lines 67 to 68, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
src/lib.rs lines 544 to 546, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
src/lib.rs line 603, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
Why now
0.3.0 settled the surface so that a fourth language could copy it once. The copy is cheapest while the three existing syntaxes are fresh and nothing in TypeScript has been published, because a name on npm is as permanent as one on crates.io.
Guide-level explanation
The same API in a fourth syntax
import { Engine, Input, summarize, keyphrases } from "matra";
const engine = await Engine.load({ model: "/models/english-ewt-ud-2.5-191206.udpipe" });
const doc = await engine.analyze(Input.text("The river rose. The bridge held."));
doc.metrics.total_words; // a field, typed from the published schema
doc.provenance.parser.sha256; // the model that produced this parse
const summary = await summarize(doc); // three sentences, as in Python
const phrases = await keyphrases(doc, { n: 5 });
engine.close();
| Outcome | Rust | Python | TypeScript |
|---|---|---|---|
| Build the engine | Engine::new() | Engine() | await Engine.load({ model }) |
| Analyze one | engine.analyze(Input::text(t, format)) | engine.analyze(t) | await engine.analyze(Input.text(t)) |
| Analyze many | engine.analyze_all(inputs) | engine.analyze_all(texts=...) | engine.analyzeAll(inputs) |
| Summarize | summarize(&doc, n) | summarize(doc) | await summarize(doc) |
| Keyphrases | keyphrases(&doc, n) | keyphrases(doc) | await keyphrases(doc) |
The names are the same words in TypeScript's case. What differs follows from where the code runs:
- Every call returns a promise. The parser runs in a worker so that a page stays responsive, and a message to a worker is answered later.
- The caller says where the model is.
modelis a URL the page may fetch, or the bytes themselves. The package holds no model and no default address (Who serves the model). - A
Documentis data. It is the JSON the command line prints, as an object, with no methods. Sosummarizeandkeyphrasestake a document from anywhere: this engine, a filematra analyze --jsonwrote, a server running Python. A page that only summarizes documents parsed elsewhere never downloads the model. - There is no
Input.file, no directory input and no configuration. A browser has no file system and no home directory. They are absent from the types, so a call to one fails when the code is compiled, not when it runs. A Node caller reads the file and passesInput.bytes, and so does a page given a file: its bytes, notfile.text(), which turns what it cannot decode into replacement characters without an error. - There is no
clustersyet. Embeddings have not been measured in WebAssembly.
When something fails
A failure the library reports rejects with a MatraError, which carries the same three facts as the JSON envelope and the Python exception: kind, message and details.
try {
await engine.analyze(Input.bytes(upload));
} catch (e) {
if (e instanceof MatraError && e.kind === "input_too_large") {
show(`Too large: ${e.details.actual} bytes, limit ${e.details.limit}`);
}
}
A model that is not the one matra pins is refused before it is loaded, with model_invalid. There is no way to load another.
What it costs a page
| What | Gzipped | Notes |
|---|---|---|
| The parser, compiled | 228 KB | fetched only when an engine is loaded |
| matra's core | 659 KB to 760 KB | the lean and the full spike builds; the size of the package's own build is unmeasured |
| The model | 16.3 MB | does not compress; about 95% of the download |
Loading the model took about one second and left the worker holding about 181 MB, which it keeps until engine.close() ends the worker. A 3,347-word essay took about two seconds to analyze, and a page-length text 61 to 69 ms. What the spike measured has every number and how it was taken.
Reference-level explanation
The pieces
The pieces at run time: the page holds a thin wrapper, one worker holds both modules
Proposed: the caller talks to the package; the package talks to one worker
show the data
| state | drawn | element | id | label |
|---|---|---|---|---|
| proposed | proposed | box | caller | Caller's code page or Node |
| proposed | proposed | box | wrapper | matra package Engine, Input, outcomes, errors |
| proposed | proposed | box | types | Types from schema 2.x |
| proposed | proposed | box | core | matra core Rust, wasm32 reads, checks, measures |
| proposed | proposed | box | parser | UDPipe 1, C++ Emscripten build |
| proposed | proposed | box | model | Model bytes digest checked before load |
| proposed | proposed | arrow | caller>wrapper | caller to wrapper |
| proposed | proposed | arrow | wrapper>types | wrapper to types |
| proposed | proposed | arrow | wrapper>core | wrapper to core |
| proposed | proposed | arrow | core>parser | core to parser |
| proposed | proposed | arrow | model>parser | model to parser |
4cac941Two WebAssembly modules, as the spike built them. The parser is UDPipe 1 compiled by Emscripten. The core is matra itself, compiled for wasm32-unknown-unknown without the udpipe feature. The core calls the parser once per paragraph through a function the wrapper hands it, and checks what comes back exactly as it checks any parser: the calls to conform::read and conform::parse sit in the engine, not in an adapter observed lib.rs:562 lib.rs:576, and the spike saw them fire on a word not in the text and on a head out of range. The size cap holds for the same reason.
src/lib.rs line 562, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
src/lib.rs line 576, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
The Rust side
The binding is a module of the crate behind a non-default wasm feature, as the Python binding is a module of the crate behind python observed lib.rs:865-866 grounds for decision 1 (decision 1). Working names, fixed when built:
src/lib.rs lines 865 to 866, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
src/wasm/exports what the wrapper calls and nothing else. It is held to boundary rule 7 assrc/python/is: the public surface,configanddomain, never a port, an adapter or a crate-private module. The rule gets a semgrep check and a fixture with it.- A second adapter of the parser port, beside the UDPipe one, behind the same feature. It holds the function the wrapper supplied and the identity the pin gave it, and implements
NlpProvider. The composition root wires it, in one constructor compiled only underwasm. - One copy of the mapping. The UDPipe adapter does three things after the C++ call returns: it groups words into sentences, builds each token, and walks one cursor through the paragraph to place each span. Those three move, unchanged, into one crate-private function over plain column values. The native adapter turns each
udpipe_rsword into those values and calls it; the new adapter turns what the worker returned into the same values and calls it.nlp/udpipe.rsstays the only file that importsudpipe_rs. The span rule inside the walk is not part of this function: it is public by then, asSpan::of_formunder The ports and the names, because every parser needs it. What stays crate-private is the grouping and the building, whose shape is UDPipe's words. - The pin out from behind
udpipe. The model's name, size, digest and licence are constants in the UDPipe adapter, which theudpipefeature compiles observed udpipe.rs:20 udpipe.rs:25 udpipe.rs:28. They move to where both builds compile them, so the WebAssembly build checks the same digest from the same line of source.
src/nlp/udpipe.rs line 20, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
src/nlp/udpipe.rs line 25, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
src/nlp/udpipe.rs line 28, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
The parser trait requires Send , and the spike's adapter met it with no unsafe of its own only because wasm-bindgen marks a JavaScript value Send on builds without threads observed nlp/mod.rs:94 the spike. A threads build was not tried and is not proposed.
src/nlp/mod.rs line 94, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
The ports and the names
Settled before the core is compiled for a new target, because each binding copies what it binds (milestone 2). EPR-0006 made the ports public so that a caller could plug in a reader or a parser of their own, checked before it reaches a document; this gives that caller what matra's own adapters already have. Working names, fixed when built:
impl ReadBlock {
// A paragraph whose text is its pieces' texts, in order.
pub fn assembled(kind: ParagraphKind, span: Span, pieces: impl IntoIterator<Item = TextPiece>) -> ReadBlock;
}
impl TextPiece {
pub fn verbatim(text: impl Into<String>, from: Span) -> TextPiece; // the analysed text at `from`, byte for byte
pub fn whole(text: impl Into<String>, from: Span) -> TextPiece; // stands for all of `from`: an entity, a tag
}
impl Span {
// Where `form` lies in `text`, from the first byte at or after `from` that is not whitespace.
pub fn of_form(text: &str, from: usize, form: &str) -> Option<Span>;
}
pub trait Decomposer: Send {
fn decompose(&self, input: &[u8]) -> Result<Read>;
fn identity(&self) -> Identity; // new, required, as the parser's is
}
impl SummaryMethod { // KeyphraseMethod the same
pub const ALL: &'static [SummaryMethod];
pub fn name(self) -> &'static str; // "tfidf", as provenance.method records it
pub fn named(name: &str) -> Option<SummaryMethod>;
}
impl Format {
pub fn name(&self) -> &str; // the crate-private wire_name, made public
}
- A paragraph from pieces. An HTML reader reads
<p>The <b>river</b> rose & fell.</p>. The parser must readThe river rose & fell., and every span should point into the HTML the caller passed, so thatprovenance.source.sha256is the digest of that HTML and a token found in the parse lands on its bytes. The reader emits a paragraph at 0..40 from five pieces:Theverbatim from 3..7,riverfrom 10..15,rosefrom 19..25,&whole from 25..30 andfell.from 30..36. The text is the pieces joined, so the text and its map cannot disagree. What is left to check, that each piece comes from inside the paragraph's span and a verbatim piece is the analysed text it names, the engine already checks on every paragraph, with the reasonparagraph_text_mismatchobserved conform.rs:82. matra's markdown reader already builds its maps a piece at a time, each a text, where it came from and whether it is verbatim observed markdown.rs:285, so it moves to the public constructor andReadBlock::mappedleaves the crate: the reader matra ships is the constructor's first user.TextMapstays crate-private, so how a map is held can change. Naming an extracted text as the analysed text stays the rule for a binary format, where there is no source text to point into. - The cap on what reaches the parser. A paragraph built from pieces is not a slice of the analysed text, so the analysed text's length no longer bounds it: a whole piece may stand for five bytes and carry a megabyte. The engine therefore holds the paragraphs' texts together to
MAX_INPUT_BYTES, failing withinput_too_largeand thewhatofinput, so the law that no oversized input reaches the parser holds for a reader matra did not write. - The span rule, public.
Span::of_formis the UDPipe adapter'slocateudpipe.rs:768 made public, and the adapter calls it in place of its own. The rule it applies,form_end, stays crate-private, withconform::parsestill checking every parser's spans by it. A span a parser places withSpan::of_formis therefore a span the engine accepts, and one placed any other way is still checked. The parser contract names it besideToken::builder. Of the three pieces the spike copied, this is the one every parser needs; the other two, grouping and building, stay crate-private under The Rust side. - Which reader ran.
Decomposergainsidentity, required, returning theIdentitythe parser and the embedder report. EPR-0006 gave those two one shape so that "one reader reads both" observed EPR-0006:516, and a third use keeps it. A reader with no model reports its name alone, and the parser's rule applies: two that can read the same input differently "must not report the same identity" observed nlp/mod.rs:103-104, so an outside reader's name carries its version. The engine records it inprovenance.readeron every document, and a derived result copies it with the rest. matra's own readers reportmarkdown,plain_textandblocks, andInput::blocksreports the block reader's, since that reader's code assembles the blocks. Like every field of provenance it is a record of what a component said, not proof. - Schema 2.1. A minor version adds optional fields only, and a reader of
2.0accepts a2.1document observed spec/README.md:89. SoSCHEMA_VERSIONbecomes2.1, the schema listsreaderunder provenance without requiring it, and the Rust field is anOptionthat isNoneonly on a document written before 2.1, so a saved2.0document still reads back. The conformance fixtures that carry a whole provenance pin its schema version observed parity.json:26: the parity, outcome and command-line fixtures are regenerated with2.1and a reader, and the runners that validate with every object closed fail until the schema lists the field, which keeps the two in step. The generated types of milestone 1 pick the field up by regeneration, which is why that milestone comes first. - Registering a reader twice. What
with_readerdoes for a format that already has a reader is decision 7. - One spelling of each name. Each selector has one public list and a mapping each way,
nameandnamed, with the list written beside a match that has no wildcard arm, so a new variant fails to compile until it has a name and a place in the list. Routing an error to its Python class is held the same way observed errors.rs:102. The configuration's check, the command line's--method, Python'smethod=, the method caps'whatlabels and the TypeScript binding call it, and none of them writes a name.nameis whatprovenance.methodrecords, so the name a caller passes is the name the result carries back. Formats have one direction public already,Format::named;Format::namemakes the other public domain.rs:2291, and the command line's--formatand Python'sformat=parse throughFormat::namedand refuse a format this build has no reader for, rather than each listing the names. - Python's types from the schema.
python/matra/types.pyis generated fromspec/schema/v2/by the step that generates the TypeScript types, committed, and held by the same check that regenerates both and compares (milestone 1). Its prose moves into the schema'sdescriptions, so a TypeScript reader gets the same text and the published schema says what each field means. What the schema does not describe stays hand-written beside it: theEmbedderprotocol, and the unions of method and format names, which the schema leaves out becauseprovenance.methodis an open set there. Each binding's tests compare its unions with the list the core exports, in full. Generated rather than checked: Checktypes.pyagainst the schema. - Text and bytes. Whether
Input::textdetects the format asInput::bytesdoes is decision 8. Either way a law joins the seven equivalence laws: text and bytes of the same content read to the same document, for every text if the decision says (b), and for every text without a signature if it says (a).
src/conform.rs line 82, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
src/decompose/markdown.rs line 285, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
blueprints/proposals/0006-the-0-3-0-surface.md line 516, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
src/nlp/mod.rs lines 103 to 104, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
spec/README.md line 89, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
spec/tests/parity/parity.json line 26, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
src/python/errors.rs line 102, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
Parser identity and the digest
The caller states where the bytes are. Nobody states what they are. The worker:
- Refuses bytes whose length is not the pinned size, and stops reading a response once it runs past that size.
- Computes the SHA-256 of its own copy of the bytes and compares it with the pin, in the core, beside the constant.
- Only then hands that same copy to the C++ loader. Nothing is fetched or read a second time.
- Builds the identity from the pin: the name, the digest and the licence the native adapter reports for the same bytes.
A mismatch rejects with model_invalid and nothing reaches the loader. That closes both findings of the corrupted-model run at once: the model that crashes the loader and the model that loads silently are each refused by the digest, and no other check tells the second from a good model inferred grounds for decision 4. It is also what the native side does: the function that verifies the model returns the verified bytes, and those bytes are what load observed udpipe.rs:658 udpipe.rs:136.
src/nlp/udpipe.rs line 658, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
src/nlp/udpipe.rs line 136, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
Accepting only a pinned model is not a new rule. 0.3.0 removed loading a model file of the caller's choosing from Rust and Python, because "the model is a pin" observed EPR-0006:408. When the permissively licensed model arrives as a second pin, the digest of the bytes says which pin they are, so TypeScript needs no id parameter.
blueprints/proposals/0006-the-0-3-0-surface.md line 408, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
The digest is computed by the core's own SHA-256, the function that already computes provenance.source.sha256, fed in pieces so the core's memory does not grow by the size of the model. The spike used the browser's built-in digest instead and measured 10 to 26 ms; that function exists only on secure origins, which a page served over plain HTTP on a local network is not. The core's own speed over 16 MB is unmeasured.
The TypeScript surface
class Engine {
static load(options: { model: string | URL | ArrayBuffer | Uint8Array }): Promise<Engine>;
analyze(input: Input): Promise<Document>;
analyzeAll(inputs: Iterable<Input> | AsyncIterable<Input>): AsyncIterable<Document | DocumentError>;
close(): void;
}
class Input {
static text(text: string, options?: { format?: Format }): Input;
static bytes(data: ArrayBuffer | Uint8Array, options?: { format?: Format }): Input;
static blocks(blocks: Block[]): Input;
}
function summarize(doc: Document, options?: { n?: number; method?: SummaryMethod }): Promise<Summary>;
function keyphrases(doc: Document, options?: { n?: number; method?: KeyphraseMethod }): Promise<Keyphrases>;
class MatraError extends Error {
readonly kind: ErrorKind;
readonly details: Record<string, string | number | boolean>;
}
const limits: { readonly maxInputBytes: number /* and the rest, as Python's matra.limits */ };
- Types.
Document,Summary,Keyphrases,Blockand the error body are generated fromspec/schema/v2/, by the step that generates Python's (The ports and the names). The generated file is committed and a check regenerates and compares it, as the docsite'sllms.txtis held. Field names stay as the schema spells them (total_words,byte_start), because a document is the same JSON in every language. - Defaults follow Python. With no format named, text is plain text and bytes are detected; whether a signature outranks a format named on text is decision 8, and the core decides it for every language.
summarizereturns three sentences andkeyphrasesten phrases by default observed _core.pyi:437 _core.pyi:452.methodis optional and takes the names in the core's list, the names Python takes; no function name carries one. - A document is data. The domain already says so: a document "can come from
Engine::analyze, from JSON a caller saved and read back", and every outcome holds on one whose parts contradict each other observed domain.rs:2007. Methods do not cross into another language, fields do. So TypeScript'sDocumentis the parsed JSON, and an outcome function sends it to the core, which reads it back as the Rust type. That round trip is unmeasured (what was not measured). - Outcome functions need no engine. They start the core alone, with no parser and no model. Because they return promises, whether they run on the calling thread or in a worker can change without changing a signature; the milestone that builds them measures both.
analyzeAllyields one item per input, in input order, as each finishes. A failed document is an item,{ path, error }, not an exception, as in Rust and Python (decision 3).closeends the worker. It is the only way to give the model's memory back: the parser's memory did not shrink when the model was freed.- One call at a time. Calls on one engine queue and run in order. An engine is not shared between workers, as an
Enginein Rust is not shared between threads. - One package. The browser build starts a module worker; the Node build a worker thread; the caller's code is the same. The entry is chosen by the package's export conditions. Only the browser half of that was measured.
limitsis generated from the Rust constants when the package is built, so the wrapper can refuse an oversized input before copying it to the worker. The core refuses it regardless: the spike saw an input one byte over the 8 MiB cap fail with the library's own error observed domain.rs:50 the spike.
python/matra/_core.pyi line 437, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
python/matra/_core.pyi line 452, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
src/domain.rs line 2007, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
src/domain.rs line 50, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
Failures
Three classes, told apart by who is at fault.
| What happened | What the caller sees | The engine afterwards |
|---|---|---|
| The library reports a failure (oversized input, unknown format, a parse that breaks the contract, a model that is not the pin) | MatraError with the kind and details every other channel carries | usable |
| The parser throws a C++ exception or traps | MatraError: model_invalid while loading, parse_failed while parsing | usable after an exception; closed after a trap |
| matra's own code panics | an Error that is not a MatraError, carrying the panic's message and location (decision 5) | closed |
- Kinds cross as data.
Erroralready serializes as its kind, message and details observed domain.rs:455. The spike passed only the message as a string; the binding passes the serialized error, and the wrapper builds theMatraErrorfrom it. - The parser's failures are the parser's. The native adapter catches a panic at the C++ boundary and reports
ParseFailedobserved udpipe.rs:695 grounds for decision 5. The wrapper does the same at the one place it calls the parser module. In the spike a C++ exception thrown inside the parser, from inside a call the core was making, arrived asparse failed, and the same instance then loaded a good model and parsed observed the spike. A trap is different: memory after an abort is not to be trusted, so the engine closes and the caller loads a new one. - A panic cannot be caught in the core. On this target a panic aborts. A hook records its message and location before it does; the wrapper catches the trap, attaches what the hook recorded, closes the engine, and rejects. A call on a closed engine rejects with an
Errorsaying it is closed.
src/domain.rs line 455, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
src/nlp/udpipe.rs line 695, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
Building the parser
The C++ is the copy the udpipe-rs crate vendors, at the version and checksum Cargo.lock already pins for the native build observed Cargo.lock:1781-1784, found through cargo metadata. One pin, two builds, which is why the output is the same. The spike used Emscripten 7.0.0 with -O2; the build pins the compiler by version and digest, as the repository pins its other tools.
Cargo.lock lines 1781 to 1784, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
Which exception handling the build uses is decision 4. The build the spike measured in browsers uses the older instruction, which Firefox reports as deprecated.
The package ships UDPipe compiled, as the crate and the wheels do. What UDPipe's licence asks of a binary on npm (its text, a pointer to the source) is checked in the milestone that builds it, not here.
The package: where it lives, how it is built and published
- Where.
typescript/at the repository root, besidepython/: the wrapper's source, the generated types, the tests andpackage.json. Nothing built is committed. - Name and version. One name and one version on every registry (decision 6).
scripts/check-version-sync.shreadstypescript/package.jsonwith the other version-carrying files. - Build. One
justrecipe: the core throughwasm-bindgen(the spike used wasm-pack 0.15.0 and wasm-bindgen 0.2.130), the parser through Emscripten, the types from the schema, then the TypeScript compiler. The tarball holds the two.wasmfiles, the JavaScript, the type declarations and the licence texts. It never holds the model. - Serving. With the
.wasmfiles served asapplication/octet-stream, streaming compilation was refused in all three engines and both loaders fell back and worked; no cross-origin isolation headers were sent and none were needed observed the spike. So the package does not depend on how its host labels a file. - Publishing. A release publishes through
release.yml, where each registry is a job behind an environment that waits for the owner observed release.yml:527 release.yml:582. npm joins as a third: a job that builds the tarball, the existing attestation extended to cover it, a dry run in the verify job, andpublish-npmbehind a newnpmenvironment. Nothing reaches npm without that approval, and an agent never gives it.
.github/workflows/release.yml line 527, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
.github/workflows/release.yml line 582, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
Conformance and continuous integration
The conformance suite was written expecting this: its fixtures are language-agnostic and its README names "a TypeScript package" as the binding to come observed spec/README.md:6. A TypeScript runner reads the same files:
spec/README.md line 6, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
- The parse fixtures in
spec/tests/, throughEngine.loadandanalyze. - The outcome fixture, a whole document and what
summarizeandkeyphrasesreturn over it. This is the round trip of a document through JSON, held by a fixture that already exists. - The error kinds, against
spec/schema/v2/error.schema.json. - Parity. The texts of
spec/tests/parity/through TypeScript, compared with what Rust writes for the same text, field for field.
A new typescript job in ci.yml builds both modules and runs that suite four ways: in Node, and in Chromium, Firefox and WebKit through Playwright, in a module worker, with the model served from the job's own machine. The docsite already pins Playwright and installs a browser in its floor observed site/package.json:30.
site/package.json line 30, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
Playwright's builds are the engines, not the browsers people install. The job therefore also runs the suite in the Chrome the runner image carries, and on a macOS runner in Safari through its own driver. Neither was tried in the spike. The milestone finds out whether each works, and the package's support statement names only what the job runs. iOS is not reachable from this job and stays unmeasured.
The existing wasm32 check job, which holds the core to compiling for the target observed ci.yml:106, gains a line for the wasm feature.
.github/workflows/ci.yml line 106, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
What the spike measured
One machine: Apple M1 Max, macOS 26.5, Node 26.0.0, Playwright 1.65.1 with its builds of Chromium 159.0.8154.6 (the headless shell), Firefox 159.0 and WebKit 27.2. The spike's files are not in the repository. Each milestone repeats its own part in continuous integration.
Stage 1: the parser alone, in Node. UDPipe 1 built with Emscripten 7.0.0, -fwasm-exceptions -O2.
| Measure | Native | WebAssembly in Node |
|---|---|---|
Output over the 12 texts of site/inputs/ (57 paragraphs, 6,529 words) | reference | byte-identical (cmp, and the same SHA-256) |
| Model load | 751 to 757 ms | 826 to 828 ms; 974 to 1,029 ms on a first load |
| Parse, a 3,347-word essay, median of 5, 3 invocations | 1,010 to 1,018 ms | 1,758 to 1,767 ms |
| Words per second | 3,289 to 3,312 | 1,894 to 1,903 |
| Memory | about 233 MB peak resident | 17.1 MB at start; 181.5 MB after one model load, unchanged after parsing, after five loads, and after freeing the model |
Four other builds (optimised for size, no file system, JavaScript exceptions, no exceptions) gave the same bytes. A fixed memory of 128 MB aborted; 192 MB worked. The module imports only the system shims Emscripten supplies: no threads, no shared memory, no eval.
Corrupted models: truncated and random models failed cleanly on every build, and the same instance then loaded a good model. Of 41 single-byte changes and truncations, 39 failed cleanly, one (the first byte) threw the uncaught exception, and one (offset 40) loaded and parsed. With the first byte changed, the build with WebAssembly exceptions threw something JavaScript caught; the build with none aborted; in each case the instance then loaded a good model and parsed.
Stage 2: the core, in Node. matra 0.3.0 with default features off, wasm-bindgen 0.2.130, the parser handed in from JavaScript.
| Measure | Result |
|---|---|
| Document JSON against the native command line, three texts | identical, compared one JSON token at a time (219,399, 7,819 and 8,524 JSON tokens: each string, number, literal and punctuation mark of the output, not words of the text) when both read text; against a file read natively, only provenance.source.path differs |
| Summary and keyphrases over the same documents | identical |
| The comparer itself | caught a document with the last digit of one number changed |
| Whole process, the long essay | native 1.80 s; Node with WebAssembly 2.88 s |
| Markdown and block readers; the compression metric | work |
| Unknown format, invalid UTF-8, input one byte over the cap | fail with the library's own errors |
| A parser that throws, returns nonsense, a word not in the text, or a head out of range | each parse failed, from the core's own checks |
| A panic, with no hook | RuntimeError: unreachable; the instance still answered once, then failed when freed |
Stage 3: browsers. The same two binaries, by SHA-256. Three runs each on the main thread and in a module worker, a fresh browser process per run, a local server sending only a content type and a length.
| Measure | Chromium | Firefox | WebKit |
|---|---|---|---|
| Output against native, all runs | identical | identical | identical |
| Words per second, median, main thread and worker | 1,715 and 1,688 | 1,586 and 1,600 | 1,734 and 1,723 |
| Model load, cold, median | 1,059 and 1,071 ms | 1,242 and 1,266 ms | 985 and 1,000 ms |
| Both exception probes (older and newer instruction) | run | run, with a deprecation warning for the older | run |
Analyzing the long essay took 1.95 to 2.2 s, and the two short texts 61 to 69 ms. On the main thread the page was blocked for about one second while the model loaded and about two for the essay; in a worker a heartbeat on the page showed no gap. The parser's memory was 17.1 MB at start and 181.5 MB with the model; the core's was 1.9 MB at start and 5.9 MB after three documents. The browser process grew by about 280 to 320 MB.
Sizes.
| File | Raw | Gzip | Brotli |
|---|---|---|---|
The parser, -O2 with WebAssembly exceptions | 668,753 | 227,843 | 176,778 |
| The parser, no exceptions | 630,390 | 215,128 | not taken |
| The parser's loader script | 59,022 | 16,491 | not taken |
| The core, lean (analyze only) | 1,446,259 | 659,303 | 534,547 |
| The core, with outcomes and the spike's probes | 1,692,251 | 760,247 | not taken |
| The core's glue script | 12,953 | 3,339 | not taken |
| The model | 16,309,608 | 16,314,639 | 16,309,636 |
By code size, the lean core is 32.6% the compressor behind one metric, 13.1% binding shims, 11.9% the markdown reader and 8.0% matra.
What was not measured
- Safari itself, any browser on iOS, and Chrome with a window. Chromium ran as the headless shell and WebKit as Playwright's macOS build.
- A model fetched over a real network. Every fetch was from the same machine.
- How GitHub Pages labels a
.wasmor a.udpipefile, and which origins it lets fetch one. It was not contacted. - A parser build on the newer exception instruction. Only a small probe of that instruction ran.
- A worker thread in Node. Node ran both modules on its main thread.
- A document's round trip through JSON into an outcome function, and the outcome functions' time alone.
- The core's own SHA-256 over the model; the size-tuned core's speed; a bundler; a strict content security policy; Deno and Bun.
- Embeddings in WebAssembly, and a threads build.
- Whether memory the engines agree to reserve can be committed on a small device. A phone's limit is unknown.
- Any second machine.
The boundary test and the resilience floor
The ACES boundary test, on the design as recommended:
| Question | More | Less |
|---|---|---|
| Adaptable | A new target behind a non-default feature. The model's address is the caller's argument, so who hosts it can change with no release. Promises everywhere, so where work runs can change with no signature. The exception flavour is a build setting. Every document names the reader that read it, so replacing one is visible where its output is. | Only a pinned model loads. That is deliberate: a trust anchor is pinned, not configurable. The new adapter relies on a build without threads. |
| Composable | A second adapter of an existing port. One shared function replaces a rule that would otherwise exist twice. The binding is held to rule 7, and the composition root stays the only place adapters are wired. An outside reader and parser build with the constructor and the span rule matra's own adapters use, so neither has to route around a port. | The crate's manifest gains optional dependencies for a target most Rust callers never build. If decision 7 says (b), with_reader returns a Result where with_source does not. |
| Extensible | The fourth syntax uses the first three's names, its types come from the schema, and the conformance fixtures tell a contributor when it drifts. A new method reaches every binding with no edit outside the core, and Python's types come from the schema TypeScript's do. | Two toolchains, Emscripten and wasm-bindgen, stand between a contributor and a built package. One recipe and one job hide that, and do not remove it. Decomposer gains a required method, so every outside reader changes once, and schema 2.1 regenerates every fixture that carries a provenance. |
The resilience floor, discipline by discipline:
| Discipline | Here |
|---|---|
| Size cap at the entry point | The wrapper refuses an oversized input before copying it; the core refuses it again. A model is refused by length before it is hashed, and a response is cut off at the pinned size. A paragraph built from pieces can be longer than the text it maps to, so the paragraphs' texts together are capped as the analysed text is, before the parser sees one. |
| Symlink rejection | Not applicable: the surface reads no file. |
| Atomic writes | Not applicable: the package writes nothing. |
| Verified bytes are the bytes loaded | The digest is taken over the worker's own copy, and that copy is what the loader reads. |
| Panic boundary at foreign code | Weaker than native, and said so. A C++ exception or trap becomes one of the library's kinds, as natively. A panic in the core cannot be caught on this target, so it costs the engine, not only the call; it is loud, and it names itself. |
| Cycle-safe walks | Unchanged: the same Rust runs. A document from anywhere is already an input the outcomes must survive. |
Milestones
In the order they are built, each needing the ones before it unless it says otherwise. Each is one pull request with its documentation and CHANGELOG entry, except milestone 2, whose parts are independent and may land one pull request each. Each leaves the tree shipping what it shipped before. None starts before acceptance.
- Types.
typescript/with the types generated fromspec/schema/v2/,python/matra/types.pygenerated from the same schema by the same step with its prose moved into the schema's descriptions, the check that regenerates both and compares, andMatraError. No WebAssembly. A TypeScript program can already read whatmatra analyze --jsonwrites. - The ports and the names. Everything under The ports and the names, in Rust and Python: the paragraph from pieces with the markdown reader moved onto it and the cap on what reaches the parser;
Span::of_form, which the UDPipe adapter calls; the reader's identity,provenance.readerand schema 2.1 with its fixtures regenerated;with_readeras decision 7 rules; one list and mapping per selector, which the configuration, the command line and Python call; andInput::textas decision 8 rules, with the new law. Needs milestone 1, so the new field reaches both languages' types by regeneration. Native output changes only inprovenance, and for text that carries a signature if decision 8 says (b). - One copy of the mapping. The Rust refactor under The Rust side: the shared function, which places spans with milestone 2's
Span::of_formand stays crate-private, the pin moved, and the builder's default if decision 2 says so. Native output is held unchanged by the existing conformance suite. - The core. The
wasmfeature,src/wasm/, rule 7's check, errors as data, the panic hook, andsummarizeandkeyphrasesover a document, theirmethodnames taken from milestone 2's lists, with the outcome fixture running in Node and the three engines. Measures the round trip. - The parser. The Emscripten build pinned to
Cargo.lock, the digest before load, the worker,Engine.load,analyze,analyzeAllandclose, with the parse and parity fixtures in Node and the three engines, then in Chrome and Safari. Measures what decision 4 rests on, the Node worker, and the core's digest. - The release. The npm job and environment in
release.yml, the docsite's install page, guide and reference, theffi-surfaceskill brought up to date, a newcomer pass, and the first npm release, which publishes only on the owner's approval. The owner may instead release after milestone 1 or 4; each leaves a package that is useful alone. - A component that renders a parse. It needs only a document's JSON and the types of milestone 1, so it works with or without the engine and may be built any time after milestone 1, beside milestones 2 to 6. Its element name and attributes are new public names and come as their own API change proposal.
- The playground. A docs page that loads the package and shows what it returns with the component of milestone 7. It waits on who serves the model.
Drawbacks
- About half the native speed, and 181 MB a page keeps. Both measured. Neither is tuned, and neither may be tunable.
- The model is 95% of the download and cannot be made smaller by compression.
- A panic costs the engine. Natively a caught panic costs one call.
- A second toolchain. Emscripten joins the build, pinned, for one artefact.
- A release has a third approval, and a third registry where a mistake is permanent.
- A third of the core is one metric's compressor. The proposal keeps it: a document that differs by build is a worse cost than its size.
- Every outside reader changes once.
Decomposergains a requiredidentity, and if decision 7 says (b)with_readerreturns aResult. Either lands before any TypeScript does, so the change is made once. - A schema minor before the first TypeScript release.
provenance.readermovesprovenance.schemato2.1in every document and regenerates every fixture that carries a provenance. - Python's prose moves into JSON. Field descriptions written in the schema are harder to write than docstrings, and a generated
types.pyreads less like hand-written Python. - The evidence is one spike on one machine. The list of what was not measured is longer than the list of engines that were.
Rationale and alternatives
Do nothing
The roadmap keeps a statement the spike disproved, and a TypeScript caller keeps reading matra's JSON with hand-written types. The owner has asked for more than that.
One module, not two
Compile matra with its udpipe feature for Emscripten's Rust target, so the native adapter runs as it is and nothing is shared or copied. It was not tried. The udpipe feature brings a network client that has no meaning in a browser, and the binding layer the spike used is built for the other target. Two modules are what was measured to work, and the core in that design is also the artefact that serves a page with no parser.
A Document class holding a handle to Rust memory
It avoids sending a document back to the core for each outcome. It also gives every document a lifetime the caller must end, makes a document from a server a different kind of thing from a document from the engine, and cannot cross from the worker to the page without becoming data anyway. The spike saw what a stale handle does after a panic.
A server that parses
The roadmap named it. It is a product, not a library: it needs hosting, and it sends a caller's text to someone else's machine. A caller who wants one can build it on the Python package, and the TypeScript types and outcome functions here serve its output.
Let the caller name any model
The caller passes a model and its digest, and the package checks one against the other. That verifies only that the caller hashed their own file. It reopens what 0.3.0 closed in Rust and Python, and it puts the model that loads silently with one byte changed one typo away. A caller with a parser of their own is a future addition, as Engine::with_parser is in Rust.
Fetch the model from a default address
It would make the first call one line shorter. The model's licence forbids the package from deciding that for a commercial caller, its host refuses the request, and a default address is a promise to keep a file online for as long as any released version exists.
Publish the text map as it is
Make TextMap and ReadBlock::mapped public, with a constructor taking offsets into the paragraph's text. It is less to write, and it is the form the markdown reader uses inside the crate. It also hands a caller the text and its map separately, so the two can disagree, by a gap, an overlap or a piece past the end, in ways the engine checks only in part, and it publishes how a map is held. Built from pieces, the text is the map, and those disagreements cannot be written.
Check types.py against the schema
Keep Python's types written by hand, and add a test that walks every definition in the schema and compares it in full with its TypedDict: the keys, which are required, and each value's type. Python's prose stays where Python readers look for it. But that test is a second reader of the schema, as large as the step that writes the types, and it can only say that the two differ, where generation writes what they should be. TypeScript needs the generator anyway, so Python's types are one more output of it.
Prior art
- matra's Python binding. A module of the crate behind a feature, held to the public surface by a boundary rule, with a typed stub and a conformance runner. This proposal follows it step for step.
udpipe_flutteron pub.dev (0.2.1, published 2026-06-14) ships UDPipe 1 as "native FFI on desktop/mobile and WebAssembly on the web". Someone else already runs this parser in a browser, which the roadmap's entry did not know.- The conformance suite's own plan.
spec/README.mdhas named a TypeScript binding since the fixtures were written. - The
ffi-surfaceskill sketched this as awasmfeature and asrc/wasm.rs, or a separate crate, to be decided when a caller appeared. Decision 1 decides it.
Unresolved questions
Decision · open · reversible
1. Where does the Rust side of the binding live?
- Rests on
- observed The rule that places a token's span, the function that calls it, and the mapping from one of…
- observed so the spike, written as an outside crate, carried its own copy of each
- observed the Python binding is a module of the crate behind python
- observed The function that says whether a token's span lines up with its form is crate-private, and it…
- Depends on
- no other decision
A In the crate, behind a `wasm` feature recommended
src/wasm/ and a second parser adapter, as The Rust side describes. The mapping from a word's columns to a token is shared inside the crate, and no Rust name is published beyond the span rule, which every parser gets under either choice.
B In a second crate that uses only the public surface
A crate beside matra's in a workspace, depending on matra as any caller does, as the spike was built. The span rule is public under either choice (The ports and the names); the mapping from a word's columns to a token becomes public too, so the second crate has one copy to call.
Claude recommends A, In the crate, behind a `wasm` feature. inferred, a judgment
The crate already holds a binding this way, and the reasons carry over: one version, one composition root, one place the boundary rules are checked. The copies disappear with nothing added to the Rust surface but the span rule, which every outside parser needs and so is published anyway; every name there is permanent. A second crate would need a workspace, a second manifest kept in step, and the column mapping published for one caller.
The strongest case against it
The spike is the proof that the public surface was almost enough: an outside crate built a working binding and needed three private pieces. Choice (b) would keep that proof running on every commit, so that a power the ports withhold from an outside implementer shows up as a build failure, not in the next review. It also keeps wasm-bindgen out of matra's manifest.
The owner's decision: open. Answer with accept, accept with a reservation, object or redirect, in a comment on this block or on the pull request; Claude then records your ruling here, as a named step in a pull request. A comment is never read as a ruling. Silence is not assent.
Decision · open · costly to reverse
2. What does a token built through the builder say in a column the parser left out?
- Rests on
- Depends on
- no other decision
A `_` for `deps`, as matra's own adapter writes recommended
Token::builder starts deps at _. A parser with no enhanced graph then says so the way every document matra has produced says so.
B Leave the default empty and document the difference
The contract already allows _ or empty. Each outside parser sets deps itself if it wants to match.
Claude recommends A, `_` for `deps`, as matra's own adapter writes. inferred, a judgment
Two spellings of "nothing here" in one field make two documents over the same text differ for no reason a reader can use, and the spike only matched native output after setting this by hand. It is a one-word change to a default, in 0.x, with a line in the CHANGELOG.
The strongest case against it
It changes what an existing caller's parser emits without that caller changing anything. It also leaves xpos and misc starting empty, so the builder would then be inconsistent with itself where today it is only inconsistent with the adapter.
The owner's decision: open. Answer with accept, accept with a reservation, object or redirect, in a comment on this block or on the pull request; Claude then records your ruling here, as a named step in a pull request. A comment is never read as a ruling. Silence is not assent.
Decision · open · costly to reverse
3. What does `analyzeAll` return?
- Rests on
- Depends on
- no other decision
A A promise of an array
const docs = await engine.analyzeAll(inputs), as first sketched. Every result is held until the last is done, each a document or a { path, error } item.
B An async iterable recommended
for await (const item of engine.analyzeAll(inputs)). Items arrive in input order as each finishes. await Array.fromAsync(engine.analyzeAll(inputs)) is choice (a) in one line.
Claude recommends B, An async iterable. inferred, a judgment
Python's analyze_all is an iterator, "lazily and in input order", and so is Rust's observed _core.pyi:249 grounds for decision 3, so (b) is the same contract in a third syntax and (a) is a different one under the same name. A long document takes about two seconds observed the spike grounds for decision 3, so a page analysing ten wants the first after two seconds, not all ten after twenty, and a Node program walking a folder wants neither every input nor every result in memory at once.
The strongest case against it
The owner was shown (a), and an awaited array is what a JavaScript reader expects. With no directory input in this surface, a caller who wants results one at a time can loop over analyze, which makes analyzeAll a convenience either way, and the simpler convenience is the array.
The owner's decision: open. Answer with accept, accept with a reservation, object or redirect, in a comment on this block or on the pull request; Claude then records your ruling here, as a named step in a pull request. A comment is never read as a ruling. Silence is not assent.
python/matra/_core.pyi line 249, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
Decision · open · reversible
4. Which exception handling does the parser build use, and which browsers does that admit?
- Rests on
- observed The build the spike measured contains only the older instruction
- observed Firefox reports that instruction as deprecated and asks for the newer one
- observed All three engines ran a probe of each
- inferred That closes both findings of the corrupted-model run at once: the model that crashes the loader…
- inferred A parser built on the newer instruction will give the same bytes in those three engines: five…
- assumed It also assumes the people who will use a new package run engines recent enough to have the…
- Depends on
- no other decision
A The older instruction, as measured
Ship the build the spike ran in three engines. It works today everywhere it was tried, and Firefox logs a deprecation warning on every load.
B The newer instruction recommended
Build with the instruction Firefox asks for. The package then needs an engine that has it, and the support statement says which, from what the conformance job runs.
C No exception handling in the binary
Any C++ throw becomes an abort. The smallest build, with no dependence on either instruction. A throw UDPipe would have caught itself becomes a closed engine.
Claude recommends B, The newer instruction. inferred, a judgment
The build the spike measured contains only the older instruction observed the spike grounds for decision 4, and Firefox reports that instruction as deprecated and asks for the newer one observed the spike grounds for decision 4. All three engines ran a probe of each observed the spike grounds for decision 4. A parser built on the newer instruction will give the same bytes in those three engines: five builds with different settings already did, and the engines accept the instruction inferred likely grounds for decision 4. That is a prediction, and milestone 5 tests it before anything ships; if it fails, (a) is one build setting away.
The direction matters more than the flavour. Moving from (b) to (a) later widens what the package runs on and breaks nobody. Moving from (a) to (b) later, when an engine drops the older instruction, narrows it under callers who are already relying on it. The digest check means no model can reach the throw the spike found, so what remains is defence in depth, and (c) gives up the one thing exceptions buy: UDPipe behaving as it does natively when it catches its own.
The strongest case against it
(a) is the only flavour that has parsed a document in a browser. (b) rests on a probe and a prediction. It also assumes the people who will use a new package run engines recent enough to have the newer instruction assumed grounds for decision 4, and the one platform most likely to lag, a phone's browser, is the one nothing here measured. A warning in a console is not a failure, and no date for the older instruction's removal is in evidence.
The owner's decision: open. Answer with accept, accept with a reservation, object or redirect, in a comment on this block or on the pull request; Claude then records your ruling here, as a named step in a pull request. A comment is never read as a ruling. Silence is not assent.
Decision · open · costly to reverse
5. What does a caller see when matra's own code panics?
- Rests on
- Depends on
- no other decision
A An error outside the kinds, and a closed engine recommended
The rejection is an Error that is not a MatraError, with the panic's message and location and the trap as its cause. The engine closes.
B A new kind, `internal`, on every channel
MatraError with kind: "internal", added to the error schema and to Rust, Python and the command line.
C Report it as `parse_failed`
No new name anywhere. A defect in matra reads as a parser failure.
Claude recommends A, An error outside the kinds, and a closed engine. inferred, a judgment
A kind is a failure the library reports and a caller can act on. A panic is a defect in matra, and each language already has its own word for that which is outside the error contract: an unwinding panic in Rust, an exception that is not a MatraError in Python. TypeScript's is an Error that is not a MatraError. It must be loud, name itself and say where, which the hook provides, and it must close the engine, because the spike saw the instance misbehave afterwards. (c) hides a defect behind a kind that tells the caller to check their input.
The strongest case against it
"Failures reject with an error carrying the library's kind" stops being true of every rejection. The schema calls the kinds "an open set" observed error.schema.json:10 grounds for decision 5, so (b) is an addition, not a break, and it would give a caller one catch that handles everything and a kind to log.
The owner's decision: open. Answer with accept, accept with a reservation, object or redirect, in a comment on this block or on the pull request; Claude then records your ruling here, as a named step in a pull request. A comment is never read as a ruling. Silence is not assent.
spec/schema/v2/error.schema.json line 10, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
Decision · open · costly to reverse
6. What is the package called on npm?
- Depends on
- no other decision
A `matra` recommended
The name on crates.io and PyPI. On 2026-11-18 the registry at registry.npmjs.org/matra answered 404, so no package holds it; that is not a promise npm will grant it.
B A scoped name under an organisation the owner controls
@scope/matra. Always available, and it shows who publishes it.
Claude recommends A, `matra`. inferred, a judgment
matra ships under "one name, one version and one attribution" observed EPR-0003:13 grounds for decision 6. import ... from "matra" in a third language is the same rule.
The strongest case against it
An unscoped name is a claim on a shared namespace that npm may refuse or a similar name may contest, and a scope is the registry's own answer to who a package belongs to. If the name is taken between this text and the first release, (b) is what remains.
The owner's decision: open. Answer with accept, accept with a reservation, object or redirect, in a comment on this block or on the pull request; Claude then records your ruling here, as a named step in a pull request. A comment is never read as a ruling. Silence is not assent.
blueprints/proposals/0003-distribution-and-provisioning.md line 13, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
Decision · open · costly to reverse
7. What does `with_reader` do for a format that already has a reader?
- Rests on
- observed with_reader replaces whatever reader the format had, the one matra ships or one the caller…
- observed Replacing the reader matra ships is the documented use: a reader "in place of the reader this…
- observed The reader trait has one method, and provenance records the parser, the format and the source…
- observed EPR-0006 gave with_reader and with_source one meaning: each will "replace one part of an engine"
- Depends on
- no other decision
A Replace, as today, with the reader recorded recommended
with_reader keeps returning Engine, and the last registration for a format wins, whether it replaces the reader matra ships or one the caller registered. Each document's provenance.reader says which reader ran.
B Replace a shipped reader; refuse a second of the caller's own
Replacing the reader matra ships for a format stays what with_reader is for, and provenance shows it. Registering a reader for a format the caller has already given one is refused: with_reader returns Result<Engine>, failing with invalid_input, its reason naming the check (reader_registered_twice, a working name) and its message both readers' identities.
C Refuse every replacement, and replace with a second call
with_reader refuses a format that has a reader, shipped or not, and a replace_reader replaces one. Nothing is replaced without a call that says so.
Claude recommends A, Replace, as today, with the reader recorded. inferred, a judgment
The finding was silence, and the reader in provenance ends it: every document names the reader that read it, so a replacement shows wherever its output goes. What is left is the case (b) refuses, a caller who registers two readers for one format, and the engine cannot tell that mistake from a choice. EPR-0006 gave with_reader and with_source one meaning: each will "replace one part of an engine" observed EPR-0006:408 grounds for decision 7, and with_source replaces the one source without complaint. A library that hands back an engine with its readers registered, which an application then overrides for one format, is that meaning used on purpose, and (b) refuses it. (b) also has the engine remember which readers it shipped and which a caller gave it, and puts a ? on every with_reader chain, in each language, for a failure most callers cannot hit. Under (a) the last word wins, the order the shipped defaults already follow, and no signature changes. (c) makes the common case, replacing matra's reader, a second name for one act.
The strongest case against it
Replacing the reader matra ships is the documented use: a reader "in place of the reader this build ships for it" observed lib.rs:372-373 grounds for decision 7. A second registration of the caller's own is a different thing: from the moment it lands the first is code that never runs, and only the person who wrote both knows which they meant. Under (a) the engine hands back documents from whichever came last and leaves the mistake to whoever reads provenance.reader; (b) refuses it and names both readers before any document is read.
The owner's decision: open. Answer with accept, accept with a reservation, object or redirect, in a comment on this block or on the pull request; Claude then records your ruling here, as a named step in a pull request. A comment is never read as a ruling. Silence is not assent.
blueprints/proposals/0006-the-0-3-0-surface.md line 408, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
src/lib.rs lines 372 to 373, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
Decision · open · costly to reverse
8. Is text detected as bytes are?
- Rests on
- observed The Input documentation says a PDF or DOCX signature is the strongest evidence of a format…
- observed text goes straight to the reader for the format it was given, while bytes and files are…
- observed matra already answers that against content that decodes and a format the caller named: the law…
- observed in JavaScript a string is weak evidence of text: reading a file as text decodes what is not…
- observed The constructor's own documentation reads that way, where the type's does not: "Text in a…
- inferred And (b) closes two signatures of a wider problem: a Latin-1 file read with file.text() decodes…
- Depends on
- no other decision
A Text keeps the format its caller names
A signature is evidence for bytes and files only. The Input documentation is corrected to say so, and the new law holds text and bytes of the same content to the same document when the content carries no signature.
B Text is detected as bytes are recommended
Input::text(t, f) reads exactly as Input::bytes(t, Some(f)): a PDF or DOCX signature outranks the named format, as it does for bytes and files. The documentation holds as written, and the law holds for every text, with no exception.
Claude recommends B, Text is detected as bytes are. inferred, a judgment
A caller who holds a string has said that it decodes, and a signature does not ask that; decoding does. A signature asks whether the content is a PDF, and matra already answers that against content that decodes and a format the caller named: the law tests refuse %PDF-1.7 passed as bytes named plain text, and a file holding it read as markdown observed lib.rs:1159 lib.rs:1666-1667 grounds for decision 8. Under (a) the same content is refused on two routes and read on the third. And in JavaScript a string is weak evidence of text: reading a file as text decodes what is not UTF-8 into replacement characters without an error, and Node 26, run on 2026-11-19, turned a PDF's opening bytes into a string starting %PDF-1.7 with no complaint observed the Encoding Standard grounds for decision 8. A page that passes Input.text(await file.text()) for a PDF gets, under (a), a document of noise and no error, and under (b) unsupported_format naming pdf, which tells it to pass the bytes. One rule of evidence for every constructor also leaves annotate one route fewer: text is bytes already known to decode.
The strongest case against it
The type is already evidence, and in Rust the caller has also named a format: two statements a lying file name does not make. The constructor's own documentation reads that way, where the type's does not: "Text in a format the caller names" observed lib.rs:96 grounds for decision 8. Under (b) no constructor but blocks reads, as text, a text that opens with %PDF, and a string's bytes go to a PDF reader when a caller has registered one. And (b) closes two signatures of a wider problem: a Latin-1 file read with file.text() decodes into noise with no signature to catch, under either choice inferred grounds for decision 8, so the defence that covers JavaScript is the guide telling a page to pass a file's bytes, which this proposal does either way.
The owner's decision: open. Answer with accept, accept with a reservation, object or redirect, in a comment on this block or on the pull request; Claude then records your ruling here, as a named step in a pull request. A comment is never read as a ruling. Silence is not assent.
src/lib.rs line 1159, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
src/lib.rs lines 1666 to 1667, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
src/lib.rs line 96, at 4cac941
Claude read these lines at this commit; that is a reading, not a test. Open them on GitHub at 4cac941.
Who serves the model
This is the owner's decision and Claude recommends no option. The facts: the model is CC BY-NC-SA 4.0; the package will not carry it; a browser cannot fetch it from its publisher. So every browser caller needs it at an address their page may fetch, and the playground is a browser caller. The npm package behaves the same under all three options: the caller passes a URL or bytes.
| Option | What it gives | What it costs |
|---|---|---|
| The docs site serves the current model, with attribution | The playground works as soon as the package does. The docs can point a browser caller at a working address. | The project redistributes a work licensed for non-commercial use only, under that licence's attribution and share-alike terms. Whether the project's hosting, and each use made of a public address, is non-commercial is a judgment about a licence, and it is the owner's. 16.3 MB joins every deploy and every first visit to the playground. How GitHub Pages serves the file, and to which other sites, is unmeasured. |
| The playground waits for the permissively licensed model | Nothing the project hosts carries a non-commercial term. One address could then serve every caller. | The playground waits on a model that is not yet trained or measured, with no date. Until then a browser caller hosts the current model themselves. |
| The caller always supplies it, the playground included | The project redistributes nothing. | The playground asks each visitor to download 16 MB by hand and select the file, since the page cannot fetch it for them. That is not "nothing installed". |
Under every option the package assumes a browser caller is able to put a 16 MB file on an origin of their own assumed.
What the owner should confirm or strike, collected from the claims above that stand on no grounds yet:
2 claims stand on no grounds yet. Confirm or strike each with a comment on it.
- It also assumes the people who will use a new package run engines recent enough to have the newer instruction where: Unresolved questions bears on decision 4
- Under every option the package assumes a browser caller is able to put a 16 MB file on an origin of their own where: Who serves the model bears on no decision
Future possibilities
clusters, once embeddings are measured in WebAssembly. The embedding adapter already compiles for the target.- A caller's own parser, as
Engine::with_parserallows in Rust, when a caller has one. Input.filefor Node, as a Node-only entry, when a Node caller asks for the path in provenance.- A progress notice for the model download, as the native engine offers one. Today a caller who wants progress fetches the bytes and passes them.
- The permissively licensed model, as a second pin, recognised by its digest.
- A smaller core, by measuring the size-tuned build's speed.
- The feed for text arriving over time, in TypeScript when it lands in the other three syntaxes.