{
  "generator": {
    "matra": "0.3.0",
    "udpipe_model": {
      "name": "english-ewt-ud-2.5-191206",
      "sha256": "784bd0fa85e3d831fd02a55290d0acfd05c953159dc38cc33d52e1b28add9957"
    }
  },
  "page": "architecture/design.md",
  "paragraphs": [
    {
      "compression_ratio": null,
      "kind": "paragraph",
      "lexical_density": 0.5714,
      "readability_grade": null,
      "sentences": 1,
      "text": "engine.analyze(Input::file(\"essay.md\")) yields a Document or an Error.",
      "words": 10
    },
    {
      "compression_ratio": null,
      "kind": "paragraph",
      "lexical_density": 0.6154,
      "readability_grade": 6.2723,
      "sentences": 2,
      "text": "Take a 40 KB markdown file. Analysing it runs five things in order, each a module hiding one decision, and only the last two are expensive.",
      "words": 26
    },
    {
      "compression_ratio": 0.5488,
      "kind": "paragraph",
      "lexical_density": 0.5814,
      "readability_grade": 8.6851,
      "sentences": 7,
      "text": "Fetch. Engine::annotate reads the file through the engine's source, FileSource unless the caller supplied another with Engine::with_source. FileSource calls symlink_metadata, which does not traverse. A symlink is refused, and a directory is refused naming Inputs::dir. Anything else that is not a regular file is refused. A file whose metadata size exceeds MAX_INPUT_BYTES, 8 MiB, is refused before a byte is read, so a 1 GB file costs one stat call rather than a GB of resident memory. Then all 40 KB are read in as bytes.",
      "words": 90
    },
    {
      "compression_ratio": 0.5092,
      "kind": "paragraph",
      "lexical_density": 0.5225,
      "readability_grade": 10.9727,
      "sentences": 6,
      "text": "Detect. The bytes are checked against the cap again (the same limit under the \"input\" label, because bytes and text handed in directly never passed the first gate), then given a format: a PDF or DOCX signature first, then a format the caller named, then the extension, so .md is Format::Markdown. A format with no reader in the engine's table is Error::UnsupportedFormat. Otherwise the reader registered for the format gets the bytes; a text reader decodes them as UTF-8, and bytes that are not are Error::UnsupportedFormat too. Unless the reader names an analysed text of its own, as a PDF reader would, the bytes become the analysed text, with no copy.",
      "words": 115
    },
    {
      "compression_ratio": 0.5,
      "kind": "paragraph",
      "lexical_density": 0.5702,
      "readability_grade": 11.4992,
      "sentences": 6,
      "text": "Read. The markdown reader runs a CommonMark parser over the text once, and turns its events into headings and paragraphs, each with its span in the text and its own text with the inline markup removed: a second copy of the prose. Front matter, code, tables and HTML are recorded in skipped with their spans rather than dropped. The engine then checks what the reader returned against the reader's contract: every span inside the text and on character boundaries, blocks in document order, no two spans overlapping, every heading at level 1 or more, and every paragraph's text lined up with the text its span covers. The structure step then makes the blocks a section tree, a heading opening each section.",
      "words": 124
    },
    {
      "compression_ratio": 0.4894,
      "kind": "paragraph",
      "lexical_density": 0.5852,
      "readability_grade": 9.2594,
      "sentences": 6,
      "text": "Parse and measure. annotate walks every paragraph in document order, skips the quotes, and calls the parser on each remaining paragraph's text. Sixty body paragraphs means sixty calls into UDPipe. Each call returns sentences of tokens, and the engine checks them before it uses them: ids 1..=n, every head in range, one root, no cycle, and every span a span of the paragraph, in order, over its token's form. Then it maps the spans from the paragraph's text into the file's bytes and builds each Sentence itself, its text and the fields derived from its tree included, so no derived field depends on which parser ran. Then compose runs four metric functions over the attached tree; the document-level pair aggregates over the same attachment, so there is no second sentence set to keep in agreement.",
      "words": 143
    },
    {
      "compression_ratio": null,
      "kind": "paragraph",
      "lexical_density": 0.5,
      "readability_grade": 14.9731,
      "sentences": 1,
      "text": "At peak, three representations of the same document are resident at once: the original String, the paragraph copies, and the parsed tokens hanging off the paragraphs.",
      "words": 26
    },
    {
      "compression_ratio": null,
      "kind": "paragraph",
      "lexical_density": 0.6207,
      "readability_grade": 7.1547,
      "sentences": 2,
      "text": "For a 40 KB input none of that matters. For a caller sizing a batch job against a memory ceiling it does, because every Token carries nine owned Strings.",
      "words": 29
    },
    {
      "compression_ratio": 0.5779,
      "kind": "paragraph",
      "lexical_density": 0.6182,
      "readability_grade": 11.7273,
      "sentences": 3,
      "text": "A directory changes the shape by streaming. Inputs::dir lists the entries up front but reads nothing until pulled, and each pull of analyze_all runs one document through the whole pipeline before the next file is read. One document's representations are resident at a time; only what you retain accumulates, and collecting the stream retains everything.",
      "words": 57
    },
    {
      "compression_ratio": null,
      "kind": "paragraph",
      "lexical_density": 0.75,
      "readability_grade": 13.3425,
      "sentences": 2,
      "text": "Everything above assumes nlp already exists. Constructing it is the one genuinely slow operation matra performs.",
      "words": 16
    },
    {
      "compression_ratio": 0.5705,
      "kind": "paragraph",
      "lexical_density": 0.525,
      "readability_grade": 13.69,
      "sentences": 4,
      "text": "Udpipe::english(dir) looks for the English UD-EWT model in the directory you name. If it is absent, the bytes are verified in memory and then written to a temporary subdirectory named for this call, and moved into place with a single rename, so two processes pointing at the same directory cannot leave each other a half-written file. The name carries more than the process id, because in a container every process is pid 1 and two cold starts would otherwise collide.",
      "words": 84
    },
    {
      "compression_ratio": 0.5098,
      "kind": "paragraph",
      "lexical_density": 0.5063,
      "readability_grade": 16.0395,
      "sentences": 3,
      "text": "Udpipe::from_config(&cfg) is the same call with the directory resolved rather than named, and it is what the public constructors reach: Engine::new() with a Config resolved from the environment alone, Engine::from_config(&cfg) with the one you pass. Config is read once, at construction; nothing in the call path above consults it, which is why configuration cannot change what a call computes. Model2Vec::potion_base_8m(dir) and Model2Vec::from_config(&cfg) do the same for the reference embedding model, against a three-file digest rather than a single one.",
      "words": 87
    },
    {
      "compression_ratio": 0.5574,
      "kind": "paragraph",
      "lexical_density": 0.5679,
      "readability_grade": 8.0306,
      "sentences": 5,
      "text": "The cached file is then checked against a size constant, 16,309,608 bytes, and a SHA-256 constant pinned in nlp/udpipe.rs. A size mismatch fails before the file is hashed. A mismatch downloads the model, and a download that also fails the digest is retried once, so a call makes at most two downloads. The file is replaced only if downloaded bytes verify; if both downloads fail, the call returns Error::ModelInvalid and leaves the old file untouched. An unverified model is never loaded.",
      "words": 83
    },
    {
      "compression_ratio": 0.5552,
      "kind": "paragraph",
      "lexical_density": 0.5,
      "readability_grade": 9.39,
      "sentences": 3,
      "text": "The verify step returns the bytes it hashed, and those exact bytes go to the loader. Nothing re-reads the disk in between. That ordering is the whole point: a hash-then-reopen sequence leaves a window in which an attacker with write access to the model directory can swap the file after the check and before the read.",
      "words": 58
    },
    {
      "compression_ratio": null,
      "kind": "paragraph",
      "lexical_density": 0.5,
      "readability_grade": 7.3518,
      "sentences": 2,
      "text": "No surface loads a model file the caller names. The pinned model is the only one a parse can come from, so the identity on every result is proof as well as a name.",
      "words": 34
    },
    {
      "compression_ratio": null,
      "kind": "paragraph",
      "lexical_density": 0.5135,
      "readability_grade": 4.2824,
      "sentences": 4,
      "text": "Load once, reuse. After construction, the model is a live handle to a C++ object. Parsing does not reload it. The Engine owns the provider, so a service builds one engine and keeps it for its lifetime.",
      "words": 38
    },
    {
      "compression_ratio": 0.5718,
      "kind": "paragraph",
      "lexical_density": 0.5733,
      "readability_grade": 9.0292,
      "sentences": 4,
      "text": "One engine per thread. The underlying Model is Send and deliberately not Sync: its parse path mutates internal workspace caches. The Engine that owns it is Send and not Sync for the same reason: an engine moves to another thread, but sharing one between two is a compile error rather than a race. To parallelize you build an engine on each thread, paying another 16 MB load for each, or serialize calls behind a mutex.",
      "words": 76
    },
    {
      "compression_ratio": null,
      "kind": "paragraph",
      "lexical_density": 0.5641,
      "readability_grade": 10.3569,
      "sentences": 3,
      "text": "Python gets the same constraint through a different mechanism. Its Engine is declared unsendable, so PyO3 records the creating thread and refuses access from any other at runtime. A ProcessPoolExecutor is unaffected, because each process loads its own model.",
      "words": 40
    },
    {
      "compression_ratio": null,
      "kind": "paragraph",
      "lexical_density": 0.5526,
      "readability_grade": 6.7395,
      "sentences": 3,
      "text": "Parsing per paragraph looks wasteful. Sixty FFI calls where one would do, and a parser that never sees a paragraph's neighbours as context. The reason it is worth that is a defect class the alternative could not avoid.",
      "words": 39
    },
    {
      "compression_ratio": null,
      "kind": "paragraph",
      "lexical_density": 0.7073,
      "readability_grade": 12.2635,
      "sentences": 2,
      "text": "The earlier implementation joined every paragraph into one string, parsed once, then wired each returned sentence back to a paragraph by matching text prefixes. Three regression tests in src/lib.rs hold the current behaviour in place, one per way that matching failed:",
      "words": 41
    },
    {
      "compression_ratio": null,
      "kind": "list_item",
      "lexical_density": 0.6818,
      "readability_grade": 9.0818,
      "sentences": 2,
      "text": "Two paragraphs sharing their first thirty characters. A sentence lands in whichever one matched first, and the other paragraph silently loses it.",
      "words": 22
    },
    {
      "compression_ratio": null,
      "kind": "list_item",
      "lexical_density": 0.7778,
      "readability_grade": 9.5533,
      "sentences": 2,
      "text": "One paragraph containing another paragraph's opening words mid-text. The greedy match steals the sentence from its rightful owner.",
      "words": 19
    },
    {
      "compression_ratio": null,
      "kind": "list_item",
      "lexical_density": 0.6364,
      "readability_grade": 12.3,
      "sentences": 1,
      "text": "An empty paragraph with trailing whitespace, which confused the match entirely.",
      "words": 11
    },
    {
      "compression_ratio": null,
      "kind": "paragraph",
      "lexical_density": 0.4545,
      "readability_grade": 10.5118,
      "sentences": 3,
      "text": "Follow that down and the root is not the matching algorithm. It is that the relationship between a sentence and its paragraph was being reconstructed from text, and text does not carry identity. Two paragraphs can be character-for-character similar and still be different paragraphs.",
      "words": 46
    },
    {
      "compression_ratio": null,
      "kind": "paragraph",
      "lexical_density": 0.5625,
      "readability_grade": 8.5958,
      "sentences": 3,
      "text": "So any reconstruction from that data has to guess, and a guess in this position fails silently. The Document that comes out is still well-formed, still serializes, still passes every shape check. It is simply wrong about which paragraph said what, and nothing in the output says so.",
      "words": 49
    },
    {
      "compression_ratio": 0.589,
      "kind": "paragraph",
      "lexical_density": 0.4915,
      "readability_grade": 7.3625,
      "sentences": 4,
      "text": "The two identical sentences are the ones from the regression fixture in src/lib.rs. On the left nothing in the text distinguishes them, so the assignment is a coin toss and one sentence is left over. On the right the arrows run the other way: the call produced the sentences, so the owner is known before any text is compared.",
      "words": 59
    },
    {
      "compression_ratio": null,
      "kind": "paragraph",
      "lexical_density": 0.575,
      "readability_grade": 7.9,
      "sentences": 3,
      "text": "Per-paragraph parsing deletes the wiring step instead of improving it. A paragraph's sentences are the return value of the call made on that paragraph's text. The relationship comes from the call graph, so there is nothing left to get wrong.",
      "words": 42
    },
    {
      "compression_ratio": 0.5695,
      "kind": "paragraph",
      "lexical_density": 0.5167,
      "readability_grade": 14.7933,
      "sentences": 2,
      "text": "Blockquote paragraphs are skipped at this stage, which is why they reach the end with no sentences and all three metric slots at None. And a parse failure in any single paragraph aborts that document with Error::ParseFailed; the partial document is dropped, not returned, and in a stream the failure travels as that document's DocumentError while the next document proceeds.",
      "words": 62
    },
    {
      "compression_ratio": 0.614,
      "kind": "paragraph",
      "lexical_density": 0.6735,
      "readability_grade": 11.3426,
      "sentences": 4,
      "text": "The metric suite is four functions with one signature, Box<dyn Fn(&mut Document)>, run in sequence by compose. Readability, lexical density, and compression write per-paragraph slots. The document pass writes vocabulary_ttr, nominalization_ratio and passive_ratio. None of the four reads another's output, so the suite is a list, not a chain.",
      "words": 52
    },
    {
      "compression_ratio": null,
      "kind": "paragraph",
      "lexical_density": 0.4667,
      "readability_grade": 11.5,
      "sentences": 1,
      "text": "Each carries its own applicability condition, and this is where unexplained None values come from:",
      "words": 15
    },
    {
      "compression_ratio": null,
      "kind": "paragraph",
      "lexical_density": 0.6364,
      "readability_grade": 10.1545,
      "sentences": 1,
      "text": "A word here is a non-punctuation token, the count Paragraph::word_count returns.",
      "words": 12
    },
    {
      "compression_ratio": 0.6258,
      "kind": "paragraph",
      "lexical_density": 0.5636,
      "readability_grade": 9.5818,
      "sentences": 3,
      "text": "None is not zero. A three-word paragraph has no meaningful Flesch-Kincaid grade, and the slot says so rather than reporting a number nobody should use. The compression cap is a CPU bound: the brotli window is 2^18 bytes, and a paragraph larger than one window is skipped instead of pegging a core on adversarial input.",
      "words": 57
    },
    {
      "compression_ratio": null,
      "kind": "paragraph",
      "lexical_density": 0.4839,
      "readability_grade": 10.2485,
      "sentences": 3,
      "text": "Summarization and keyphrase extraction are not part of this. The pipeline never calls them. They are free functions over the analysed Document, invoked by the caller, and they need no engine:",
      "words": 31
    },
    {
      "compression_ratio": 0.606,
      "kind": "paragraph",
      "lexical_density": 0.5517,
      "readability_grade": 11.0202,
      "sentences": 4,
      "text": "Keeping them out of the pipeline is a cost decision. TextRank builds a dense sentence-similarity matrix that is quadratic in sentence count, roughly 32 MB of f64 at its 2,000-sentence ceiling. Folding it into the analysis pass would charge that to every caller who only wanted a readability grade. Inputs above a cap return Error::InputTooLarge rather than allocating.",
      "words": 61
    },
    {
      "compression_ratio": null,
      "kind": "paragraph",
      "lexical_density": 0.6154,
      "readability_grade": 4.0031,
      "sentences": 1,
      "text": "Every guard sits at the earliest point where the check is still cheap.",
      "words": 13
    },
    {
      "compression_ratio": 0.5574,
      "kind": "paragraph",
      "lexical_density": 0.5769,
      "readability_grade": 7.7354,
      "sentences": 3,
      "text": "Three separate lanes, because these are three separate call paths. Only the middle one runs during a pipeline call. The model check fires once when you construct the engine, and the outcome caps fire only when you call an outcome function, which is why neither shows up in a pipeline stack trace.",
      "words": 52
    },
    {
      "compression_ratio": null,
      "kind": "paragraph",
      "lexical_density": 0.5263,
      "readability_grade": 10.1411,
      "sentences": 2,
      "text": "The catch_unwind is the one that matters. A panic crossing an FFI boundary does not unwind into your code, it aborts the process: interpreter death in Python, a trap in WASM, from a library that promised typed errors.",
      "words": 38
    },
    {
      "compression_ratio": 0.576,
      "kind": "paragraph",
      "lexical_density": 0.4915,
      "readability_grade": 8.9625,
      "sentences": 3,
      "text": "The boundary that converts such a panic into an Error lives in nlp/udpipe.rs, and that file is the only one permitted to import the UDPipe crate. A second import path would be a second process-abort surface with no boundary in front of it. That boundary covers parsing only; model loading translates the loader's own error and is not wrapped.",
      "words": 62
    },
    {
      "compression_ratio": 0.561,
      "kind": "paragraph",
      "lexical_density": 0.5424,
      "readability_grade": 13.715,
      "sentences": 2,
      "text": "Every cap a gate checks, and every minimum a metric needs, is a constant in domain::limits, defined once and read where it applies. It sits inside the domain, the one module every row may import, so the gates in the adapters, the outcome functions and the metrics all read the same number without boundary rule 5 bending for it.",
      "words": 59
    },
    {
      "compression_ratio": 0.4886,
      "kind": "paragraph",
      "lexical_density": 0.551,
      "readability_grade": 10.581,
      "sentences": 5,
      "text": "The parser and the readers are ports anyone can implement, so the engine trusts none of them, matra's own included: what a reader returns is checked once per document, and what the parser returns once per paragraph, before either reaches a Document. A break is Error::ParseFailed with a reason naming the check and a cause naming the token or block (Errors lists the checks). Each check is a pass over what it reads. Measured over the docsite's inputs and its own reference and guide pages, nearly two thousand paragraphs, the parse checks cost about 0.014% of UDPipe's parse time, roughly 3 microseconds a paragraph, and the reader checks about 7% of the readers' own time, under a tenth of a millisecond for a 140 KB page: together 0.015% of reading and parsing. Every document an engine returns therefore holds the invariants Sentence documents, whichever parts built it.",
      "words": 156
    },
    {
      "compression_ratio": 0.555,
      "kind": "paragraph",
      "lexical_density": 0.5333,
      "readability_grade": 12.6285,
      "sentences": 4,
      "text": "Two guards live in the domain types rather than at an edge, because a document can arrive from outside the engine, read back from JSON or built by hand, and its shape is not guaranteed. Sentence::tree_depth is computed by walking head references with a visited set, and is None when they form a cycle. Sentence::subtree carries the same visited set so it terminates. The None is deliberate: an earlier version capped depth at a magic 20, which silently truncated malformed parses and legitimate deep ones alike, and a sentinel number would read as a very deep tree to anyone who did not check for it.",
      "words": 106
    },
    {
      "compression_ratio": 0.6127,
      "kind": "paragraph",
      "lexical_density": 0.62,
      "readability_grade": 11.442,
      "sentences": 3,
      "text": "Directory reads fail differently. analyze_all yields successes and per-document failures side by side, so one unreadable file does not cost you the other ninety-nine; Inputs::dir is Err only when the listing itself fails. Full detail on every gate, every variant, and the Python exception each maps to is in Errors.",
      "words": 52
    },
    {
      "compression_ratio": 0.4829,
      "kind": "paragraph",
      "lexical_density": 0.5556,
      "readability_grade": 10.3169,
      "sentences": 4,
      "text": "Every arrow points down, and that is the rule: a module imports from the row below it, never the row above. One arrow runs sideways: src/bin/matra.rs reaches into src/cli/, which is the whole of that file: the command line lives in the library so both launchers run one program, and the binary only collects arguments, locks the streams, and returns the code. One arrow from src/cli/ reaches past lib.rs, and it still points down: matra clusters builds Model2Vec from the configuration, because the library builds no embedder inside an outcome call and choosing the default one is the application's decision. Boundary rule 7 admits embed for the command line as it does for the Python binding; the other three ports are public too, and neither needs them, so the rule keeps them out of both.",
      "words": 135
    },
    {
      "compression_ratio": 0.5414,
      "kind": "paragraph",
      "lexical_density": 0.5342,
      "readability_grade": 10.4398,
      "sentences": 3,
      "text": "src/config.rs sits in the composition-root row beside lib.rs, and it is the one box an adapter is allowed to import from the row above. Udpipe::from_config and Model2Vec::from_config take a &Config, which is the traffic RFC-0011 intends and boundary rule 7 records: Config imports no port and no adapter, so nothing travels back down with it, and an adapter that reads it is reading a resolved path rather than reaching for the environment itself.",
      "words": 79
    },
    {
      "compression_ratio": 0.5547,
      "kind": "paragraph",
      "lexical_density": 0.5574,
      "readability_grade": 13.6187,
      "sentences": 3,
      "text": "Config is not one of the extension points below, and the distinction is worth keeping. An extension point substitutes an implementation; Config only supplies values a caller could have passed as arguments. That is why it carries locations and defaults and never behavior: the moment a file selects which metrics run, configuration stops being adaptability and becomes dispatch that nothing type-checks.",
      "words": 62
    },
    {
      "compression_ratio": 0.5021,
      "kind": "paragraph",
      "lexical_density": 0.5517,
      "readability_grade": 12.0914,
      "sentences": 6,
      "text": "Every port is open to outside implementers, and each is one or two methods over domain types. NlpProvider has two, parse and identity: a paragraph in, sentences of tokens out, and who parsed them, which the engine records in every document's provenance. Decomposer has one, decompose: a format's bytes in, blocks and skipped constructs out, built with Read::new and ReadBlock::heading and ReadBlock::paragraph. Source has one, read: a path in, bytes out. Embedder is the same shape one tier over: clusters takes it as &dyn Embedder, and its identity travels into every result's provenance so the scores stay attributable to the model that produced them. That is what makes UDPipe an implementation detail rather than the library.",
      "words": 123
    },
    {
      "compression_ratio": 0.4636,
      "kind": "paragraph",
      "lexical_density": 0.5581,
      "readability_grade": 9.7785,
      "sentences": 6,
      "text": "The engine holds a parser, a table mapping each Format to a boxed reader, and a source, and a caller replaces any of them without forking matra. Engine::with_parser(parser) builds an engine around a parser of your own, in any build, with the shipped readers and the file source; .with_reader(format, reader) registers a reader for a format, replacing a shipped one, supplying one for Format::Pdf or Format::Docx, or reading a Format::Other you name; .with_source(source) reads the paths inputs name from somewhere else. Both methods compose with the standard constructors, so Engine::new()?.with_reader(Format::Pdf, MyPdf) is UDPipe with your PDF reader. It is one constructor and two consuming methods rather than a builder type: every part but the parser has a default, so there is nothing for a builder to hold that the engine does not already hold. Inputs::dir still lists the local filesystem, and Input::bytes and Input::blocks reach no source and no reader, so a caller with bytes or blocks from anywhere needs neither. Bring your own parser or reader walks through one of each.",
      "words": 186
    },
    {
      "compression_ratio": 0.5072,
      "kind": "paragraph",
      "lexical_density": 0.6154,
      "readability_grade": 12.0847,
      "sentences": 4,
      "text": "The traits are frozen once someone implements them, since a new method would break every implementation, so they ask for only what the engine cannot derive. A parser returns tokens, not Sentence values: the sentence's text, span and every field derived from its tree are the engine's, computed the same way whichever parser ran. A reader returns opaque blocks, so the block vocabulary can grow without breaking one. And neither trait promises well-formed output in its signature; the engine checks it instead, at the one place each kind of output enters.",
      "words": 95
    },
    {
      "compression_ratio": 0.5704,
      "kind": "paragraph",
      "lexical_density": 0.4737,
      "readability_grade": 8.1744,
      "sentences": 3,
      "text": "The metric suite is data as well, inside the crate: a list of functions over &mut Document that compose runs. It is private, so a new measure is a function and a line in the list. A caller who wants another measure writes a function over the Document it was handed, which is all a metric is.",
      "words": 58
    },
    {
      "compression_ratio": 0.5877,
      "kind": "paragraph",
      "lexical_density": 0.5185,
      "readability_grade": 9.2063,
      "sentences": 4,
      "text": "What you gain from the arrangement is concrete and mostly shows up in test suites. Nothing under metrics/ or extraction/ imports a port, which is the one column above that skips the port row entirely, so those functions run with no model loaded. A new metric is testable against hand-built Sentence values, with no 16 MB download and no C++ toolchain. The rules that keep it that way, and how weakly some of them are enforced, are listed in Boundary rules.",
      "words": 83
    },
    {
      "compression_ratio": 0.5858,
      "kind": "paragraph",
      "lexical_density": 0.5926,
      "readability_grade": 8.0306,
      "sentences": 4,
      "text": "The Rust side runs the same pipeline, then hands the Document to pythonize, which walks the serde representation and builds a Python dict. That is a full deep copy. Every token becomes a dict of eleven keys. For a document with seven thousand tokens, that is seven thousand dicts allocated on the Python heap.",
      "words": 54
    },
    {
      "compression_ratio": 0.5866,
      "kind": "paragraph",
      "lexical_density": 0.5882,
      "readability_grade": 10.1905,
      "sentences": 5,
      "text": "Fields cross. Methods do not. A Rust method has no serde representation, so it has nothing to cross with, which is why every aggregate a reader needs is stored: compose writes the counts, sentence-length statistics and ratios into DocumentMetrics and ParagraphMetrics, and Sentence::new sets each sentence's word_count, tree_depth and passive (RFC-0008). A Python caller reads them rather than recomputing them. Domain types draws that boundary member by member.",
      "words": 72
    },
    {
      "compression_ratio": 0.5135,
      "kind": "paragraph",
      "lexical_density": 0.5455,
      "readability_grade": 9.5104,
      "sentences": 5,
      "text": "Errors cross by type and as data. Each variant is raised as a class of its own, named as the variant is and deriving from matra.MatraError, and the exception carries kind and details, the values Error::kind and Error::details compute, so a method's answer crosses as a field. The choice of class is a match with no wildcard arm, so adding a variant to domain::Error fails to compile until someone decides which class it becomes. A wildcard would let new failure modes fall through to another variant's class unnoticed.",
      "words": 93
    },
    {
      "compression_ratio": null,
      "kind": "paragraph",
      "lexical_density": 0.6429,
      "readability_grade": 7.9914,
      "sentences": 2,
      "text": "Every Python method that takes text routes through the same pipeline, so the 8 MiB text cap fires uniformly. The outcome methods add their per-method caps on top.",
      "words": 28
    },
    {
      "compression_ratio": null,
      "kind": "paragraph",
      "lexical_density": 0.5556,
      "readability_grade": 10.4411,
      "sentences": 1,
      "text": "What matra gives you is the output side: every value the pipeline produces and what each one measures.",
      "words": 18
    },
    {
      "compression_ratio": null,
      "kind": "paragraph",
      "lexical_density": 0.5,
      "readability_grade": 8.14,
      "sentences": 1,
      "text": "Domain types is the type graph: what each type owns, which values are stored and which are computed on demand.",
      "words": 20
    },
    {
      "compression_ratio": null,
      "kind": "paragraph",
      "lexical_density": 0.45,
      "readability_grade": 10.5,
      "sentences": 1,
      "text": "Boundary rules is the enforcement side: the eight rules, why each exists, and which of them have a mechanical gate.",
      "words": 20
    }
  ]
}
