matra

on this page
grade9.3density0.49compression0.53

matra reports the structure of text and measurements over it. It parses a document into a typed tree of sections, paragraphs, sentences, and tokens, measures that tree, and ranks what is in it. Every value it returns is structure or a number over structure. What any of it means for your purpose is your code's decision, which is what makes the output reusable across purposes.

Four tiers of output

  • Structure. Sections, paragraphs, sentences, and full CoNLL-U tokens, plus the structural primitives read off each dependency tree: negations, modals, clausal complements, root adverbials, Hearst pairs, and the root clause's mood.
  • Measures. Readability, lexical density, and compression ratio per paragraph; counts, sentence length, vocabulary TTR, nominalization ratio, and passive ratio per document. Every document also records where it came from: the parser and its model, and a digest of the analysed text.
  • Extraction. Two extractive summarizers (TF-IDF, TextRank) and two keyphrase extractors (RAKE, YAKE).
  • Semantic clusters. Sentence embeddings grouped by cosine similarity, behind the model2vec feature, carrying the identity of the model that produced them.

End to end

import matra

engine = matra.Engine()
text = (
    "The committee approved the proposal without debate. "
    "Three amendments were submitted by the working group."
)
doc = engine.analyze(text)

print(doc["metrics"]["passive_ratio"])
print(doc["sections"][0]["paragraphs"][0]["metrics"]["readability_grade"])
gradenonedensity0.40compressionnone

The same analysis from Rust, and from the command line:

use matra::domain::Format;
use matra::{Engine, Input};

let engine = Engine::new()?;

let text = "The committee approved the proposal without debate. \
            Three amendments were submitted by the working group.";
let doc = engine.analyze(Input::text(text, Format::PlainText))?;

println!("{:?}", doc.metrics.passive_ratio);
$ matra analyze essay.md
essay.md
  sentences          35
  words              332
  mean sentence len  9.5
  sentence len sd    7.4
  passive ratio      0.057
grade7.1density0.52compressionnone

All three return the same shape: a struct in Rust, a read-only Document that reads as a dict in Python, a table or --json at the command line. The second sentence in the example is passive, and amendments carries the relation nsubj:pass. matra records that and stops there.

Where to go next

grade8.8density0.53compressionnone

Understand the model. Concepts covers the document tree, the four tiers, and what a structural primitive is. Situation model covers what the output is for and what it deliberately withholds.

grade6.2density0.63compressionnone

Write against it. Programming model is the surface: Input, Engine, ports, bounds, and what Python exposes. Pragmatics answers the choices you have to make.

gradenonedensity0.67compressionnone

Get it running. Installation, then Rust, Python, or CLI.

grade7.0density0.73compressionnone

Look something up. What matra gives you lists every field and function the pipeline returns.

This page, measured by matra 0.3.0 with english-ewt-ud-2.5-191206: 7 paragraphs, in the margin. the data