Your first analysis

on this page
grade12.7density0.57compressionnone

You will run matra on two short paragraphs of meeting minutes, then find four things in what it returns: the document's structure, the grammar it read off each sentence, the measures of each paragraph, and the measures of the whole document. You need matra installed; Installation covers every route.

Save the text

Minutes of a committee meeting, two paragraphs, matra contributors. MIT, as part of the matra repository. 36 words.

The calls read it as minutes.txt in the directory you run them from. Save it there:

curl -fLO https://mox-labs.github.io/matra/example-files/first-analysis/minutes.txt

or download minutes.txt.

The committee did not approve the budget on Tuesday. The chair said that a second vote may follow next week.

Three amendments were submitted by the working group. Each one narrows the scope of the plan.

Run matra

grade3.9density0.53compressionnone

Pick the language you will use. Each tab installs matra and runs the same analysis.

Rust

1Install

$ cargo new first-analysis && cd first-analysis
$ cargo add matra@0.3 serde_json

2Run: save this as src/main.rs, then run it

// cargo add matra@0.3 serde_json
use matra::{Engine, Input};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Loads the English model, downloading it on first use.
    let engine = Engine::new()?;
    let doc = engine.analyze(Input::file("minutes.txt"))?;
    println!("{}", serde_json::to_string_pretty(&doc)?);
    Ok(())
}
$ cargo run

Python

1Install

$ pip install 'matra>=0.3'

2Run: save this as first_analysis.py, then run it

# pip install 'matra>=0.3'
import json

import matra

engine = matra.Engine()  # downloads the English model on first use
doc = engine.analyze(file="minutes.txt")  # a str alone would be text, not a path
print(json.dumps(doc.to_dict(), indent=2, ensure_ascii=False))
$ python first_analysis.py

CLI

1Install

$ pip install 'matra>=0.3'   # or: cargo install matra --version '^0.3' --features cli

2Run

$ matra analyze --json minutes.txt

It prints the output below, inside an envelope that names the command and the input:

{
  "format_version": 2,
  "command": "analyze",
  "input": "minutes.txt",
  "result": /* the output below */,
  "error": null
}

Read the output

grade2.6density0.55compressionnone

Every tab prints the same value. The words of each sentence are shown here as a count; the next section finds them.

Trimmed: tokens shown as a count, long strings shortened.

{
  "provenance": {
    "matra": "0.3.0",
    "schema": "2.0",
    "parser": {
      "name": "udpipe",
      "model": "english-ewt-ud-2.5-191206",
      "sha256": "784bd0fa85e3d831fd02a55290d0acfd05c953159dc38cc33d52e1b28add9957",
      "license": "CC BY-NC-SA 4.0 (non-commercial)"
    },
    "format": "plain_text",
    "source": {
      "path": "minutes.txt",
      "bytes": 204,
      "sha256": "b4f55f24335ecfae3b641caec0db70fb9566f76143bba54401a99869e29185d2"
    }
  },
  "sections": [
    {
      "heading": null,
      "level": 0,
      "span": null,
      "paragraphs": [
        {
          "span": {
            "byte_start": 0,
            "byte_end": 108
          },
          "text": "The committee did not approve the budget on Tuesday. The chair said tha…",
          "kind": "paragraph",
          "sentences": [
            {
              "span": {
                "byte_start": 0,
                "byte_end": 52
              },
              "text": "The committee did not approve the budget on Tuesday.",
              "tokens": [ /* 10 items */ ],
              "word_count": 9,
              "tree_depth": 2,
              "passive": false,
              "negations": [
                {
                  "cue_id": 4,
                  "cue_lemma": "not",
                  "head_id": 5
                }
              ],
              "modals": [],
              "clausal_complements": [],
              "root_adverbials": [
                {
                  "adv_id": 4,
                  "adv_lemma": "not"
                }
              ],
              "hearst_pairs": [],
              "root_mood": {
                "root_id": 5,
                "mood": "Ind"
              }
            },
            {
              "span": {
                "byte_start": 53,
                "byte_end": 108
              },
              "text": "The chair said that a second vote may follow next week.",
              "tokens": [ /* 12 items */ ],
              "word_count": 11,
              "tree_depth": 3,
              "passive": false,
              "negations": [],
              "modals": [
                {
                  "aux_id": 8,
                  "aux_lemma": "may",
                  "head_id": 9
                }
              ],
              "clausal_complements": [
                {
                  "verb_id": 3,
                  "verb_lemma": "say",
                  "ccomp_id": 9,
                  "subject_id": 2,
                  "subject_lemma": "chair"
                }
              ],
              "root_adverbials": [],
              "hearst_pairs": [],
              "root_mood": {
                "root_id": 3,
                "mood": "Ind"
              }
            }
          ],
          "metrics": {
            "readability_grade": 3.650000000000002,
            "lexical_density": 0.55,
            "compression_ratio": null
          }
        },
        {
          "span": {
            "byte_start": 110,
            "byte_end": 203
          },
          "text": "Three amendments were submitted by the working group. Each one narrows …",
          "kind": "paragraph",
          "sentences": [
            {
              "span": {
                "byte_start": 110,
                "byte_end": 163
              },
              "text": "Three amendments were submitted by the working group.",
              "tokens": [ /* 9 items */ ],
              "word_count": 8,
              "tree_depth": 2,
              "passive": true,
              "negations": [],
              "modals": [],
              "clausal_complements": [],
              "root_adverbials": [],
              "hearst_pairs": [],
              "root_mood": {
                "root_id": 4,
                "mood": "Ind"
              }
            },
            {
              "span": {
                "byte_start": 164,
                "byte_end": 203
              },
              "text": "Each one narrows the scope of the plan.",
              "tokens": [ /* 9 items */ ],
              "word_count": 8,
              "tree_depth": 3,
              "passive": false,
              "negations": [],
              "modals": [],
              "clausal_complements": [],
              "root_adverbials": [],
              "hearst_pairs": [],
              "root_mood": {
                "root_id": 3,
                "mood": "Ind"
              }
            }
          ],
          "metrics": {
            "readability_grade": 3.7550000000000026,
            "lexical_density": 0.5625,
            "compression_ratio": null
          }
        }
      ]
    }
  ],
  "metrics": {
    "total_words": 36,
    "total_sentences": 4,
    "paragraph_count": 2,
    "mean_sentence_length": 9,
    "sentence_length_std": 1.4142135623730951,
    "vocabulary_ttr": 0.8611111111111112,
    "nominalization_ratio": 0,
    "passive_ratio": 0.25
  },
  "skipped": []
}

The whole output: output.json, and what the CLI prints, cli.json. The examples gate runs all three calls and compares what each prints with these files.

Find the structure

grade5.6density0.53compressionnone

The output is a tree. The document holds sections; this one has a single section, because the text has no headings. The section holds paragraphs, two of them, one for each block of text separated by a blank line. Each paragraph holds its sentences, and each sentence its tokens.

grade5.2density0.51compressionnone

The tokens are shown above only as a count. Each one carries its text, its lemma, its part of speech (pos), the id of the word it depends on (head) and the relation between them (dep). The quick start shows one sentence's tokens in full.

Find the grammar

grade6.0density0.47compressionnone

matra reads a few facts off each sentence's parse and stores them on the sentence.

  1. In the first sentence, negations holds one entry. Its cue_lemma is not, and its head_id is 5: not negates the fifth word, approve.
  2. In the second sentence, modals holds may, over word 9, follow. clausal_complements holds say: the chair (subject_lemma) said something, and what was said is the clause headed by word 9.
  3. The third sentence is passive: the amendments were submitted, and its passive is true. Every sentence also carries its word_count, its tree_depth, and its root_mood: the grammatical mood of its main clause, Ind for a plain statement.

Find the paragraph measures

grade9.0density0.48compression0.57

Each paragraph carries three measures in its metrics. readability_grade is a grade level, and lexical_density is the share of content words. compression_ratio is null for both paragraphs: matra computes it only for a paragraph of more than 50 words, and these are shorter. A null means the measure declined to run, never that it came out as zero.

Find the document measures

grade7.8density0.51compression0.56

The document's own metrics describe the whole of it: the counts (total_words, total_sentences, paragraph_count), the mean and spread of sentence length, and three ratios. passive_ratio is 0.25: one sentence of the four is passive. vocabulary_ttr is the share of distinct lemmas among all of them. nominalization_ratio is the share of words that are nouns ending in a suffix such as -tion or -ment. It is 0 here: the one candidate, amendments, ends in an s, and the test reads the word as written (Methodology has the rule).

Where to go next

  • How-to guides: the same calls on longer, real texts, one task each.
  • Concepts: why the output is a tree, and what each part of it records.
  • Domain model: every field named above, with its type.

This page, measured by matra 0.3.0 with english-ewt-ud-2.5-191206: 8 paragraphs, in the margin. the data