Parse a sentence into its dependency tree

on this page
grade11.5density0.48compressionnone

The first sentence of Pride and Prejudice, parsed: every word with its lemma, its part of speech, the word it depends on, and the relation between them.

Input

Pride and Prejudice, the first sentence, Jane Austen. Public domain (first published 1813). 23 words.

The calls read it as austen-first-sentence.txt in the directory you run them from. Save it there:

curl -fLO https://mox-labs.github.io/matra/example-files/parse-a-sentence/austen-first-sentence.txt

or download austen-first-sentence.txt.

It is a truth universally acknowledged, that a single man in possession of a good fortune must be in want of a wife.

The call

Rust

// cargo add matra@0.3 serde_json
use matra::{Engine, Input};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Loads the English model, downloading it on first use.
    let engine = Engine::new()?;
    let doc = engine.analyze(Input::file("austen-first-sentence.txt"))?;
    println!("{}", serde_json::to_string_pretty(&doc)?);
    Ok(())
}

Python

# pip install 'matra>=0.3'
import json

import matra

engine = matra.Engine()  # downloads the English model on first use
doc = engine.analyze(file="austen-first-sentence.txt")  # a str alone would be text, not a path
print(json.dumps(doc.to_dict(), indent=2, ensure_ascii=False))

CLI

$ matra analyze --json austen-first-sentence.txt

It prints the output below, inside an envelope that names the command and the input:

{
  "format_version": 2,
  "command": "analyze",
  "input": "austen-first-sentence.txt",
  "result": /* the output below */,
  "error": null
}

Output

Trimmed: only result.sections[0].paragraphs[0].sentences[0].tokens, lists cut to the first 3.

[
  {
    "id": 1,
    "span": {
      "byte_start": 0,
      "byte_end": 2
    },
    "text": "It",
    "lemma": "it",
    "pos": "PRON",
    "xpos": "PRP",
    "features": {
      "Case": "Nom",
      "Gender": "Neut",
      "Number": "Sing",
      "Person": "3",
      "PronType": "Prs"
    },
    "head": 4,
    "dep": "nsubj",
    "deps": "_",
    "misc": "",
    "is_punct": false
  },
  {
    "id": 2,
    "span": {
      "byte_start": 3,
      "byte_end": 5
    },
    "text": "is",
    "lemma": "be",
    "pos": "AUX",
    "xpos": "VBZ",
    "features": {
      "Mood": "Ind",
      "Number": "Sing",
      "Person": "3",
      "Tense": "Pres",
      "VerbForm": "Fin"
    },
    "head": 4,
    "dep": "cop",
    "deps": "_",
    "misc": "",
    "is_punct": false
  },
  {
    "id": 3,
    "span": {
      "byte_start": 6,
      "byte_end": 7
    },
    "text": "a",
    "lemma": "a",
    "pos": "DET",
    "xpos": "DT",
    "features": {
      "Definite": "Ind",
      "PronType": "Art"
    },
    "head": 4,
    "dep": "det",
    "deps": "_",
    "misc": "",
    "is_punct": false
  }
  // … 22 more
]

The whole output: output.json, and what the CLI prints, cli.json. The examples gate runs all three calls and compares what each prints with these files.

grade2.1density0.40compressionnone

The same 25 tokens as a table, then as arcs drawn from each word's head:

Dependency parse

1It is a truth universally acknowledged, that a single man in possession of a good fortune must be in want of a wife.

Dependency arcs for: It is a truth universally acknowledged, that a single man in possession of a good fortune must be in want of a wife.rootdetadvmodpunctamodcaseamodcasedetcopconjdetnmoddetcopcasensubjcaseauxnmodnmodnsubjobjccomppunctItPRONisAUXaDETtruthNOUNuniversallyADVacknowledgedVERB,PUNCTthatPRONaDETsingleADJmanNOUNinADPpossessionNOUNofADPaDETgoodADJfortuneNOUNmustAUXbeAUXinADPwantNOUNofADPaDETwifeNOUN.PUNCT
show the data
#WordLemmaPOSHeadRelation
1ItitPRON4truthnsubj
2isbeAUX4truthcop
3aaDET4truthdet
4truthtruthNOUN0rootroot
5universallyuniversallyADV6acknowledgedadvmod
6acknowledgedacknowledgeVERB4truthconj
7,,PUNCT6acknowledgedpunct
8thatthatPRON21wantobj
9aaDET11mandet
10singlesingleADJ11manamod
11manmanNOUN21wantnsubj
12ininADP13possessioncase
13possessionpossessionNOUN11mannmod
14ofofADP17fortunecase
15aaDET17fortunedet
16goodgoodADJ17fortuneamod
17fortunefortuneNOUN13possessionnmod
18mustmustAUX21wantaux
19bebeAUX21wantcop
20ininADP21wantcase
21wantwantNOUN6acknowledgedccomp
22ofofADP24wifecase
23aaDET24wifedet
24wifewifeNOUN21wantnmod
25..PUNCT4truthpunct
  • Core argument
  • Modifier
  • Function word
  • Other

Each arc runs from a word's head to the word, whose arrow it ends at, labelled with the relation. The line groups the relations the way Universal Dependencies does: the core arguments of a predicate, the modifiers, and the function words. Point at or tap a word, here or in the table, to keep its arcs and dim the rest.

Input: Pride and Prejudice, the first sentence, Jane Austen. Public domain (first published 1813). Produced by matra 0.3.0 with the UDPipe model english-ewt-ud-2.5-191206 (SHA-256 784bd0fa85e3). Data: austen-first-sentence/parse.json.

What to notice

  • A document is sections of paragraphs of sentences, even when it is one sentence, so the tokens are at sections[0].paragraphs[0].sentences[0].tokens. In Rust, doc.sentences() walks every sentence in document order.
  • Each token carries its CoNLL-U columns under matra's names: id (from 1 within the sentence), text, lemma, pos (the Universal Dependencies tag), head (the id of the word it depends on, 0 for the root) and dep (the relation). Here truth is the root, and It is its nsubj.
  • lemma is the dictionary form: is becomes be, and acknowledged becomes acknowledge.
  • The parse is the model's, reported as the model returned it. The model reads that (token 8) as a pronoun and the object of want, where a grammar would call it the conjunction that opens the clause. matra does not correct the parser; Methodology names the model and its limits.
  • The whole output also carries the structural primitives read off this tree: a modal (must, governing want) and a clausal complement (acknowledge, whose clause is headed by want). The next example reads those across a document.

This page, measured by matra 0.3.0 with english-ewt-ud-2.5-191206: 2 paragraphs, in the margin. the data