# Read a Markdown document's structure

A short Markdown document with two headings and a quote, read into sections and paragraphs.

## Input

[Field notes: a short Markdown document with a heading, a quote and two sections](https://github.com/mox-labs/matra/blob/main/site/inputs/field-notes.txt), matra contributors. MIT, as part of the matra repository.

The calls read it as `field-notes.md` in the directory they run from:

```bash
curl -fLO https://mox-labs.github.io/matra/example-files/markdown-structure/field-notes.md
```
## The call

Rust:

```rust
// cargo add matra@0.3 serde_json
use matra::{Engine, Input};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Loads the English model, downloading it on first use.
    let engine = Engine::new()?;
    let doc = engine.analyze(Input::file("field-notes.md"))?;
    println!("{}", serde_json::to_string_pretty(&doc)?);
    Ok(())
}
```

Python:

```python
# pip install 'matra>=0.3'
import json

import matra

engine = matra.Engine()  # downloads the English model on first use
doc = engine.analyze(file="field-notes.md")  # a str alone would be text, not a path
print(json.dumps(doc.to_dict(), indent=2, ensure_ascii=False))
```

The command line, which prints the output below inside an envelope:

```bash
matra analyze --json field-notes.md
```

```jsonc
{
  "format_version": 2,
  "command": "analyze",
  "input": "field-notes.md",
  "result": /* the output below */,
  "error": null
}
```
## Output

Trimmed: `sentences` shown as a count, long strings shortened.

```jsonc
{
  "provenance": {
    "matra": "0.3.0",
    "schema": "2.0",
    "parser": {
      "name": "udpipe",
      "model": "english-ewt-ud-2.5-191206",
      "sha256": "784bd0fa85e3d831fd02a55290d0acfd05c953159dc38cc33d52e1b28add9957",
      "license": "CC BY-NC-SA 4.0 (non-commercial)"
    },
    "format": "markdown",
    "source": {
      "path": "field-notes.md",
      "bytes": 564,
      "sha256": "bbf7f1c703c0bf2b998b1a8fc61e1fe0f0e6aeb670b169fd5e2da372c62fe55f"
    }
  },
  "sections": [
    {
      "heading": "Field notes",
      "level": 1,
      "span": {
        "byte_start": 0,
        "byte_end": 13
      },
      "paragraphs": [
        {
          "span": {
            "byte_start": 15,
            "byte_end": 156
          },
          "text": "The survey team reached the northern ridge on the third day. Snow had c…",
          "kind": "paragraph",
          "sentences": [ /* 2 items */ ],
          "metrics": {
            "readability_grade": 4.91076923076923,
            "lexical_density": 0.6538461538461539,
            "compression_ratio": null
          }
        },
        {
          "span": {
            "byte_start": 158,
            "byte_end": 197
          },
          "text": "Radio contact was lost for six hours.",
          "kind": "quote",
          "sentences": [],
          "metrics": {
            "readability_grade": null,
            "lexical_density": null,
            "compression_ratio": null
          }
        }
      ]
    },
    {
      "heading": "Findings",
      "level": 2,
      "span": {
        "byte_start": 199,
        "byte_end": 210
      },
      "paragraphs": [
        {
          "span": {
            "byte_start": 212,
            "byte_end": 563
          },
          "text": "Three of the five markers were intact and matched their recorded positi…",
          "kind": "paragraph",
          "sentences": [ /* 4 items */ ],
          "metrics": {
            "readability_grade": 6.8750000000000036,
            "lexical_density": 0.484375,
            "compression_ratio": 0.5897435897435898
          }
        }
      ]
    }
  ],
  "metrics": {
    "total_words": 91,
    "total_sentences": 6,
    "paragraph_count": 3,
    "mean_sentence_length": 15.166666666666666,
    "sentence_length_std": 3.3714487489307423,
    "vocabulary_ttr": 0.6923076923076923,
    "nominalization_ratio": 0.01098901098901099,
    "passive_ratio": 0.5
  },
  "skipped": []
}
```

The whole output: https://mox-labs.github.io/matra/example-files/markdown-structure/output.json, and what the CLI prints: https://mox-labs.github.io/matra/example-files/markdown-structure/cli.json.
[Programming model](../explanation/programming-model.md#two-values-input-into-engine) steps this same document through each stage of the pipeline, to show what each stage adds.

## What to notice

- `sections` follow the headings: each has `heading`, the heading's text, `level`, the number of `#` marks, and `span`, where the heading lies in the file, `#` marks included.
- The quote is kept as a paragraph whose `kind` is `quote`, and it is never parsed: its `sentences` list is empty and all three of its measures are `null`. Its `span` covers the `>` markers its text leaves out.
- The Markdown reader is chosen by the `.md` extension in all three calls. Read as plain text, the same file is one section with no heading, and the heading lines and the quote are parsed as sentences, `#` marks and all.
