# Extract a document's keyphrases

The ten phrases RAKE ranks highest in the same Darwin text as the [summary example](summarize.md).

## Input

[On the Origin of Species, the opening of chapter III, Struggle for Existence](https://www.gutenberg.org/ebooks/1228), Charles Darwin. Public domain (first published 1859).

The calls read it as `origin-struggle.txt` in the directory they run from:

```bash
curl -fLO https://mox-labs.github.io/matra/example-files/keyphrases/origin-struggle.txt
```
## The call

Rust:

```rust
// cargo add matra@0.3 serde_json
use matra::domain::Format;
use matra::{Engine, Input, KeyphraseMethod};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let engine = Engine::new()?;
    let text = std::fs::read_to_string("origin-struggle.txt")?;
    let doc = engine.analyze(Input::text(text, Format::PlainText))?;
    // Parse once; keyphrases are a function of the document.
    let phrases = matra::keyphrases_with(&doc, 10, KeyphraseMethod::Rake)?;
    println!("{}", serde_json::to_string_pretty(&phrases)?);
    Ok(())
}
```

Python:

```python
# pip install 'matra>=0.3'
import json

import matra

engine = matra.Engine()
with open("origin-struggle.txt", encoding="utf-8") as f:
    doc = engine.analyze(f.read())  # read as plain text
# Parse once; keyphrases are a function of the document.
phrases = matra.keyphrases(doc, n=10, method="rake")
print(json.dumps(phrases, indent=2, ensure_ascii=False))
```

The command line, which prints the output below inside an envelope:

```bash
matra keyphrases --method rake -n 10 --json --stdin-filename origin-struggle.txt - < origin-struggle.txt
```

```jsonc
{
  "format_version": 2,
  "command": "keyphrases",
  "input": "origin-struggle.txt",
  "result": /* the output below */,
  "error": null
}
```
## Output

Trimmed: lists cut to the first 5.

```jsonc
{
  "provenance": {
    "matra": "0.3.0",
    "schema": "2.0",
    "parser": {
      "name": "udpipe",
      "model": "english-ewt-ud-2.5-191206",
      "sha256": "784bd0fa85e3d831fd02a55290d0acfd05c953159dc38cc33d52e1b28add9957",
      "license": "CC BY-NC-SA 4.0 (non-commercial)"
    },
    "format": "plain_text",
    "source": {
      "path": null,
      "bytes": 12808,
      "sha256": "858cb32e93b9a95293e00afae7bd4049e8c0fda79abca22ea01b456f2a18a580"
    },
    "method": "rake",
    "parameters": {
      "n": 10
    }
  },
  "phrases": [
    {
      "phrase": "larger domestic animal tend",
      "score": 12.636363636363637,
      "occurrences": [
        {
          "byte_start": 10132,
          "byte_end": 10161
        }
      ]
    },
    {
      "phrase": "elder de candolle",
      "score": 9,
      "occurrences": [
        {
          "byte_start": 3228,
          "byte_end": 3245
        }
      ]
    },
    {
      "phrase": "fulmar petrel lay",
      "score": 9,
      "occurrences": [
        {
          "byte_start": 10791,
          "byte_end": 10809
        }
      ]
    },
    {
      "phrase": "mere theoretical calculation",
      "score": 8.5,
      "occurrences": [
        {
          "byte_start": 8016,
          "byte_end": 8045
        }
      ]
    },
    {
      "phrase": "great horticultural knowledge",
      "score": 8.2,
      "occurrences": [
        {
          "byte_start": 3501,
          "byte_end": 3530
        }
      ]
    }
    // … 5 more
  ]
}
```

The whole output: https://mox-labs.github.io/matra/example-files/keyphrases/output.json, and what the CLI prints: https://mox-labs.github.io/matra/example-files/keyphrases/cli.json.
RAKE against YAKE on the same text, as ranks:

<figure-keyphrases input="origin-struggle" />

## What to notice

- The result carries the text's `provenance` with `method` `rake` and `parameters` `{"n": 10}`. Each of its `phrases` has `phrase`, `score` and `occurrences`, the span of every place in the input it was found, highest score first, and a higher score means more relevant. RAKE's score sums each word's co-occurrence degree over its frequency.
- Phrases are built from lemmas, so the parser's lemma errors show through. `great destruction fal` is "great destruction falling", with `falling` read as `fal`, and `slow -breeding man` is "slow-breeding man" split at the hyphen. RAKE takes the words between stopwords and punctuation, which is why `larger domestic animal tend` runs "animals tends" together.
- Two phrases tie at 9.0. Equal scores come back in the order the phrases first occur, the same on every run, so a cut that falls inside a tie is decided by position in the text, not by merit. This example asks for ten, where the tenth scores 5.972 and no phrase outside the list ties it, so no tie decides the list.
- The figure shows how little RAKE and YAKE agree on a text like this one. Neither method is right; they rank by different evidence ([Methodology](../reference/methodology.md)).
