Migrate from 0.2 to 0.3
on this page
- Update the version
- Every rename at a glance
- Rust
- Build the engine
- Analyze one document
- Analyze a directory
- Read the measures
- Read the record facts
- Summaries, keyphrases and clusters
- Errors
- What left the public surface
- Configuration
- Python
- Build the engine
- Analyze
- Read a document
- Summaries, keyphrases and clusters
- Exceptions
- Types
- Command line
- The config file
- The JSON envelope
- Failures under --json
- Standard input
- What is new
- The JSON record
- Results that change with no rename
0.3.0 changes Rust, Python, the command line, the JSON and the config file at once. Nothing keeps its old name. There are no deprecated aliases, no hidden flags, no fields written under both names and no setting that brings back the 0.2 output. A renamed or removed call fails at once in 0.3 rather than running with a different meaning: a compile error in Rust, an AttributeError or TypeError in Python, an unknown key in the config file, and a new format_version and schema major in the JSON. A few results do change for the same call, and the last section lists them.
This page lists every change a 0.2 caller meets, with the 0.2 form and the 0.3 form side by side. Work through the section for each surface you use.
Update the version
Move the version requirement first. The old requirement keeps you on 0.2, because cargo reads a change in the first non-zero component as breaking.
cargo add matra@0.3 # was matra@0.2
cargo install matra --version '^0.3' --features cli # was '^0.2'
pip install 'matra>=0.3' # was 'matra>=0.2'
uvx 'matra>=0.3' --skill # the skill for 0.3
Every rename at a glance
| 0.2 | 0.3 | Where |
|---|---|---|
Engine::new(nlp, decomposers), Engine::with_defaults() | Engine::new(), Engine::from_config(&Config::resolve()?) | Rust |
Ingest::text, Ingest::path | Input::text, Input::file, Inputs::dir | Rust |
Engine::analyze(Ingest), Engine::analyze_one | Engine::analyze(Input), Engine::analyze_all(inputs) | Rust |
Engine::annotate, Engine::compose | none: analyze is the call | Rust |
Ingested, RawDocument, CorpusEntry, Corpus, CorpusResult | none: Document, its path in provenance.source.path | Rust |
extraction::tfidf_summarize, textrank_summarize | summarize, summarize_with(.., SummaryMethod::TextRank) | Rust |
extraction::rake_keyphrases, yake_keyphrases | keyphrases, keyphrases_with(.., KeyphraseMethod::Yake) | Rust |
embed_and_cluster, extraction::semantic_clusters | clusters | Rust |
ScoredSentence, SemanticClusters, SemanticCluster, SemanticEdge | SummarySentence in a Summary, Clusters, Cluster, SimilarityEdge | Rust, Python types |
domain::MAX_INPUT_BYTES | domain::limits::MAX_INPUT_BYTES | Rust |
Error::InvalidInput(message) | Error::InvalidInput { reason, message } | Rust |
Error::ParseFailed(message) | Error::ParseFailed { reason, cause } | Rust |
Embedder::identity() -> &str | Embedder::identity() -> Identity | Rust |
Config::summarize_n, summarize_algorithm, keyphrases_algorithm | Config::summary_n, summary_method, keyphrases_method | Rust |
Config::udpipe_model, Config::semantic_threshold | none; Config::clusters_threshold is new | Rust |
Engine::new(Box::new(parser), decomposers) | Engine::with_parser(parser), then .with_reader(format, reader) per reader | Rust |
NlpProvider::parse -> Vec<Sentence> | NlpProvider::parse -> Vec<Vec<Token>>, one list of tokens per sentence | Rust |
Decomposer::decompose(&str) -> Vec<Section> | Decomposer::decompose(&[u8]) -> Result<Read>, built from ReadBlock::heading and ReadBlock::paragraph | Rust |
Source::read -> Vec<RawDocument>, Source::accepts | Source::read -> Vec<u8>; accepts is gone | Rust |
Udpipe, Decomposers, MarkdownDecomposer, PlainTextDecomposer, FileSource, DirectorySource, standard_decomposers, metrics, hearst, extraction | private to the crate | Rust |
Matra.english(), Matra.from_path(path) | matra.Engine() | Python |
Matra.analyze(text), analyze_markdown(text) | engine.analyze(text), engine.analyze(text, format="markdown") | Python |
Matra.analyze_path(path) | engine.analyze(file=path), engine.analyze_all(dir=path) | Python |
Matra.tfidf_summarize(text, n), textrank_summarize | matra.summarize(doc, n, method=...) | Python |
Matra.rake_keyphrases(text, max_phrases), yake_keyphrases | matra.keyphrases(doc, n, method=...) | Python |
Matra.semantic_clusters(text, threshold, model) | matra.clusters(doc, model, threshold) | Python |
matra.semantic_clusters(vectors, threshold, model_hash) | none: a cosine over the vectors you hold | Python |
an embedder's identity() -> str, Model2Vec.identity() | identity() returning an Identity dict, {name, model, sha256, license} | Python |
Model2Vec.potion_base_8m() reading the config file | Model2Vec.from_config() | Python |
Document, CorpusEntry, CorpusItem (types) | matra.Document (a class), DocumentDict, and Document or DocumentError items | Python |
paragraph["readability_grade"] and its siblings | paragraph["metrics"]["readability_grade"] | JSON, Python |
doc["vocabulary_ttr"] and its siblings | doc["metrics"]["vocabulary_ttr"] | JSON, Python |
Paragraph.in_blockquote | Paragraph.kind ("quote") | all |
Sentence.reportings, Reporting, reportings_in | Sentence.clausal_complements, ClausalComplement, clausal_complements_in | all |
Sentence.bare_assertion | Sentence.root_mood | all |
Token.feats (a string) | Token.features (a map) | all |
Sentence.tree_depth() returning usize::MAX on a cycle | Sentence.tree_depth, None on a cycle | all |
SemanticClusters.model_hash | Clusters.provenance.parameters.embedder.sha256 | all |
InputTooLarge what: file_source, config_file, udpipe_download, embedding_download | file_read, config_file_read, parser_model_download, embedding_model_download | all |
invalid_input details.reason, the message text | details.reason, the name of the check that failed | all |
model_invalid, parse_failed and io details.reason, the error's text | details.cause | all |
Format serialized as "Markdown", "PlainText" | "markdown", "plain_text", "blocks" | Rust serde, JSON |
[summarize], algorithm, [semantic] threshold, [models] udpipe | [summary], method, [clusters] threshold, none | config file |
--json envelope format_version: 1 | format_version: 2, with an error key | command line |
Rust
Build the engine
Engine::new takes no arguments and builds the pinned parser itself. It reads no configuration file. Engine::with_defaults is gone, because it read one under a name that did not say so.
// 0.2
let engine = Engine::with_defaults()?;
let engine = Engine::new(Box::new(Udpipe::english(dir)?), standard_decomposers());
// 0.3
let engine = Engine::new()?;
let engine = Engine::from_config(&Config::resolve()?)?; // reads the config file
let engine = Engine::from_config(&Config::resolve()?.with_model_dir(dir))?; // a model directory you name
There is no provider to construct and no reader table to pass for the standard pipeline. No surface loads a UDPipe model file of your choosing: the model is a pin.
A parser, reader or source of your own still plugs in, through the engine rather than a table:
// 0.2
let engine = Engine::new(Box::new(MyParser), standard_decomposers().with(Format::Markdown, Box::new(MyReader)));
// 0.3
let engine = Engine::with_parser(MyParser).with_reader(Format::Markdown, MyReader);
let engine = Engine::new()?.with_reader(Format::Pdf, MyPdfReader); // UDPipe, plus a reader of yours
let engine = Engine::new()?.with_source(MyStore); // paths read from somewhere else
The traits changed shape. NlpProvider::parse returns one Vec<Token> per sentence instead of Sentence values: the engine builds each sentence from the tokens, its text and every derived field included, so drop your Sentence::new call and return the tokens. Spans are required now, relative to the paragraph text parse was handed. Decomposer::decompose takes the format's bytes and returns a Result<Read> of ReadBlock values with their spans, where it returned sections; a text reader decodes the bytes itself. Source::read returns the document's bytes, and accepts is gone, since the engine never asked it. Whatever a parser or a reader returns is checked against its trait's contract, and a break is Error::ParseFailed naming the check (Errors); 0.2 passed it through unchecked. Bring your own parser or reader has the contracts.
Analyze one document
Input is one document. Engine::analyze returns its Document or the error. There is no stream to unwrap and no CorpusEntry around the document.
// 0.2
let analysis = engine
.analyze(Ingest::path("essay.md")?)
.next()
.unwrap()
.map_err(|e| e.error)?
.analysis;
let doc = engine
.analyze_one(RawDocument::new(text, None, Format::Markdown))
.map_err(|e| e.error)?
.analysis;
// 0.3
let doc = engine.analyze(Input::file("essay.md"))?;
let doc = engine.analyze(Input::text(text, Format::Markdown))?;
Input has five constructors: text, bytes, file, file_as (a file read as the format you name) and blocks (headings and paragraphs you supply for a format matra has no reader for). Input::file on a directory fails with InvalidInput, naming Inputs::dir.
Analyze a directory
Ingest::path on a directory is Inputs::dir, and analyze_all streams one result per file. A failed file is an Err item carrying its path. The path of a document that analyzed is doc.provenance.source.path.
// 0.2
for item in engine.analyze(Ingest::path("docs/")?) {
match item {
Ok(entry) => println!("{:?}: {}", entry.path, entry.analysis.total_words()),
Err(e) => eprintln!("{:?}: {}", e.path, e.error),
}
}
// 0.3
for item in engine.analyze_all(Inputs::dir("docs/")?) {
match item {
Ok(doc) => println!("{:?}: {}", doc.provenance.source.path, doc.metrics.total_words),
Err(e) => eprintln!("{e}"),
}
}
Corpus and CorpusResult are gone, with Corpus::total_words, passive_ratio and mean_readability. Collect the stream yourself. A count such as total_words sums across documents. A ratio such as vocabulary_ttr does not, because it falls as a document grows.
Read the measures
The aggregate methods are fields now, so they reach Python and the JSON too. The paragraph measures moved into Paragraph.metrics, and the document measures into Document.metrics.
// 0.2
doc.total_words();
doc.passive_ratio(); // 0.0 on a document with no sentences
doc.vocabulary_ttr;
paragraph.readability_grade;
sentence.word_count();
sentence.is_passive();
sentence.tree_depth(); // usize::MAX on a cycle
// 0.3
doc.metrics.total_words;
doc.metrics.passive_ratio; // None on a document with no sentences
doc.metrics.vocabulary_ttr;
paragraph.metrics.readability_grade;
sentence.word_count;
sentence.passive;
sentence.tree_depth; // None on a cycle
DocumentMetrics also carries total_sentences, paragraph_count, mean_sentence_length and sentence_length_std. In 0.2 these were methods, so they never reached Python or the JSON.
Read the record facts
The facts on a sentence are named for what the detector finds. ClausalComplement is a verb governing a clausal complement. Which of those verbs report speech is your lexicon, as before.
// 0.2
sentence.reportings;
sentence.reportings_in(&["say", "claim"]);
sentence.bare_assertion;
token.feats; // "Mood=Ind|Tense=Past"
// 0.3
sentence.clausal_complements;
sentence.clausal_complements_in(&["say", "claim"]);
sentence.root_mood; // Option<RootMood { root_id, mood }>
token.features; // BTreeMap: {"Mood": "Ind", "Tense": "Past"}
The 0.2 bare_assertion flag is root_mood.mood == "Ind" with no modal whose head_id is root_mood.root_id. Token::feat("Mood") reads one feature, as before. Paragraph.in_blockquote is Paragraph.kind: in_blockquote: true is ParagraphKind::Quote, and a list item is now ParagraphKind::ListItem.
Summaries, keyphrases and clusters
The outcomes are free functions at the crate root. They take the analysed &Document, not a slice of its sentences, and need no engine, so they run in a build without UDPipe. The method is optional and never in the name. Each returns a record whose provenance names the method that ran and its parameters.
// 0.2
let sentences: Vec<Sentence> = doc.sentences().cloned().collect();
let top = extraction::tfidf_summarize(&sentences, 3)?; // Vec<ScoredSentence>
let ranked = extraction::textrank_summarize(&sentences, 3)?;
let phrases = extraction::yake_keyphrases(&sentences, 10)?; // Vec<Keyphrase>
let groups = embed_and_cluster(&doc, &model, 0.85)?; // SemanticClusters
// 0.3
let top = matra::summarize(&doc, 3)?; // Summary
let ranked = matra::summarize_with(&doc, 3, SummaryMethod::TextRank)?;
let phrases = matra::keyphrases_with(&doc, 10, KeyphraseMethod::Yake)?; // Keyphrases
let groups = matra::clusters(&doc, &model, 0.85)?; // Clusters
What moved inside the results:
ScoredSentence { text, score, position }isSummarySentence { index, span, text, score }, inSummary.sentences.positionisindex.Keyphrasegainsoccurrences, every span the phrase was found in, and sits inKeyphrases.phrases.SemanticClusters { model_hash, threshold, clusters }isClusters { provenance, clusters, edges }. The embedder's identity isprovenance.parameters.embedder, anIdentitylike the parser's (the old hash is itssha256), and the thresholdprovenance.parameters.threshold. The edges moved out of each cluster into one list, ordered byaand thenb. EachClusterholds onlymembers.- The threshold and every score are
f64, notf32. Model2Vec::potion_base_8m(dir)takes the models directory, the oneMATRA_MODEL_DIRnames, and keeps the three files indir/potion-base-8M/. In 0.2dirwas the model's own directory. Pass the parent:potion_base_8m(cfg.model_dir())where 0.2 tookpotion_base_8m(cfg.model_dir().join("potion-base-8M")). Python'sModel2Vec.potion_base_8m(dir)changed the same way.
Errors
Error keeps its variants and kind(). Three shapes changed.
// 0.2
Error::UnsupportedFormat(format)
Error::InvalidInput(message)
Error::ParseFailed(message)
domain::MAX_INPUT_BYTES
// 0.3
Error::UnsupportedFormat { format, feature } // both Option
Error::InvalidInput { reason, message } // reason: "threshold_not_finite", ...
Error::ParseFailed { reason, cause } // reason: None, or "head_out_of_range", ...
domain::limits::MAX_INPUT_BYTES
InvalidInput's reason names the check that failed, a stable name the errors reference lists, and details["reason"] carries it, where 0.2 put the message text there. On model_invalid, parse_failed and io the error's own text moves from details["reason"] to details["cause"], so reason names a check wherever it appears. Four InputTooLarge labels name what you read rather than matra's modules: file_source is file_read, config_file is config_file_read, udpipe_download is parser_model_download and embedding_download is embedding_model_download. A match on the old strings no longer fires.
Error::details() is new. It returns every value in the message by name, so no caller parses a message. Error and DocumentError serialize as {kind, message, details}, and DocumentError adds path. A failed read now names its path: io error: cannot read essay.md: No such file or directory (os error 2). A file whose bytes are not UTF-8 is UnsupportedFormat, not Io. Every cap is in domain::limits, with the same values as 0.2. Errors lists every kind and its details.
What left the public surface
These modules and items are private to the crate in 0.3: the Udpipe adapter, metrics with Metric, default_suite, run_suite and readability::flesch_kincaid_grade, hearst, extraction, the readers MarkdownDecomposer and PlainTextDecomposer with the Decomposers table, FileSource and DirectorySource, and standard_decomposers. The port modules nlp, decompose and source stay public with their traits, reshaped as above. The measures stay public as fields of the Document. For another measure, write a function over &Document. Embedder stays public and keeps its shape. Its identity returns a domain::Identity, the shape the parser reports, where it returned &str: Identity::new("my-service", Some("v3".into()), None, None) in place of "my-service/v3". Return ModelInvalid, Io or InputTooLarge for your own failures, as the trait documents.
Configuration
Config reads the renamed sections, so its accessors follow them.
// 0.2
cfg.summarize_n();
cfg.summarize_algorithm();
cfg.keyphrases_algorithm();
cfg.semantic_threshold(); // f32
cfg.udpipe_model();
// 0.3
cfg.summary_n();
cfg.summary_method();
cfg.keyphrases_method();
cfg.clusters_threshold(); // f64
Config::from_file(path) is new: the configuration with a file you name, where a missing file is an error.
Python
Build the engine
Matra is Engine. A model directory is named by MATRA_MODEL_DIR, or by a config file you pass to Engine.from_config. Matra.from_path is gone: no surface loads a model file of your choosing.
# 0.2
from matra import Matra
m = Matra.english()
m = Matra.english("/opt/matra/models")
# 0.3
import matra
engine = matra.Engine() # MATRA_MODEL_DIR, else the default
engine = matra.Engine.from_config("matra.toml") # a config file you name
engine = matra.Engine.from_config() # the config file the environment names
Analyze
engine.analyze takes exactly one input. A str alone is text, never a path, and reads as plain text unless you name a format. It returns a matra.Document, not a dict.
# 0.2
doc = m.analyze(text)
doc = m.analyze_markdown(text)
items = m.analyze_path("essay.md") # a list of one
items = m.analyze_path("docs/") # a list, built in full
# 0.3
doc = engine.analyze(text)
doc = engine.analyze(text, format="markdown")
doc = engine.analyze(file="essay.md") # read as its extension and bytes say
for item in engine.analyze_all(dir="docs/"): # lazy, one item per file
...
analyze_path returned CorpusEntry dicts ({"path", "analysis"}) and DocumentError dicts ({"path", "error": {"kind", "message"}}). analyze_all yields a matra.Document or a DocumentError dict, {"path", "kind", "message", "details"}. Tell them apart with isinstance(item, matra.Document). The path of a document is item["provenance"]["source"]["path"].
# 0.2
for item in m.analyze_path("docs/"):
if "error" in item:
print(item["path"], item["error"]["kind"])
else:
print(item["path"], item["analysis"]["passive_ratio"])
# 0.3
for item in engine.analyze_all(dir="docs/"):
if isinstance(item, matra.Document):
print(item["provenance"]["source"]["path"], item["metrics"]["passive_ratio"])
else:
print(item["path"], item["kind"])
Read a document
A matra.Document reads like the dict it replaced: doc["metrics"]["total_words"], list(doc) and len(doc) work. It is read-only. Call doc.to_dict() where your code needs a dict, changes the result, or pickles it. The keys follow the schema 2.0 record, so the measures moved:
# 0.2
doc["passive_ratio"]
doc["sections"][0]["paragraphs"][0]["readability_grade"]
doc["sections"][0]["paragraphs"][0]["in_blockquote"]
# 0.3
doc["metrics"]["passive_ratio"]
doc["sections"][0]["paragraphs"][0]["metrics"]["readability_grade"]
doc["sections"][0]["paragraphs"][0]["kind"] == "quote"
Summaries, keyphrases and clusters
The four algorithm-named methods and semantic_clusters are module functions over a Document. A text is parsed once however many outcomes you ask of it, and in the format you analysed it in: 0.2's methods read every text as plain text, so a README's headings could land in its summary. Analyse a markdown file with file=, or a markdown string with format="markdown", and its headings stay headings. max_phrases is n. Each returns a dict whose provenance names the method and its parameters.
# 0.2
top = m.tfidf_summarize(text, 3) # [{"text", "score", "position"}]
ranked = m.textrank_summarize(text, 3)
phrases = m.yake_keyphrases(text, max_phrases=10) # [{"phrase", "score"}]
groups = m.semantic_clusters(text, 0.85, model)
# 0.3
doc = engine.analyze(text)
top = matra.summarize(doc, 3) # {"provenance", "sentences": [{"index", "span", "text", "score"}]}
ranked = matra.summarize(doc, 3, method="textrank")
phrases = matra.keyphrases(doc, 10, method="yake") # {"provenance", "phrases": [{"phrase", "score", "occurrences"}]}
groups = matra.clusters(doc, model, 0.85)
The vectors-in matra.semantic_clusters(vectors, threshold, model_hash) is gone, because Rust never had it. To compare vectors you already hold, compute the cosine yourself, as the semantic clusters guide shows.
Exceptions
Every exception is a matra.MatraError subclass with kind and details. Each also derives from the builtin class 0.2 raised, so except ValueError and except OSError catch what they caught.
# 0.2
try:
m.analyze(text)
except ValueError as e: # which ValueError: read the message
...
# 0.3
try:
engine.analyze(text)
except matra.InputTooLargeError as e: # also a ValueError
print(e.kind, e.details["limit"], e.details["actual"])
A missing file was a bare OSError in 0.2. It is now matra.IoNotFoundError, which is a FileNotFoundError and an OSError, so except FileNotFoundError now catches it. A permission failure is matra.IoPermissionError, a PermissionError.
Types
matra.types follows the record. ScoredSentence is SummarySentence inside Summary; SemanticClusters, SemanticCluster and SemanticEdge are Clusters, Cluster and SimilarityEdge; Reporting is ClausalComplement; CorpusEntry and CorpusItem are gone. The TypedDict that was named Document is DocumentDict, the shape to_dict() returns, and matra.Document is the class.
Command line
The config file
A config file from 0.2 does not load. Every command that reads one fails with invalid_input, naming the file and each key's replacement. Edit the file, or write a fresh one with matra config init --force.
# 0.2
[models]
udpipe = "english-ewt-ud-2.5-191206"
embedding = "potion-base-8M"
[semantic]
threshold = 0.85
[summarize]
n = 3
algorithm = "tfidf"
[keyphrases]
n = 10
algorithm = "rake"
# 0.3
[models]
embedding = "potion-base-8M"
[summary]
n = 3
method = "tfidf"
[keyphrases]
n = 10
method = "rake"
[clusters]
threshold = 0.85
models.udpipe goes without a replacement, because the parsing model is pinned in the source beside its digest.
The JSON envelope
format_version is 2, and the envelope gains a fifth key, error. All five are always present, and exactly one of result and error is null. A consumer pinned to format_version: 1 should fail, and then move.
{ "format_version": 1, "command": "analyze", "input": "essay.md", "result": {} }
{ "format_version": 2, "command": "analyze", "input": "essay.md", "result": {}, "error": null }
result changed too:
analyze: a schema 2.0Document. See The JSON record below.summarize: aSummaryrecord,{provenance, sentences}, not a bare list of{text, score, position}.keyphrases: aKeyphrasesrecord,{provenance, phrases}, not a bare list of{phrase, score}.
# 0.2
matra summarize essay.md --json | jq '.result[0].text'
matra keyphrases essay.md --json | jq '.result[].phrase'
# 0.3
matra summarize essay.md --json | jq '.result.sentences[0].text'
matra keyphrases essay.md --json | jq '.result.phrases[].phrase'
Failures under --json
In 0.2 a failure under --json wrote nothing to stdout. In 0.3 it writes the envelope there, with result null and error holding the kind, message and details. It still writes matra: <message> to stderr and still exits 2, so a script that reads only the exit code is unchanged. A usage error under --json is an envelope too.
# 0.3
matra analyze missing.md --json | jq -r '.error.kind' # io
The message for a missing file changed with it: io error: cannot read missing.md: ... where 0.2 printed no such file: missing.md. Branch on error.kind and read error.details, never the message.
Standard input
Standard input is read as bytes, and its format is detected as a file's is. --stdin-filename still names it, and its extension still picks the format. Bytes that are not UTF-8 fail as unsupported_format, as a file's do, where 0.2 failed with a read error that had no kind.
What is new
These do not break a 0.2 command, and are listed so you know they exist: matra analyze DIR, one result per file; matra clusters; --format on every document command; --depth on analyze --json; and matra --schema. The CLI guide has each.
The JSON record
The same record is the Rust Document, the Python matra.Document and analyze --json's result. It now names its own schema version in provenance.schema, "2.0". The schema is published: matra --schema document prints it.
| 0.2 | 0.3 |
|---|---|
| no provenance | provenance: the matra version, the schema, the parser and its model's SHA-256, the format, and the input's path, and the size and SHA-256 of the analysed text |
vocabulary_ttr, nominalization_ratio, passive_ratio | metrics.vocabulary_ttr, metrics.nominalization_ratio, metrics.passive_ratio |
| absent | metrics.total_words, total_sentences, paragraph_count, mean_sentence_length, sentence_length_std |
| absent | skipped: each construct the reader saw and did not analyse, with its kind and span |
sections[].heading, level, paragraphs | the same, and span (the heading's, null with no heading) |
paragraph.readability_grade, lexical_density, compression_ratio | paragraph.metrics.readability_grade, and so on |
paragraph.in_blockquote | paragraph.kind: paragraph, quote or list_item |
| absent | span on every paragraph, sentence and token, in UTF-8 bytes of the analysed text |
| absent | sentence.word_count, tree_depth (null on a cycle), passive |
sentence.reportings | sentence.clausal_complements |
sentence.bare_assertion | sentence.root_mood: {root_id, mood}, or null |
token.feats: one string of pipe-separated features | token.features: {"Mood": "Ind", "Tense": "Past"} |
Read the record as the schema asks: ignore a field you do not recognize, treat every string set you receive as open, and reject a major you do not know.
Results that change with no rename
A few results differ for the same input, and nothing about the call tells you so.
- Markdown is read by a CommonMark parser. 0.2 dropped tables without a trace, ended the document at a
## Referencesline, read#hashtagas a heading, and passed inline markup such as**and link syntax to the parser as words. In 0.3 tables, code, front matter and HTML areskippedentries, a## Referencesheading is a heading like any other, a heading needs a space after its#, list items are paragraphs of kindlist_item, and inline markup is removed from the text the parser reads. The counts and measures of a markdown document can change. - Plain text splits paragraphs at a line of only whitespace, so
\r\nendings and a line of spaces separate paragraphs as an empty line does. - Keyphrases with equal scores come in the same order on every run, by first occurrence, where 0.2 followed a hash map's order.
- TF-IDF scores are the same to the last bit on every run.
- A binary file is
unsupported_format, naming the format when it is a PDF or DOCX. In 0.2 it was an I/O error. - The size cap's message reads
input too large, where 0.2 readinput input too large. Model2Vec.potion_base_8m(dir)looks indir/potion-base-8M/, where 0.2 looked indiritself. A 0.2 call that passed the model's own directory still runs, and downloads a second copy one level down. Pass the models directory instead.Model2Vec.potion_base_8m()with no argument reads no config file. It finds the model directory from the environment, asEngine()does, so amodel_dirormodels.embeddingset only in the config file no longer moves it.Model2Vec.from_config()reads the file.- Python's constructors announce a download.
Engine(),Engine.from_config()andModel2Vec.potion_base_8m()write two lines tosys.stderrbefore they download a model, the ones the command line prints. Once the model is on disk they write nothing. - An empty result says so.
matra summarize,keyphrasesandclustersstill exit 1 when they find nothing, and now also write one line to stderr saying so, unless--jsonor--quietis given.
This page, measured by matra 0.3.0 with english-ewt-ud-2.5-191206: 43 paragraphs, in the margin. the data