What the clustering threshold does
matra can group the sentences of a document that say the same thing in different words. It embeds each sentence as a vector, scores every pair by cosine similarity, and links each pair whose score clears a threshold you choose. The clusters are the connected groups those links make. Cluster sentences by meaning has the calls.
Everything else matra returns is structure read off the text, checkable against its bytes. Clusters are not: they depend on a model's representation of meaning. So they arrive as their own value from a separate call, never as a field on Document, and their provenance carries the identity of the model that produced them and the threshold you chose.
Three things the shape means
- Co-membership is transitive, not pairwise. Clusters are connected components, so sentence A and sentence C can share a cluster because both resemble B, without resembling each other. The edges travel in the result so you can see which pairs actually cleared the bar. A missing edge is no claim, not a low score.
- Singletons are always excluded. A sentence with no pair above the threshold appears in no cluster, so "not in any cluster" is a meaningful count.
- The threshold is yours, and it does not travel. Published cutoffs for paraphrase detection span 0.67 to 0.9 with no consensus; the working value depends on the model, the domain and the text length. A cutoff calibrated on sentences is not the cutoff for whole documents, because a document vector is the mean over far more tokens.
Watch the threshold move
Ten short sentences, with the reference model. Before you move the threshold, predict: sentence 8 is about a different decision by the same committee. As the threshold falls from 0.85, at which value do you expect it to join sentences 1, 2 and 3, and through which of them? Then drag the threshold and find out.
Semantic clusters as the threshold moves
- 1The committee approved the budget on Tuesday.
- 2On Tuesday the committee passed the budget.
- 3The budget was approved by the committee.
- 4Heavy rain closed the mountain road overnight.
- 5The mountain road was shut overnight by heavy rain.
- 6The new release fixes a crash when saving large files.
- 7Saving large files no longer crashes the program in this release.
- 8The committee approved a new park plan on Monday.
- 9Bread rises faster in a warm kitchen.
- 10The museum opens at nine on weekdays.
At 0.85, matra's default: 3 clusters, 3 of 10 sentences in none.
show the data
| Threshold | Clusters | Edges (cosine) |
|---|---|---|
| 0.50 | {1, 2, 3, 8} {4, 5} {6, 7} | 1-2 (0.8521), 1-3 (0.8812), 1-8 (0.7239), 2-3 (0.6923), 2-8 (0.5646), 3-8 (0.6405), 4-5 (0.9592), 6-7 (0.8590) |
| 0.55 | {1, 2, 3, 8} {4, 5} {6, 7} | 1-2 (0.8521), 1-3 (0.8812), 1-8 (0.7239), 2-3 (0.6923), 2-8 (0.5646), 3-8 (0.6405), 4-5 (0.9592), 6-7 (0.8590) |
| 0.60 | {1, 2, 3, 8} {4, 5} {6, 7} | 1-2 (0.8521), 1-3 (0.8812), 1-8 (0.7239), 2-3 (0.6923), 3-8 (0.6405), 4-5 (0.9592), 6-7 (0.8590) |
| 0.65 | {1, 2, 3, 8} {4, 5} {6, 7} | 1-2 (0.8521), 1-3 (0.8812), 1-8 (0.7239), 2-3 (0.6923), 4-5 (0.9592), 6-7 (0.8590) |
| 0.70 | {1, 2, 3, 8} {4, 5} {6, 7} | 1-2 (0.8521), 1-3 (0.8812), 1-8 (0.7239), 4-5 (0.9592), 6-7 (0.8590) |
| 0.75 | {1, 2, 3} {4, 5} {6, 7} | 1-2 (0.8521), 1-3 (0.8812), 4-5 (0.9592), 6-7 (0.8590) |
| 0.80 | {1, 2, 3} {4, 5} {6, 7} | 1-2 (0.8521), 1-3 (0.8812), 4-5 (0.9592), 6-7 (0.8590) |
| 0.85 | {1, 2, 3} {4, 5} {6, 7} | 1-2 (0.8521), 1-3 (0.8812), 4-5 (0.9592), 6-7 (0.8590) |
| 0.90 | {4, 5} | 4-5 (0.9592) |
| 0.95 | {4, 5} | 4-5 (0.9592) |
- in a cluster, named by its letter
- in no cluster
- what you point at or tap, with its cluster
Each arc is a pair of sentences whose cosine similarity cleared the threshold, labelled with the score. A cluster is a connected group, so two sentences can share one through a third without their own pair clearing the bar. The embeddings are fixed; only the threshold moves, over the values matra was run at, and between two of them the figure blends the two runs rather than compute one of its own. matra ships 0.85 as a starting point, not a verdict: which threshold is right depends on the text and the question.
Input: Three things said two or three ways each, and three said once, matra contributors. MIT, as part of the matra repository.
Produced by matra 0.3.0 with the UDPipe model english-ewt-ud-2.5-191206
(SHA-256 784bd0fa85e3) and
the embedding model potion-base-8M (SHA-256 81c359215087). Data: paraphrases/clusters.json.
What sentence 8 did
It joins at 0.70, through sentence 1 alone: their pair scores 0.7239 and clears the bar, while its pairs with sentences 2 and 3 do not, yet it shares their cluster. A cluster can grow through a chain of pairs, on shared words ("committee approved") rather than shared meaning, and the clusters dissolve as the bar rises past 0.85.
Why a static model
The reference model is a static embedding model, a lookup table rather than a transformer: inference is a row gather, a mean, and a normalize. That costs roughly ten percent of a small transformer's benchmark quality, and buys bit-identical vectors on every platform and in every language binding, which is what lets the conformance suite pin exact vectors rather than tolerances.
This page, measured by matra 0.3.0 with english-ewt-ud-2.5-191206: 4 paragraphs, in the margin. the data