N-grams

N-grams are overlapping sequences of content words in a text. They provide an efficient way to identify passages shared by two texts. By locating sequences of content words that occur in both texts, the TLG can identify possible intertextual similarities.

The comparison highlights the words shared by the two passages. Small differences may occur between highlighted portions of a match; these may represent interpolations, deletions, or other variation between the passages.

Compare two texts

TLG Compare Texts N-grams page with Homer and Plato selected and the Wordform and Lemma menu open
Figure 1. Compare Texts N-grams selection page

The N-grams page compares two selected texts side by side.

  1. Enter or select the first text in the box on the left.
  2. Enter or select the second text in the box on the right.
  3. Choose Wordform or Lemma from the menu between the two texts.
  4. Click Go to begin the comparison.
Note: A comparison can take some time to generate. While the search is running, the results page displays its progress.

Reading the results

TLG N-gram comparison results for Homer's Iliad and Plato's Cratylus
Figure 2. N-gram comparison results

Matching passages are displayed in paired rows, with the first text on the left and the second text on the right. Words participating in the match are highlighted, making the shared sequence easy to identify. Each result includes a citation for the passage.

The icons beside a result provide access to the passage and related browsing functions. Use the navigation controls at the upper right of the results area to move through additional matches.

Extended passages in common may appear as several consecutive matches rather than as a single result.

Wordform and lemma comparisons

Wordform

Compares the forms that actually occur in the two texts and is therefore useful for finding close verbal correspondences.

Lemma

Compares words at the lemma level. This allows the comparison to recognize related inflected forms even when the exact wordforms differ.

N-gram matching can also tolerate limited differences in word order, because the order of content words within an n-gram is not always significant.

General rules

Content words. N-grams are restricted to content words. Common stop words that contribute little meaning are ignored so that they do not obscure more significant similarities.

Trigrams and bigrams. The comparison normally relies on sequences of content words, especially trigrams. For very short works, shorter sequences may be necessary in order to detect meaningful correspondences.

Length of matches. Shorter matching requirements produce more results, including more incidental similarities, and can take longer to calculate. Very short texts may also require special treatment because longer n-gram sequences may not occur often enough to support comparison.

N-grams elsewhere in the TLG

N-gram analysis is also available while browsing texts. In Browse Texts → One Text, N-grams can be used to find other occurrences of a phrase in the corpus. In Parallel Browsing, the Highlight similarities option uses related matching techniques to identify similarities between the two displayed passages.

Related feature: Compare Editions in Parallel Browsing is designed for finer comparison of editions. It uses differences in individual wordforms, punctuation, and related textual features, while using n-grams to help align the two texts.

Quick reference

Left text: first work to compare.   Right text: second work to compare.   Wordform: emphasizes matching forms as they occur in the texts.   Lemma: allows differences of inflection.   Go: starts the comparison.   Highlighted text: words participating in a detected match.

N-gram results identify verbal similarities for further examination; users should evaluate the significance and context of individual matches.