Examining dictionary unknowns

Measure dictionary flags and the errors without a flag

word processing
spell checking
workforce research
Learn how hunspell puts a mark on unknown workforce words and how to examine constructed spelling errors.

A vocabulary list is almost ready for checks. The team asks whether a spell checker can find obvious typing errors before human checks.

Proper names and real-word typing errors complicate the task. A dictionary finds strings absent from its entries. It cannot establish the writer’s intended meaning.

Spell checking here means asking hunspell for words that are absent from its dictionary. The result is a list of unknown words, not a list of all errors.

Note

The author created the fictional Riverton Workforce Lab, its job board, and its training flyer for teaching. The deliberate misspellings are constructed examples from Riverton words.

TipWhat you will learn

By the end of this lesson, you will be able to:

  • run hunspell() on the Riverton vocabulary.
  • count word types that the dictionary puts a mark on.
  • build a small constructed spelling test.
  • explain why known dictionary misses do not measure general error detection.
  • explain why a dictionary does not put a mark on real-word errors.

Examine the real vocabulary

The readr package imports text, and dplyr and tibble create the word list. The purrr package applies dictionary calls to each item. The tidytext package creates tokens, and hunspell supplies dictionary flags. The code uses the stemming lesson’s tokenization rule to build the same lowercased 96-word vocabulary.

library(readr)
library(dplyr)
library(tibble)
library(purrr)
library(tidytext)
library(hunspell)

sentences <- read_csv(
  "data/workforce/workforce_sentences.csv",
  na = character(),
  col_types = cols(
    sentence_id = col_character(),
    document_id = col_character(),
    source_line = col_character(),
    text = col_character(),
    reference_label = col_character(),
    uncertainty = col_character(),
    annotator_id = col_character(),
    rationale = col_character(),
    codebook_version = col_character(),
    codebook_hash = col_character(),
    derived = col_character(),
    transformation = col_character()
  )
)

vocabulary <- sentences |>
  select(sentence_id, text) |>
  unnest_tokens(word, text) |>
  distinct(word) |>
  arrange(word) |>
  pull(word)

known_to_hunspell <- \(words) map_lgl(hunspell(words), \(result) length(result) == 0L)

flagged_types <- vocabulary[!known_to_hunspell(vocabulary)] |>
  sort()

flagged_table <- tibble(flagged_word_type = flagged_types)
Riverton word types that hunspell puts a mark on
Flagged word type
october
riverton

Hunspell puts a mark on 2 of the 96 source word types, october and riverton. These are a lowercased month name and a fictional place name. The flags indicate dictionary unknowns, not proven spelling errors.

Build a constructed error test

The next table uses eight deliberate misspellings of real Riverton word types. This is a constructed test, not a claim about how often people make these errors.

constructed_errors <- tribble(
  ~correct_word, ~misspelled_word,
  "training", "traning",
  "provided", "provied",
  "experience", "experiance",
  "required", "requiered",
  "workers", "wrokers",
  "schedules", "scheduels",
  "communication", "communciation",
  "certificate", "certficate"
)

suggestions <- hunspell_suggest(constructed_errors$misspelled_word)

constructed_results <- constructed_errors |>
  mutate(
    flagged = !known_to_hunspell(misspelled_word),
    correct_word_wrongly_flagged = !known_to_hunspell(correct_word),
    suggestion_count = map_int(suggestions, length),
    suggestions = map_chr(suggestions, \(result) paste(result, collapse = ", "))
  )

spell_summary <- tibble(
  real_errors_caught = sum(constructed_results$flagged),
  constructed_errors = nrow(constructed_results),
  correct_words_wrongly_flagged = sum(constructed_results$correct_word_wrongly_flagged),
  correct_words_checked = nrow(constructed_results),
  caught_fraction = paste0(real_errors_caught, "/", constructed_errors),
  false_flag_fraction = paste0(correct_words_wrongly_flagged, "/", correct_words_checked)
)
Constructed spelling errors and hunspell suggestions
Correct Riverton word Constructed misspelling Flagged Correct word flagged Suggestions returned Suggestions
training traning TRUE FALSE 7 training, ranting, tracing, craning, trading, transiting, tanning
provided provied TRUE FALSE 5 provide, proved, provided, pro vied, pro-vied
experience experiance TRUE FALSE 2 experience, experiencing
required requiered TRUE FALSE 5 required, requited, requiter, requiem, require
workers wrokers TRUE FALSE 2 workers, brokers
schedules scheduels TRUE FALSE 3 schedules, scheduled, schedule
communication communciation TRUE FALSE 5 communication, commutation, annunciation, conciliation, enunciation
certificate certficate TRUE FALSE 2 certificate, artificer
Counts from the constructed spelling test
Errors flagged Constructed errors Correct words wrongly flagged Correct words examined Flagged fraction False flag fraction
8 8 0 8 8/8 0/8

Hunspell puts a mark on all 8 constructed misspellings. Each is a non-word absent from this dictionary. It incorrectly puts a mark on 0 of the 8 source words. Those source words exclude the two known dictionary unknowns. The selected examples show expected dictionary behavior, not general spelling quality. The next example addresses an excluded error type.

Show the real-word limit

A dictionary will not put a mark on a typo if the typo is also a real word. form and from are both dictionary words.

real_word_error <- tibble(
  intended_word = "from",
  typed_word = "form",
  intended_flagged = !known_to_hunspell(intended_word),
  typed_flagged = !known_to_hunspell(typed_word)
)
A real-word typo absent from the dictionary flags
Intended word Typed word Intended word flagged Typed word flagged
from form FALSE FALSE

If a writer meant from and typed form, this dictionary reports no problem. Both strings are dictionary entries. Detecting the error requires context.

What to remember

  • hunspell() returns dictionary unknowns.
  • Unknown does not always mean misspelled.
  • Individual checks of the 96 word types put a mark on october and riverton.
  • The constructed 8/8 check is a demonstration because it contains only non-word errors.
  • Real-word errors need context as well as a dictionary.

Use hunspell to create a queue for checks. Read each word with a flag in context. A dictionary separates known and unknown strings. People must still decide whether the sentence is correct.

Sources