Small grammar checks

Score three checks and keep the limits visible

sentences and paragraphs
grammar checking
workforce research
Learn how small grammar checks can find inserted errors and why they are not a full grammar checker.

The coordinator cannot approve a public summary from spelling alone. Correctly spelled words can still form ungrammatical sentences.

A spelling check compares words with accepted spellings. Grammar concerns how words combine in sentences. This lesson builds three small checks for sentences with deliberately inserted errors.

This project does not install a grammar checker. The three custom checks demonstrate selected rules rather than provide a complete checker. An offline check uses local files without sending text to a web service.

Note

The Riverton sentences and marked grammar examples are invented for this lesson.

TipWhat you will learn

By the end of this lesson, you will be able to:

  • explain the difference between spelling and grammar checks.
  • use UDPipe dependency output to compare a subject and a verb.
  • find doubled words with a stringr pattern.
  • compare a simple a and an rule with expected answers.
  • explain why a demonstration check is not a working grammar checker.

Make a marked test set

readr reads the Riverton CSV. tokenizers and purrr prepare words, stringr does two pattern checks, and a local udpipe parser processes sentences. dplyr and tibble organize the results. A dependency parser gives labels to grammatical relationships, such as the relation between a verb and its subject.

library(readr)
library(dplyr)
library(tibble)
library(purrr)
library(stringr)
library(tokenizers)
library(udpipe)

sentences <- read_csv(
  "data/workforce/workforce_sentences.csv",
  na = character(),
  col_types = cols(
    sentence_id = col_character(),
    document_id = col_character(),
    source_line = col_character(),
    text = col_character(),
    reference_label = col_character(),
    uncertainty = col_character(),
    annotator_id = col_character(),
    rationale = col_character(),
    codebook_version = col_character(),
    codebook_hash = col_character(),
    derived = col_character(),
    transformation = col_character()
  )
)

marked_sentences <- tibble(
  example_id = sprintf("g%02d", 1:14),
  text = c(
    "The worker checks the list.",
    "The worker check the list.",
    "The applicants submit forms.",
    "The applicants submits forms.",
    "Bring a badge to orientation.",
    "Bring a envelope to orientation.",
    "Apply for an evening class.",
    "Apply for a evening class.",
    "The session begins in an hour.",
    "A university partner sends trainers.",
    "The coordinator checks the the form.",
    "The team needs needs support.",
    "The employee uses a spreadsheet.",
    "The employee use a spreadsheet."
  ),
  planted_check = c(
    "none",
    "subject-verb number",
    "none",
    "subject-verb number",
    "none",
    "article letters",
    "none",
    "article letters",
    "none",
    "none",
    "doubled word",
    "doubled word",
    "none",
    "subject-verb number"
  )
) |>
  mutate(has_planted_error = planted_check != "none")
Author-marked sentences before the checks
Example ID Sentence Planted check Has planted error
g01 The worker checks the list. none FALSE
g02 The worker check the list. subject-verb number TRUE
g03 The applicants submit forms. none FALSE
g04 The applicants submits forms. subject-verb number TRUE
g05 Bring a badge to orientation. none FALSE
g06 Bring a envelope to orientation. article letters TRUE
g07 Apply for an evening class. none FALSE
g08 Apply for a evening class. article letters TRUE
g09 The session begins in an hour. none FALSE
g10 A university partner sends trainers. none FALSE
g11 The coordinator checks the the form. doubled word TRUE
g12 The team needs needs support. doubled word TRUE
g13 The employee uses a spreadsheet. none FALSE
g14 The employee use a spreadsheet. subject-verb number TRUE

The author marks seven of the 14 test sentences as correct. The other seven each contain one inserted error. Success on these examples does not establish general grammar-checking performance.

Do the three checks

The subject-verb check uses the UDPipe parser’s feats column, which can contain values such as Number=Sing or Number=Plur. It searches for an nsubj relation, short for nominal subject, pointing to a verb. In the code, \(tokens) is R shorthand for a small function applied to one sentence’s tokens.

extract_number <- function(feats) {
  str_match(coalesce(feats, ""), "Number=([^|]+)")[, 2]
}

find_subject_verb_flags <- function(annotated) {
  annotated_with_number <- annotated |>
    mutate(number = extract_number(feats))

  subjects <- annotated_with_number |>
    filter(dep_rel == "nsubj") |>
    transmute(
      doc_id,
      head_token_id,
      subject_token = token,
      subject_number = number
    )

  verbs <- annotated_with_number |>
    filter(upos %in% c("VERB", "AUX"), !is.na(number)) |>
    transmute(
      doc_id,
      verb_token_id = token_id,
      verb_token = token,
      verb_number = number
    )

  verbs |>
    inner_join(subjects, by = c("doc_id", "verb_token_id" = "head_token_id")) |>
    mutate(flag = !is.na(subject_number) & subject_number != verb_number)
}

score_check <- function(flag, expected) {
  tibble(
    true_positive = sum(flag & expected),
    false_positive = sum(flag & !expected),
    false_negative = sum(!flag & expected),
    true_negative = sum(!flag & !expected)
  )
}

parser <- udpipe_load_model("data/treebank/en_ewt-500-parser.udpipe")

marked_vertical <- marked_sentences$text |>
  tokenize_words(lowercase = FALSE, strip_punct = FALSE) |>
  map_chr(\(tokens) paste(tokens, collapse = "\n"))

marked_annotated <- udpipe_annotate(
  parser,
  x = marked_vertical,
  doc_id = marked_sentences$example_id,
  tokenizer = "vertical",
  tagger = "default",
  parser = "default"
) |>
  as.data.frame() |>
  as_tibble()

agreement_pairs <- find_subject_verb_flags(marked_annotated)
agreement_flags <- agreement_pairs |>
  summarise(agreement_flag = any(flag), .by = doc_id)

doubled_pattern <- regex("\\b([[:alpha:]]+)\\s+\\1\\b", ignore_case = TRUE)
article_pattern <- regex(
  "\\b(a)\\s+[aeiou][[:alpha:]]*|\\b(an)\\s+[^aeiou\\W][[:alpha:]]*",
  ignore_case = TRUE
)

scored_examples <- marked_sentences |>
  left_join(agreement_flags, by = c("example_id" = "doc_id")) |>
  mutate(
    agreement_flag = coalesce(agreement_flag, FALSE),
    doubled_flag = str_detect(text, doubled_pattern),
    article_flag = str_detect(text, article_pattern)
  )

check_scores <- bind_rows(
  score_check(
    scored_examples$agreement_flag,
    scored_examples$planted_check == "subject-verb number"
  ) |>
    mutate(check = "subject-verb number"),
  score_check(
    scored_examples$doubled_flag,
    scored_examples$planted_check == "doubled word"
  ) |>
    mutate(check = "doubled word"),
  score_check(
    scored_examples$article_flag,
    scored_examples$planted_check == "article letters"
  ) |>
    mutate(check = "article letters")
) |>
  select(check, everything())

expected_scores <- tibble(
  check = c("subject-verb number", "doubled word", "article letters"),
  true_positive = c(3L, 2L, 2L),
  false_positive = c(0L, 0L, 2L),
  false_negative = c(0L, 0L, 0L),
  true_negative = c(11L, 12L, 10L)
)
Subject-verb pairs found by the dependency parser
Example ID Subject Subject number Verb Verb number Flagged
g01 worker Sing checks Sing FALSE
g02 worker Sing check Plur TRUE
g03 applicants Plur submit Plur FALSE
g04 applicants Plur submits Sing TRUE
g09 session Sing begins Sing FALSE
g10 partner Sing sends Sing FALSE
g11 coordinator Sing checks Sing FALSE
g12 team Sing needs Sing FALSE
g13 employee Sing uses Sing FALSE
g14 employee Sing use Plur TRUE
Scores for the three grammar checks on marked examples
Check True positives False positives False negatives True negatives
subject-verb number 3 0 0 11
doubled word 2 0 0 12
article letters 2 2 0 10

The checks and expected answers share an author, so the score is a teaching check rather than an accuracy estimate. A true positive is an inserted error that a check puts a mark on. A false positive is a correct sentence that a check puts a mark on. A false negative is an inserted error that a check misses. A true negative is a correct sentence with no mark from the check.

The subject-verb number check finds all three inserted number errors in this small set. The doubled-word check finds both repeated-word examples. The article check finds the two inserted a mistakes but also puts a mark on the correct phrases an hour and A university.

That rule uses the first letter, while English article choice depends on sound. Hour begins with a vowel sound because its h is silent. University begins with a consonant y sound. The rule uses letters without pronunciation information.

A complete grammar checker uses thousands of rules, a trained model, or both. These three checks demonstrate selected rules. The project trains its deliberately weak UDPipe model on 500 sentences. Model errors can affect every subject-verb result. The article rule deliberately ignores pronunciation so the lesson can demonstrate the resulting errors.

Try the Riverton sentences

The code applies the same three checks to the 28 Riverton sentences. The output selects sentences for a person to examine. It does not prove that sentences without a mark are correct.

riverton_vertical <- sentences$text |>
  tokenize_words(lowercase = FALSE, strip_punct = FALSE) |>
  map_chr(\(tokens) paste(tokens, collapse = "\n"))

riverton_annotated <- udpipe_annotate(
  parser,
  x = riverton_vertical,
  doc_id = sentences$sentence_id,
  tokenizer = "vertical",
  tagger = "default",
  parser = "default"
) |>
  as.data.frame() |>
  as_tibble()

riverton_agreement_pairs <- find_subject_verb_flags(riverton_annotated)
riverton_agreement_ids <- riverton_agreement_pairs |>
  filter(flag) |>
  pull(doc_id)
riverton_pair_sentence_count <- n_distinct(riverton_agreement_pairs$doc_id)

riverton_flags <- sentences |>
  transmute(
    sentence_id,
    text,
    agreement_flag = sentence_id %in% riverton_agreement_ids,
    doubled_flag = str_detect(text, doubled_pattern),
    article_flag = str_detect(text, article_pattern)
  ) |>
  mutate(any_flag = agreement_flag | doubled_flag | article_flag)

riverton_flag_summary <- tibble(
  check = c("subject-verb number", "doubled word", "article letters", "any check"),
  flagged_sentences = c(
    sum(riverton_flags$agreement_flag),
    sum(riverton_flags$doubled_flag),
    sum(riverton_flags$article_flag),
    sum(riverton_flags$any_flag)
  )
)

riverton_coverage <- tibble(
  measure = c("sentences with subject-verb pair", "sentences without subject-verb pair"),
  sentences = c(
    riverton_pair_sentence_count,
    nrow(sentences) - riverton_pair_sentence_count
  )
)
Grammar-check flags in the 28 Riverton sentences
Check Flagged Riverton sentences
subject-verb number 0
doubled word 0
article letters 0
any check 0
Subject-verb coverage in the Riverton sentences
Coverage measure Sentences
sentences with subject-verb pair 4
sentences without subject-verb pair 24

No check puts a mark on a Riverton sentence. The subject-verb check finds only four subject-and-verb pairs in all 28 sentences. It requires a number feature on the verb, which English verbs rarely have. It cannot evaluate the other 24 sentences. Zero flags therefore mainly reflect limited coverage, not correctness.

Grammar checking in production

The three custom checks demonstrate the concept rather than provide a complete offline grammar checker. Established open-source tools such as LanguageTool provide broader functionality. Production systems distinguish these issues:

  • Detection vs. correction: Marking a possible error is safer than automatically rewriting text, particularly when the detector covers few error types.
  • Normative context and auditability: Grammar conventions depend on the intended style, such as formal business English or informal messages. Rule-based systems can explain each flag through a named rule. Deep learning models can suggest fluent corrections but often cannot explain the convention behind each suggestion.
  • Rule/model/error-type comparison: Rule-based engines can precisely detect local style and typing errors, such as doubled words or wrong articles. Neural models generally address semantic confusion in sentences whose grammatical structure appears valid.

What to remember

  • Spelling checks examine words. Grammar checks examine word relationships.
  • A small subject-verb check can use UDPipe’s feats and nsubj labels.
  • A doubled-word pattern is simple and useful for one common typo.
  • The article-letter rule falsely puts a mark on an hour and A university.
  • These three checks are demonstrations, not an offline grammar checker for R.
  • No Riverton flags here means low coverage, not clean grammar.

The three checks can select sentences for a person to examine. They cannot approve public text independently. The subject-verb check covers only four Riverton sentences.

Sources