Extracting text from an image

Run OCR. Compare the result with a known transcript.

source data loading
OCR
workforce research
Learn what optical character recognition can recover from a training flyer and how to measure its errors.

The saved job board supplied digital text. The Riverton Workforce Lab’s next fictional notice is a training flyer available only as an image. Its wording covers a stipend, a class schedule, prior experience, and an application date. Losing or reversing a phrase would change what a person understood about the program.

Optical character recognition, or OCR, gives an estimate of the text in an image. The team will compare OCR output with a known reference transcript. It will keep the image and extracted text. Text recovery precedes classification of the flyer’s claims.

Note

The flyer and program are fictional. The author generated the image for this lesson and wrote the transcript from the image’s source text.

TipWhat you will learn

By the end of this lesson, you will be able to:

  • explain the difference between digital text and text inside an image.
  • run the Tesseract OCR engine from R.
  • compare OCR output with a known reference transcript.
  • interpret character-level edit distance.
  • report character and word error rates for a damaged image.
  • describe conditions that increase OCR errors.

Examine the source image

A fictional Riverton workforce training flyer. A complete text transcript follows.
Figure 1: Fictional training flyer used for the OCR example

RIVERTON SKILLS OPEN HOUSE
DATA SUPPORT CERTIFICATE
Paid training stipend
Evening classes
No prior experience required
Apply by October 15

The adjacent text gives the complete flyer transcript. Readers need that transcript because OCR can make errors or fail to run in their browser. The image remains the source. OCR provides one interpretation of the image.

The reference transcript contains six lines.

library(readr)
library(dplyr)
library(tibble)
library(stringr)
library(digest)
library(png)
library(tesseract)

transcript_path <- str_c(
  "data/workforce/",
  "training-flyer-ground-truth.txt"
)
reference_lines <- read_lines(transcript_path)
transcript_md5 <- digest::digest(
  paste(reference_lines, collapse = "\n"),
  algo = "md5",
  serialize = FALSE
)
flyer_metadata <- read_csv(
  "data/workforce/training-flyer-metadata.csv",
  col_types = cols(
    artifact = col_character(),
    width_pixels = col_integer(),
    height_pixels = col_integer(),
    md5 = col_character(),
    source = col_character(),
    rendering_note = col_character()
  )
)
image_metadata <- flyer_metadata |>
  filter(artifact == "training-flyer.png")
transcript_metadata <- flyer_metadata |>
  filter(artifact == "training-flyer-ground-truth.txt")
recorded_ocr_metadata <- flyer_metadata |>
  filter(artifact == "training-flyer-ocr-5.3.2.txt")
degraded_ocr_metadata <- flyer_metadata |>
  filter(artifact == "training-flyer-degraded-ocr-5.3.2.txt")
image_path <- "data/workforce/training-flyer.png"
image_array <- png::readPNG(
  image_path,
  info = TRUE
)
image_dimensions <- dim(image_array)[1:2]
image_md5 <- digest::digest(
  image_path,
  algo = "md5",
  serialize = FALSE,
  file = TRUE
)

reference_lines
[1] "RIVERTON SKILLS OPEN HOUSE"   "DATA SUPPORT CERTIFICATE"    
[3] "Paid training stipend"        "Evening classes"             
[5] "No prior experience required" "Apply by October 15"

The transcript is the comparison source for this controlled example. In real research, qualified readers must examine the reference. Those readers need to understand the document’s language and layout.

Run OCR

The tesseract package connects R to the open-source Tesseract OCR engine. We select English because the flyer is written in English.

ocr_information <- tesseract::tesseract_info()
ocr_engine <- tesseract::tesseract("eng")
diagnostic_ocr_text <- tesseract::ocr(
  "data/workforce/training-flyer.png",
  engine = ocr_engine
)
diagnostic_ocr_lines <- diagnostic_ocr_text |>
  str_split_1(fixed("\n")) |>
  str_trim()
diagnostic_ocr_lines <- diagnostic_ocr_lines[
  str_length(diagnostic_ocr_lines) > 0
]
ocr_lines <- read_lines(
  "data/workforce/training-flyer-ocr-5.3.2.txt"
)

ocr_lines
[1] "RIVERTON SKILLS OPEN HOUSE"   "DATA SUPPORT CERTIFICATE"    
[3] "Paid training stipend"        "Evening classes"             
[5] "No prior experience required" "Apply by October 15"

The published transcript uses committed output from native Tesseract 5.3.2. The host engine runs a diagnostic, but its output does not change the displayed transcript or metrics. The exact match applies to this image and software version. It does not establish general OCR accuracy.

ocr_provenance <- tibble(
  component = c(
    "R tesseract package",
    "Native Tesseract engine",
    "Language data",
    "Image MD5"
  ),
  value = c(
    as.character(
      utils::packageVersion("tesseract")
    ),
    ocr_information$version,
    "eng",
    image_md5
  )
)
Software and image details for this OCR run
Component Recorded value
R tesseract package 5.2.5
Native Tesseract engine 5.3.4
Language data eng
Image MD5 5dcfe9b824159b34c95b1d11e861ed54

The lockfile pins the R package. The native engine and language data come from the operating system, so the lesson reports them at run time. The repository also preserves the raw output recorded with native Tesseract 5.3.2 in data/workforce/training-flyer-ocr-5.3.2.txt.

Measure the difference

Edit distance counts the smallest number of character insertions, deletions, or substitutions needed to convert one text to another. We normalize case and punctuation before comparing because the research question concerns the words, not the flyer styling.

normalize_for_comparison <- function(text) {
  text |>
    str_c(collapse = " ") |>
    str_to_lower() |>
    str_replace_all("[^a-z0-9]+", " ") |>
    str_squish()
}

reference_text <- normalize_for_comparison(
  reference_lines
)
observed_text <- normalize_for_comparison(
  ocr_lines
)
normalized_edit_distance <- as.integer(
  adist(reference_text, observed_text)
)
normalized_distance_share <-
  normalized_edit_distance /
    str_length(reference_text)

tibble(
  normalized_edit_distance =
    normalized_edit_distance,
  reference_characters = str_length(reference_text),
  normalized_distance_share =
    normalized_distance_share
)
# A tibble: 1 × 3
  normalized_edit_distance reference_characters normalized_distance_share
                     <int>                <int>                     <dbl>
1                        0                  138                         0

An edit distance of zero means that the normalized texts match exactly. This is not a universal character-error rate. Normalization removed case, punctuation, and line boundaries, and the denominator includes spaces. The code separately requires exact matches for every raw line and the critical experience statement.

Damage the image and measure what is lost

This 1200-pixel-wide flyer at full size represents ideal conditions for the pipeline. Real material can include angled photographs, faxes, low-resolution scans, or prints from failing toner cartridges.

The lesson degrades the image with two reproducible operations. First, it retains every third row and column, reducing each dimension to a third of its original size. Then it lightens every third remaining row to imitate scanner banding.

Both operations use arithmetic on the pixel array without randomness. Each render therefore produces the same file.

keep_every <- 3L
row_positions <- seq(1L, dim(image_array)[1], by = keep_every)
column_positions <- seq(1L, dim(image_array)[2], by = keep_every)
degraded_array <- image_array[
  row_positions,
  column_positions, ,
  drop = FALSE
]
banded_rows <- seq(1L, dim(degraded_array)[1], by = 3L)
degraded_array[banded_rows, , ] <- pmin(
  degraded_array[banded_rows, , ] + 0.35,
  1
)

degraded_path <- tempfile(fileext = ".png")
png::writePNG(degraded_array, degraded_path)

degraded_diagnostic_text <- tesseract::ocr(
  degraded_path,
  engine = ocr_engine
)
degraded_diagnostic_lines <- degraded_diagnostic_text |>
  str_split_1(fixed("\n")) |>
  str_trim()
degraded_diagnostic_lines <- degraded_diagnostic_lines[
  str_length(degraded_diagnostic_lines) > 0
]
degraded_lines <- read_lines(
  "data/workforce/training-flyer-degraded-ocr-5.3.2.txt"
)
diagnostic_matches_fixture <- c(
  identical(diagnostic_ocr_lines, ocr_lines),
  identical(degraded_diagnostic_lines, degraded_lines)
)

tibble(
  version = c("source image", "degraded copy"),
  width_pixels = c(
    dim(image_array)[2],
    dim(degraded_array)[2]
  ),
  height_pixels = c(
    dim(image_array)[1],
    dim(degraded_array)[1]
  ),
  lines_returned = c(
    length(ocr_lines),
    length(degraded_lines)
  ),
  diagnostic_matches_fixture
)
# A tibble: 2 × 5
  version       width_pixels height_pixels lines_returned diagnostic_matches_f…¹
  <chr>                <int>         <int>          <int> <lgl>                 
1 source image          1200           800              6 TRUE                  
2 degraded copy          400           267              6 FALSE                 
# ℹ abbreviated name: ¹​diagnostic_matches_fixture

The final column reports whether the host engine reproduced each committed transcript. A FALSE value is a local-version difference, while the published metrics below remain tied to the recorded 5.3.2 output.

Character and word error rates

Two standard rates describe this comparison. Character error rate, or CER, divides character-level edit distance by the number of reference characters. Word error rate, or WER, applies the same calculation to whole words. A word with one incorrect letter therefore counts as one word error.

Word-level distance needs a separate calculation because adist() compares characters. The following function fills a cost table. Rows represent reference words, and columns represent observed words. The function returns the value in the final row and column.

word_edit_distance <- function(reference_words, observed_words) {
  costs <- matrix(
    0L,
    nrow = length(reference_words) + 1L,
    ncol = length(observed_words) + 1L
  )
  costs[, 1L] <- seq.int(0L, length(reference_words))
  costs[1L, ] <- seq.int(0L, length(observed_words))

  for (row in seq_along(reference_words)) {
    for (column in seq_along(observed_words)) {
      substitution <- if (
        identical(reference_words[row], observed_words[column])
      ) {
        0L
      } else {
        1L
      }
      costs[row + 1L, column + 1L] <- min(
        costs[row, column + 1L] + 1L,
        costs[row + 1L, column] + 1L,
        costs[row, column] + substitution
      )
    }
  }

  costs[
    length(reference_words) + 1L,
    length(observed_words) + 1L
  ]
}

error_rates <- function(reference, observed) {
  reference_words <- str_split_1(reference, " ")
  observed_words <- if (str_length(observed) == 0L) {
    character()
  } else {
    str_split_1(observed, " ")
  }
  character_distance <- as.integer(adist(reference, observed))
  word_distance <- word_edit_distance(
    reference_words,
    observed_words
  )

  tibble(
    reference_characters = str_length(reference),
    reference_word_count = length(reference_words),
    cer = character_distance / str_length(reference),
    wer = word_distance / length(reference_words)
  )
}

degraded_observed <- normalize_for_comparison(degraded_lines)
rate_table <- bind_rows(
  error_rates(reference_text, observed_text) |>
    mutate(version = "source image", .before = 1),
  error_rates(reference_text, degraded_observed) |>
    mutate(version = "degraded copy", .before = 1)
)
Error rates for the source image and its degraded copy
Version Reference characters Reference words CER WER
source image 138 20 0.000 0.0
degraded copy 138 20 0.558 1.2

The source image scores zero on both rates. The degraded copy does not. Both rates come from committed Tesseract 5.3.2 transcripts, so a native engine upgrade cannot rewrite the published result. The diagnostic comparison above reports whether the local engine produced the same transcripts.

comparison <- tibble(
  line = seq_len(max(length(reference_lines), length(degraded_lines))),
  reference = c(
    reference_lines,
    rep(NA_character_, max(0L, length(degraded_lines) - length(reference_lines)))
  )[seq_len(max(length(reference_lines), length(degraded_lines)))],
  degraded_reading = c(
    degraded_lines,
    rep(NA_character_, max(0L, length(reference_lines) - length(degraded_lines)))
  )[seq_len(max(length(reference_lines), length(degraded_lines)))]
)
The transcript beside what the degraded image produced
Line Reference transcript Degraded reading
1 RIVERTON SKILLS OPEN HOUSE AIVS SPCR SKILLS OREM FUL Sk
2 DATA SUPPORT CERTIFICATE LAI SJZRCRE CEAUIF CATE
3 Paid training stipend viel tras oe, esis
4 Evening classes Logs se senate
5 No prior experience required 8 he ee eis ites
6 Apply by October 15 Apps ey Oelaber 15

Read the table alongside the rate. A CER of 0.558 means 0.558 character edits per reference character. It does not measure the fraction of information preserved and has no meaningful midpoint. Insertions can increase CER or WER above one. Here, the remaining words do not form a faithful partial transcript.

Both rates compare one transcript with the output for one image. They describe this pair, not OCR performance in general.

The word rate also assumes that spaces separate words. That assumption works for this English flyer but not for Chinese, Japanese, and Thai. Those scripts require a character rate or script-aware segmentation.

OCR errors are not evenly distributed

OCR becomes harder when a source has:

  • low resolution, blur, shadows, or skew.
  • handwriting or decorative fonts.
  • tables, columns, stamps, or text over images.
  • uncommon symbols or several languages.
  • damaged pages.
  • a reading order that is not visually obvious.

An average error rate can hide the error that matters most. Changing "No prior experience required" to "prior experience required" would reverse the meaning while affecting only a few characters. The team must examine important fields directly, not rely on one summary score.

Preserve the source and correction history

Keep:

  1. the source image.
  2. the OCR engine and language settings.
  3. the unedited OCR output.
  4. the reviewed and corrected transcript.
  5. the identity of the reader or process that performed the checks.
  6. any uncertainty about layout or unreadable text.

Do not overwrite raw OCR with corrections. Separate fields make later edits visible and let another reviewer return to the source.

What to remember

  • Text inside an image must be recognized before ordinary text analysis.
  • OCR output is an estimate, not a transcription guarantee.
  • Compare important output with a reference after human checks.
  • Report character and word error rates. Identify the source image.
  • One clean image cannot give an estimate of accuracy on damaged material.
  • Measure errors. Examine cases where errors change meaning.
  • Preserve the image, raw output, corrected text, and correction history.

The archived job cards and reference transcript provide traceable text about training, schedules, requirements, and skills. Extraction does not explain the meaning of each sentence. Human classification follows, with a versioned annotation codebook to record the decisions.

Sources