Drawing labels on text spans

Use offsets, escaping, and visible labels for annotated text

information visualization

annotated text

named entities

Learn how to draw safe inline annotations from spaCy offsets and why a table of spans must stay beside the highlights.

Sam examines short job-board and flyer sentences. A table records entity labels, but matching rows to words can require reading each sentence twice.

Annotated text visualization draws labels on spans of text while leaving the source text unchanged. A span is a continuous portion of text located by character offsets. The display helps readers examine labels. Correct offsets, escaped text, and explicit overlap rules are necessary for accurate display.

Note

The author created the fictional Riverton Workforce Lab, job board, and training flyer for teaching.

What you will learn

This lesson shows how to:

use spaCy token offsets instead of searching for entity strings
convert 0-based, end-exclusive offsets for R’s substr()
draw labels with htmltools tag objects that escape text
keep annotation layers separate when spans overlap
provide a captioned table with every span
record the spaCy and model versions beside the display.

Start from offsets, not searched text

The spaCy pipeline returns each token’s idx, the 0-based character position where that token starts in the original string. The end of a span is the first character after it. R’s substr() starts counting at 1 and includes its end position, so the slice uses start + 1 and end.

library(dplyr)
library(htmltools)
library(purrr)
library(readr)
library(spacyr)
library(stringr)
library(tibble)

source("R/use-spacy.R")

sentences <- read_csv(
  "data/workforce/workforce_sentences.csv",
  na = character(),
  col_types = cols(
    sentence_id = col_character(),
    document_id = col_character(),
    source_line = col_character(),
    text = col_character(),
    reference_label = col_character(),
    uncertainty = col_character(),
    annotator_id = col_character(),
    rationale = col_character(),
    codebook_version = col_character(),
    codebook_hash = col_character(),
    derived = col_character(),
    transformation = col_character()
  )
)

display_sentences <- sentences |>
  filter(sentence_id %in% c("s001", "s023", "s028")) |>
  transmute(
    sentence_id,
    source_text = text
  )

pipeline <- use_project_spacy()
pipeline_info <- spacy_pipeline_version()

parsed <- spacy_parse(
  setNames(display_sentences$source_text, display_sentences$sentence_id),
  pos = TRUE,
  entity = TRUE,
  dependency = TRUE,
  additional_attributes = "idx"
) |>
  as_tibble()

entity_tokens <- parsed |>
  filter(nzchar(entity)) |>
  mutate(
    label = str_remove(entity, "_[BI]$"),
    begins_span = str_detect(entity, "_B$")
  ) |>
  group_by(doc_id) |>
  mutate(span_number = cumsum(begins_span)) |>
  ungroup()

model_spans <- entity_tokens |>
  left_join(
    display_sentences,
    by = c("doc_id" = "sentence_id")
  ) |>
  group_by(doc_id, span_number, source_text, label) |>
  summarise(
    start = min(idx),
    end = max(idx + nchar(token, type = "chars")),
    .groups = "drop"
  ) |>
  transmute(
    sentence_id = doc_id,
    layer = "spaCy model",
    start,
    end,
    text = substr(source_text, start + 1L, end),
    label,
    reviewed = "model output"
  )

spaCy runtime and English model used for this display
Field	Value
spacyr	1.3.0
name	core_web_sm
version	3.8.0
lang	en
license	MIT
spacy	3.8.7

The model gives HOUSE the label ORG. That incorrect span stays visible. Lesson 27 examines the named-entity output. This lesson shows how to draw labels without losing the source positions.

The entity tags use B for the first token in a span and I for later tokens inside the same span. In s001, the three tagged tokens start at characters 5, 7, and 8. Together they form the span from 5 to 12, with position 12 excluded.

s001_token_offsets <- parsed |>
  filter(doc_id == "s001") |>
  transmute(
    token,
    start = idx,
    end = idx + nchar(token, type = "chars"),
    entity_tag = if_else(nzchar(entity), entity, "outside")
  )

spaCy token offsets and entity tags for sentence s001
Token	Start	End	Entity tag
Paid	0	4	outside
12	5	7	DATE_B
-	7	8	DATE_I
week	8	12	DATE_I
training	13	21	outside
is	22	24	outside
provided	25	33	outside
.	33	34	outside

Searching for an entity string rebuilt with spaces can fail even when the offsets are right.

s001_source <- display_sentences |>
  filter(sentence_id == "s001") |>
  pull(source_text)

offset_slice <- model_spans |>
  filter(sentence_id == "s001") |>
  transmute(
    search_text = "12 - week",
    search_found = !is.na(str_locate(s001_source, fixed(search_text))[, 1]),
    offset_slice = substr(s001_source, start + 1L, end)
  )

Offset slicing recovers the span when a searched string fails
Searched text	Search found	Offset slice
12 - week	FALSE	12-week

Examine code-point units with an accented name

Offsets need a counting rule. spaCy’s Python strings count Unicode code points, the numerical units that identify characters. R’s substr() here uses character positions, not bytes. The sentence below includes accented characters so the byte count differs from the character count.

accent_text <- "Renée Díaz runs the Tools & Dies class at Riverton Skills Centre in October."
accent_parsed <- spacy_parse(
  c(accent = accent_text),
  pos = TRUE,
  entity = TRUE,
  dependency = TRUE,
  additional_attributes = "idx"
) |>
  as_tibble()

accent_person <- accent_parsed |>
  filter(str_detect(entity, "^PERSON")) |>
  summarise(
    start = min(idx),
    end = max(idx + nchar(token, type = "chars")),
    text = substr(accent_text, start + 1L, end),
    .groups = "drop"
  )

byte_offset <- function(text, position) {
  if (position == 0L) {
    return(0L)
  }

  nchar(substr(text, 1L, position), type = "bytes")
}

person_byte_start <- byte_offset(accent_text, accent_person$start)
person_byte_end <- byte_offset(accent_text, accent_person$end)
wrong_byte_as_character_slice <- substr(
  accent_text,
  accent_person$start + 1L,
  person_byte_end
)

accent_units <- tibble(
  example = "accented constructed sentence",
  characters = nchar(accent_text, type = "chars"),
  bytes = nchar(accent_text, type = "bytes"),
  person_start = accent_person$start,
  person_end = accent_person$end,
  byte_start = person_byte_start,
  byte_end = person_byte_end,
  wrong_slice_if_bytes_are_positions = wrong_byte_as_character_slice,
  sliced_text = accent_person$text
)

Character offsets still slice the accented name correctly
Example	Characters	Bytes	Start	End	Byte start	Byte end	Wrong slice if bytes are used as positions	Sliced text
accented constructed sentence	76	78	0	10	0	12	Renée Díaz r	Renée Díaz

Do not use byte offsets as character offsets. Here, byte offsets would select two extra characters (Renée Díaz r). Every later span would also shift right from its correct position.

Draw one layer only when spans do not overlap

The renderer accepts one sentence and one layer of non-overlapping spans. It builds the line with htmltools tags, which escape source text. Escaping prevents the browser from interpreting source characters as HTML markup. Each mark contains a visible label. Background color provides only an additional cue. Two stylesheet classes supply colors and borders, so the code only names those classes.

has_layer_overlap <- function(spans) {
  ordered <- spans |>
    arrange(start, end)

  if (nrow(ordered) < 2L) {
    return(FALSE)
  }

  any(ordered$start[-1] < ordered$end[-nrow(ordered)])
}

render_annotation_layer <- function(sentence_id, layer_name, spans, source_text) {
  if (has_layer_overlap(spans)) {
    stop("This layer has overlapping spans. Show it as a table.", call. = FALSE)
  }

  ordered <- spans |>
    arrange(start, end)
  cursor <- 0L
  parts <- list(
    tags$strong(paste(layer_name, ": ", sep = ""))
  )
  plain_parts <- character()

  for (row_index in seq_len(nrow(ordered))) {
    span <- ordered[row_index, ]

    if (span$start > cursor) {
      before <- substr(source_text, cursor + 1L, span$start)
      parts <- append(parts, list(before))
      plain_parts <- c(plain_parts, before)
    }

    span_text <- substr(source_text, span$start + 1L, span$end)
    label_text <- paste("[", span$label, "]", sep = "")

    parts <- append(
      parts,
      list(
        tags$mark(
          class = "annotation-mark",
          .noWS = "outside",
          span_text,
          tags$span(class = "annotation-label", label_text)
        )
      )
    )
    plain_parts <- c(plain_parts, span_text)
    cursor <- span$end
  }

  if (cursor < nchar(source_text, type = "chars")) {
    after <- substr(source_text, cursor + 1L, nchar(source_text, type = "chars"))
    parts <- append(parts, list(after))
    plain_parts <- c(plain_parts, after)
  }

  list(
    sentence_id = sentence_id,
    layer = layer_name,
    plain_text = paste(plain_parts, collapse = ""),
    html = tags$p(
      class = "annotated-text-example",
      do.call(tagList, parts)
    )
  )
}

model_lines <- display_sentences |>
  mutate(
    spans = map(
      sentence_id,
      \(current_id) {
        model_spans |>
          filter(sentence_id == current_id, layer == "spaCy model")
      }
    ),
    rendered = pmap(
      list(sentence_id, spans, source_text),
      \(sentence_id, spans, source_text) {
        render_annotation_layer(
          sentence_id = sentence_id,
          layer_name = "spaCy model",
          spans = spans,
          source_text = source_text
        )
      }
    )
  )

tagList(map(model_lines$rendered, "html"))

spaCy model: Paid 12-week [DATE] training is provided.

spaCy model: RIVERTON SKILLS OPEN HOUSE [ORG]

spaCy model: Apply by October 15 [DATE]

The marks are useful because the original words stay in order. The table below is still necessary: it gives the offsets and labels in a form that does not depend on color or inline layout.

Keep a table of every span

Sam also shows a constructed example with nested annotations. Riverton can receive GPE, spaCy’s label for a geopolitical place such as a town. That name appears inside the organisation name Riverton Skills Centre. Separate layers keep the annotations from overlapping on either line.

review_sentence <- tibble(
  sentence_id = "constructed-01",
  source_text = "Riverton Skills Centre will run evening classes in October."
)

review_spans <- tibble(
  sentence_id = c("constructed-01", "constructed-01"),
  layer = c("reviewer organisation", "reviewer place"),
  start = c(0L, 0L),
  end = c(22L, 8L),
  text = c("Riverton Skills Centre", "Riverton"),
  label = c("ORG", "GPE"),
  reviewed = c("author-written layer", "author-written layer")
)

all_visible_spans <- bind_rows(
  model_spans,
  review_spans
)

Every span used in the inline annotated-text displays
Sentence ID	Layer	Start	End	Text	Label	Annotation source
constructed-01	reviewer organisation	0	22	Riverton Skills Centre	ORG	author-written layer
constructed-01	reviewer place	0	8	Riverton	GPE	author-written layer
s001	spaCy model	5	12	12-week	DATE	model output
s023	spaCy model	21	26	HOUSE	ORG	model output
s028	spaCy model	9	19	October 15	DATE	model output

The renderer now draws each reviewer layer on a separate line.

review_lines <- review_spans |>
  group_by(layer) |>
  group_split() |>
  map(
    \(layer_spans) {
      render_annotation_layer(
        sentence_id = "constructed-01",
        layer_name = layer_spans$layer[[1]],
        spans = layer_spans,
        source_text = review_sentence$source_text[[1]]
      )
    }
  )

tagList(map(review_lines, "html"))

reviewer organisation: Riverton Skills Centre [ORG] will run evening classes in October.

reviewer place: Riverton [GPE] Skills Centre will run evening classes in October.

The two reviewer labels are not a spaCy result. They are an author-written example showing why layers matter.

Show overlap instead of forcing it inline

The two reviewer spans overlap within one layer. The lesson displays that layer as a table instead of one inline row.

overlap_spans <- review_spans |>
  mutate(layer = "combined reviewer layer")

overlap_report <- overlap_spans |>
  mutate(
    inline_display = if_else(
      has_layer_overlap(overlap_spans),
      "table view only: spans overlap within the layer",
      "safe for inline display"
    )
  )

The table retains overlapping spans instead of combining them in one inline layer
Sentence ID	Layer	Start	End	Text	Label	Annotation source	Display decision
constructed-01	combined reviewer layer	0	22	Riverton Skills Centre	ORG	author-written layer	table view only: spans overlap within the layer
constructed-01	combined reviewer layer	0	8	Riverton	GPE	author-written layer	table view only: spans overlap within the layer

Some specialized tools can draw nested spans. Crossing spans are harder because neither tag can contain the other. Draw one non-overlapping layer at a time. Keep the complete table.

Escape source text by construction

The renderer treats source text as text. The example below contains <b> and &. They should appear as characters, not as an HTML tag or entity.

escape_text <- "Sam wrote <b> & kept it as text."
escape_span <- tibble(
  sentence_id = "constructed-escape",
  layer = "escaping check",
  start = 10L,
  end = 13L,
  text = "<b>",
  label = "TEXT",
  reviewed = "constructed hostile text"
)

escape_line <- render_annotation_layer(
  sentence_id = "constructed-escape",
  layer_name = "escaping check",
  spans = escape_span,
  source_text = escape_text
)

escape_line$html

escaping check: Sam wrote <b> [TEXT] & kept it as text.

The same rule applies to tables that show source text. Use an escaping HTML table so text such as <b> remains visible.

escape_table <- escape_span |>
  select(sentence_id, layer, start, end, text, label, reviewed)

Markup-like source text preserved as table text
Sentence ID	Layer	Start	End	Text	Label	Annotation source
constructed-escape	escaping check	10	13	<b>	TEXT	constructed hostile text

What this display does not prove

The highlights show where the model or reviewer layer placed labels. They do not prove that the labels are correct. The page retains HOUSE because it shows a model-output error from the earlier named-entity recognition lesson.

The page does not certify accessibility. Visible label text means that color is not the only indication of meaning. The style targets the automated contrast scan. Screen-reader and forced-colors behavior still need manual checks.

What to remember

An annotated text visualization draws labels on spans while keeping source text unchanged.
spaCy’s idx gives 0-based token starts. R’s substr() needs start + 1.
Slice spans from offsets instead of searching for joined entity strings.
Build inline HTML with tag objects so source text is escaped as text.
Put visible labels inside the marks.
Do not rely on color alone.
Draw one non-overlapping layer at a time.
Keep a captioned span table.

Sam can read the highlights and find each label’s sentence ID, layer, and offset pair.

Start from offsets, not searched text

Examine code-point units with an accented name

Draw one layer only when spans do not overlap

Keep a table of every span

Show overlap instead of forcing it inline

Escape source text by construction

What this display does not prove

What to remember

Sources