Resolving coreference

Link pronouns to the noun phrases they may refer to

entity enrichment
coreference resolution
workforce research
Learn how a nearest-noun baseline handles pronouns in a constructed Riverton passage and where it fails.

Pronouns complicate the short applicant note. The expected answers give she one referent, while he can refer to more than one person.

Coreference resolution links different expressions that refer to the same thing. A pronoun such as they may refer to the workers. This lesson uses a small baseline rule so its mistakes remain visible.

Note

The Riverton passage and reference setting below are fictional.

TipWhat you will learn

By the end of this lesson, you will be able to:

  • explain coreference in plain language.
  • examine noun phrases and pronouns from a spaCy parse.
  • apply a nearest-agreeing-noun baseline.
  • compare the baseline with this page’s expected answers.
  • explain why document-level context matters.

Use a constructed passage

The lesson first examines the Riverton CSV. It then uses a constructed passage because the stored sentences contain too few pronouns to teach coreference.

library(readr)
library(dplyr)
library(tibble)
library(purrr)
library(stringr)
library(spacyr)
source("R/use-spacy.R")

pipeline <- use_project_spacy()
pipeline_version <- spacy_pipeline_version()

sentences <- read_csv(
  "data/workforce/workforce_sentences.csv",
  na = character(),
  col_types = cols(
    sentence_id = col_character(),
    document_id = col_character(),
    source_line = col_character(),
    text = col_character(),
    reference_label = col_character(),
    uncertainty = col_character(),
    annotator_id = col_character(),
    rationale = col_character(),
    codebook_version = col_character(),
    codebook_hash = col_character(),
    derived = col_character(),
    transformation = col_character()
  )
)

riverton_parsed <- spacy_parse(
  setNames(sentences$text, sentences$sentence_id),
  pos = TRUE,
  lemma = TRUE
) |>
  as_tibble()

riverton_pronouns <- riverton_parsed |>
  filter(pos == "PRON")

teaching_passage <- paste(
  "The coordinator posted the training notice on Monday.",
  "The applicant read it before the bus arrived.",
  "The supervisor called the applicant after she finished the form.",
  "The supervisor told the coordinator that he could open another class.",
  "The workers thanked the supervisor because they needed evening seats.",
  "The centre moved the flyer near the entrance, and it stayed there all week.",
  sep = " "
)

parsed_raw <- spacy_parse(
  c(teaching_passage = teaching_passage),
  pos = TRUE,
  tag = TRUE,
  lemma = TRUE,
  entity = TRUE,
  dependency = TRUE,
  nounphrase = TRUE,
  additional_attributes = c("morph")
)

parsed <- parsed_raw |>
  as_tibble() |>
  mutate(
    global_token = row_number(),
    morph = map_chr(morph, as.character)
  )

phrases <- nounphrase_extract(parsed_raw)
Pronouns found in the 28 Riverton sentences
Sentence ID Token spaCy part of speech
s016 This PRON
spaCy pipeline used for the parse
Pipeline Version Language License spaCy
core_web_sm 3.8.0 en MIT 3.8.7
Noun phrases extracted from the constructed passage
Document Sentence Noun phrase
teaching_passage 1 The_coordinator
teaching_passage 1 the_training_notice
teaching_passage 1 Monday
teaching_passage 2 The_applicant
teaching_passage 2 it
teaching_passage 2 the_bus
teaching_passage 3 The_supervisor
teaching_passage 3 the_applicant
teaching_passage 3 she
teaching_passage 3 the_form
teaching_passage 4 The_supervisor
teaching_passage 4 the_coordinator
teaching_passage 4 he
teaching_passage 4 another_class
teaching_passage 5 The_workers
teaching_passage 5 the_supervisor
teaching_passage 5 they
teaching_passage 5 evening_seats
teaching_passage 6 The_centre
teaching_passage 6 the_flyer
teaching_passage 6 the_entrance
teaching_passage 6 it

spaCy marks one pronoun in the 28 Riverton sentences. One example is not enough to teach pronoun linking, so the constructed passage gives the rule more chances to fail.

The installed en_core_web_sm pipeline has named-entity, tagger, parser, and other components, but no coreference component. The rule below is a baseline, not a full coreference system.

Apply the baseline rule

The rule links each personal pronoun to the nearest preceding noun phrase with the same grammatical number. Number means singular or plural. spaCy reports that feature through morphology, the grammatical properties of a word’s form, on the noun-phrase head. The head is the phrase’s central word. An antecedent is the expression to which a pronoun refers.

number_from_morph <- function(morph_value) {
  case_when(
    str_detect(morph_value, "Number=Plur") ~ "Plur",
    str_detect(morph_value, "Number=Sing") ~ "Sing",
    TRUE ~ NA_character_
  )
}

phrase_tokens <- parsed |>
  mutate(
    np_inside = nounphrase != "",
    np_id = if_else(np_inside, cumsum(str_detect(nounphrase, "beg")), NA_integer_)
  ) |>
  filter(np_inside)

noun_phrases <- phrase_tokens |>
  summarise(
    phrase = paste(token, collapse = " "),
    root_token = token[str_detect(nounphrase, "root")][1],
    root_pos = pos[str_detect(nounphrase, "root")][1],
    root_morph = morph[str_detect(nounphrase, "root")][1],
    start_token = min(global_token),
    end_token = max(global_token),
    .by = np_id
  ) |>
  mutate(number = number_from_morph(root_morph))

antecedents <- noun_phrases |>
  filter(root_pos %in% c("NOUN", "PROPN"), !is.na(number)) |>
  rename(antecedent = phrase, antecedent_number = number) |>
  mutate(join_key = 1L)

pronouns <- parsed |>
  filter(pos == "PRON", str_to_lower(token) %in% c("it", "she", "he", "they")) |>
  transmute(
    pronoun_id = paste0("p", row_number()),
    pronoun = token,
    sentence_id,
    pronoun_token = global_token,
    pronoun_number = number_from_morph(morph),
    join_key = 1L
  )

rule_links <- pronouns |>
  left_join(antecedents, by = join_by(join_key), relationship = "many-to-many") |>
  filter(end_token < pronoun_token, antecedent_number == pronoun_number) |>
  arrange(pronoun_id, desc(end_token)) |>
  summarise(
    rule_antecedent = first(antecedent),
    .by = c(pronoun_id, pronoun, sentence_id, pronoun_number)
  )
Nearest preceding noun phrase with matching number
Pronoun ID Pronoun Sentence Number Rule antecedent
p1 it 2 Sing The applicant
p2 she 3 Sing the applicant
p3 he 4 Sing the coordinator
p4 they 5 Plur The workers
p5 it 6 Sing the entrance

The rule chooses The applicant for the first it because that is the nearest singular noun phrase. A reader can see that it means the training notice.

Score the rule

The expected answers treat he as genuinely ambiguous because the supervisor or the coordinator could open another class. The lesson counts that row separately.

marked_coreference <- tibble(
  pronoun_id = c("p1", "p2", "p3", "p4", "p5"),
  expected_antecedent = c(
    "the training notice",
    "the applicant",
    "ambiguous",
    "The workers",
    "the flyer"
  ),
  scoreable = c(TRUE, TRUE, FALSE, TRUE, TRUE)
)

coreference_score <- rule_links |>
  left_join(marked_coreference, by = join_by(pronoun_id)) |>
  mutate(
    result = case_when(
      !scoreable ~ "ambiguous reference",
      str_to_lower(rule_antecedent) == str_to_lower(expected_antecedent) ~ "correct",
      TRUE ~ "wrong"
    )
  )

score_counts <- tibble(result = c("correct", "wrong", "ambiguous reference")) |>
  left_join(
    coreference_score |>
      count(result, name = "pronouns"),
    by = join_by(result)
  ) |>
  mutate(pronouns = coalesce(pronouns, 0L))

spacy_finalize()
Coreference baseline compared with expected answers
Pronoun ID Pronoun Rule antecedent Expected answer Result
p1 it The applicant the training notice wrong
p2 she the applicant the applicant correct
p3 he the coordinator ambiguous ambiguous reference
p4 they The workers The workers correct
p5 it the entrance the flyer wrong
Correct, wrong, and ambiguous pronoun links
Result Pronouns
correct 2
wrong 2
ambiguous reference 1

Because the baseline and expected answers share an author, the score is only a check on this page. The two correct answers are easy cases. The workers is the only plural phrase before they, and the rule links she without using gender. Excluding the ambiguous he row changes the scored denominator from five pronouns to four. Real coreference systems need document context, syntax, and meaning.

Modern coreference resolution

The nearest-agreeing-noun syntax rule is a deterministic baseline, so the same input always produces the same result. It demonstrates how coreference links work. Most complex coreference tasks now use neural models rather than fixed syntax rules. Examples include models implemented with AllenNLP and large language models.

Consider these limits when evaluating modern systems:

  • Cluster metrics: Clusters group mentions of the same entity. Specialized metrics such as MUC, B³, and CEAF evaluate these groups rather than individual mention pairs. Errors can compound across long chains of linked mentions.
  • Ambiguity: Language can be ambiguous, as the he example shows. Deterministic models often choose a link where a person would preserve uncertainty.
  • Stereotype and coverage asymmetry: Coreference models can learn occupational gender stereotypes from training data. They may link he to doctor and she to nurse correctly but fail on examples that contradict those stereotypes or use gender-neutral they.

What to remember

  • Coreference links expressions that refer to the same thing.
  • en_core_web_sm does not include a coreference resolver.
  • A nearest-agreeing-noun rule is a baseline for inspection.
  • Mark ambiguous pronouns as ambiguous rather than choose an unsupported link.

After examining the five pronouns, the coordinator has two easy matches, two mistakes, and one ambiguity. That is enough to explain the baseline, not enough to use it as a resolver.

Sources