library(readr)
library(dplyr)
library(tibble)
library(stringr)
source("R/permutation-null.R")
senses <- read_csv(
"data/wordnet/wordnet-senses.csv",
na = character(),
col_types = cols(
lemma = col_character(),
part_of_speech = col_character(),
sense_id = col_character(),
synset_id = col_character(),
sense_order = col_integer()
)
)
synsets <- read_csv(
"data/wordnet/wordnet-synsets.csv",
na = character(),
col_types = cols(
synset_id = col_character(),
part_of_speech = col_character(),
definition = col_character(),
example = col_character()
)
)
members <- read_csv(
"data/wordnet/wordnet-members.csv",
na = character(),
col_types = cols(
synset_id = col_character(),
lemma = col_character()
)
)
relations <- read_csv(
"data/wordnet/wordnet-relations.csv",
na = character(),
col_types = cols(
synset_id = col_character(),
relation = col_character(),
target_synset_id = col_character(),
target_definition = col_character(),
target_members = col_character()
)
)
metadata <- read_csv(
"data/wordnet/wordnet-metadata.csv",
na = character(),
col_types = cols(
field = col_character(),
value = col_character()
)
)
metadata_summary <- tibble(
item = c("lemmas", "senses", "synsets", "synset membership rows", "relations"),
value = c(
n_distinct(senses$lemma),
nrow(senses),
nrow(synsets),
nrow(members),
nrow(relations)
)
)Reading WordNet synsets
Distinguish a wordβs spelling from its meanings
The label bank could describe river photographs, money records, or a toy building with a coin slot. The spelling alone does not show the intended meaning.
A synset is a set of words that share one sense. WordNet is a manually compiled lexical database of words and meanings. It records synsets and links them to broader meanings called hypernyms. A hypernym names a broader category in an βis-aβ hierarchy.
This lesson shows how to:
- explain a synset in plain language
- count senses for a word string
- compare a sense count with a simple chance baseline
- list synonyms inside one synset
- follow hypernym links to broader categories
- name the functions and limits of WordNet.
Load the WordNet extract
The lesson reads four local lexical CSV files made from Open English WordNet 2024. No code downloads WordNet while the page renders. The metadata file records the source, licence, and SHA-256 fingerprints, which identify file contents.
| Item | Count |
|---|---|
| lemmas | 38 |
| senses | 419 |
| synsets | 417 |
| synset membership rows | 1050 |
| relations | 2432 |
This extract covers 38 lemmas, not all of English. A lemma is the dictionary form of a word. The extract shows how a lexical database separates spelling from meaning.
Read the senses of one word
A sense is one listed meaning of a word. The string bank has 18 senses in this extract: 10 noun senses and 8 verb senses. The first noun senses split river land, finance, piles, rows, reserves, and gambling funds.
bank_senses <- senses |>
filter(lemma == "bank") |>
count(part_of_speech, name = "senses") |>
arrange(part_of_speech)
bank_noun_senses <- senses |>
filter(lemma == "bank", part_of_speech == "noun") |>
arrange(sense_order) |>
left_join(synsets, by = c("synset_id", "part_of_speech")) |>
slice_head(n = 6) |>
transmute(
sense = sense_order,
definition
)
lemma_sense_counts <- senses |>
count(lemma, name = "sense_count")
bank_sense_null <- permutation_null(
observed = sum(senses$lemma == "bank"),
replicate_fn = function() {
lemma_sense_counts$sense_count[
match(sample(lemma_sense_counts$lemma, 1L), lemma_sense_counts$lemma)
]
},
replicates = 1000L,
seed = 5801L,
alternative = "greater"
)| Noun sense | Definition |
|---|---|
| 1 | sloping land (especially the slope beside a body of water) |
| 2 | a financial institution that accepts deposits and channels the money into lending activities |
| 3 | a long ridge or pile |
| 4 | an arrangement of similar objects in a row or in tiers |
| 5 | a supply or stock held in reserve for future use (especially in emergencies) |
| 6 | the funds held by a gambling house or the dealer in some gambling games |
| Observed | Null mean | Null 5th pct | Null 95th pct | p-value | Replicates | Alternative |
|---|---|---|---|---|---|---|
| 18 | 11.352 | 2 | 48 | 0.138 | 1000 | greater |
A word string is not a unit of meaning. Treating every bank occurrence as the same meaning simplifies a search, model, or count. The null table compares the sense count with counts for randomly selected lemmas. The extract contains ambiguous teaching examples, so 18 senses for bank is not unusual within this small file.
Any lexical method will return an answer for a known word. Compare the target word with other words from the same source before interpreting the answer.
Look inside a synset
For the financial-institution sense, the synset contains four member lemmas. These are four names WordNet links to the same sense.
financial_bank_synset <- senses |>
filter(lemma == "bank", part_of_speech == "noun", sense_order == 2L) |>
pull(synset_id)
financial_bank_members <- members |>
filter(synset_id == financial_bank_synset) |>
arrange(lemma) |>
mutate(role = if_else(lemma == "bank", "queried word", "other member"))| Synset ID | Lemma | Role |
|---|---|---|
| oewn-08437235-n | bank | queried word |
| oewn-08437235-n | banking company | other member |
| oewn-08437235-n | banking concern | other member |
| oewn-08437235-n | depository financial institution | other member |
Synonymy here is local to one sense. bank and banking company share the finance sense. river bank has a different sense.
Follow an is-a hierarchy
A hypernym names a broader category for a meaning. In this extract, the professor-position sense of chair links to a position synset, then a work synset, then activity.
first_hypernym <- function(synset) {
next_row <- relations |>
filter(synset_id == synset, relation == "hypernym") |>
arrange(target_synset_id) |>
slice_head(n = 1)
if (nrow(next_row) != 1L) {
stop("No hypernym recorded for synset: ", synset, call. = FALSE)
}
next_row
}
select_chair_synset <- function() {
candidates <- senses |>
filter(lemma == "chair", part_of_speech == "noun") |>
left_join(synsets, by = c("synset_id", "part_of_speech")) |>
filter(str_detect(definition, regex("professorship|professor", ignore_case = TRUE))) |>
arrange(sense_order)
if (nrow(candidates) != 1L) {
stop("Expected one professor-position chair sense in this extract.", call. = FALSE)
}
candidates$synset_id[[1]]
}
walk_hypernyms <- function(start_synset, steps) {
path <- start_synset
for (step in seq_len(steps)) {
path <- c(path, first_hypernym(path[length(path)])$target_synset_id)
}
path
}
lookup_definition <- function(id) {
local_definition <- synsets$definition[synsets$synset_id == id]
if (length(local_definition) > 0L) {
return(local_definition[[1]])
}
external_definition <- relations$target_definition[match(id, relations$target_synset_id)]
if (is.na(external_definition)) {
stop("No definition available for synset: ", id, call. = FALSE)
}
external_definition
}
lookup_members <- function(id) {
local_members <- members |>
filter(synset_id == id) |>
arrange(lemma) |>
pull(lemma)
if (length(local_members) > 0L) {
return(paste(local_members, collapse = ", "))
}
external_members <- relations$target_members[match(id, relations$target_synset_id)]
if (is.na(external_members)) {
stop("No members available for synset: ", id, call. = FALSE)
}
external_members
}
chair_synset <- select_chair_synset()
path_ids <- walk_hypernyms(chair_synset, steps = 3L)
hypernym_path <- tibble(
level = seq_along(path_ids) - 1L,
synset_id = path_ids,
members = vapply(path_ids, lookup_members, character(1)),
definition = vapply(path_ids, lookup_definition, character(1))
)| Level | Synset ID | Members | Definition |
|---|---|---|---|
| 0 | oewn-00599171-n | chair, professorship | the position of professor |
| 1 | oewn-00587299-n | berth, office, place, position, post, situation | a job in an organization |
| 2 | oewn-00583425-n | business, job, line, line of work, occupation | the principal activity in your life that you do to earn money |
| 3 | oewn-00408356-n | activity | any specific behavior |
The lookup assumes this extract contains the chosen chair sense and three successive hypernym links. The functions stop when that assumption is false. A missing lemma therefore cannot produce an incomplete path that appears valid.
The hierarchy is useful for broadening a search. A search for occupations might group chair, office, and job under a broader category. The hierarchy can be too general when a task requires academic rank, the furniture sense, or a named committee role.
State the limits
WordNet is a curated resource with a version and a licence. It is not a direct measurement of English. It covers much common vocabulary, while domain terms, slang, new coinages, and local meanings can be missing.
Sense inventories distinguish meanings at a chosen level of detail. A distinction that helps one annotation task can be too detailed for another. Treat a WordNet sense as an inventory entry, not a definitive account of meaning.
WordNet can help teach the difference between words and meanings or expand a controlled search. It can find labels that share a listed sense. Users can also build small rule-based features from WordNet when they can examine vocabulary coverage.
limit_check <- tibble(
check = c("version recorded", "licence recorded", "target lemmas recorded"),
value = c(
metadata$value[metadata$field == "source_version"],
metadata$value[metadata$field == "source_licence"],
metadata$value[metadata$field == "n_lemmas_found"]
)
)| Check | Value |
|---|---|
| version recorded | Open English Wordnet 2024 |
| licence recorded | https://creativecommons.org/licenses/by/4.0 |
| target lemmas recorded | 38 |
What to remember
- A synset is a set of words sharing one sense.
bankhas 18 senses in this extract, so one spelling is not one meaning.- A null check compares the sense count with counts for other lemmas in this extract.
- Hypernym links make an is-a hierarchy that can broaden a search.
- WordNet is useful when a curated inventory is acceptable and users can examine its limited coverage.