Parsing prices from text

Extract pay amounts and units

entity enrichment

price parsing

workforce research

Learn how R rules extract wages and stipends while preserving ranges and missing values.

The team gives labels to sentences from the job board and training flyer. The coordinator then asks which notices contain specific pay information.

A phrase such as $18.50 per hour looks easy. A phrase such as competitive pay looks useful to a reader but gives a computer no amount to compare. The team needs a rule that extracts values and reports when the text contains no usable price.

Note

The Riverton job board and flyer in this lesson are fictional teaching materials.

What you will learn

By the end of this lesson, you will be able to:

explain why a number alone is not a price.
write stringr patterns for money-like text.
separate amount, currency, and pay period.
keep ranges and missing values visible.
compare a small rule set with this page’s expected answers.

Build examples with expected answers

readr reads the Riverton CSV, while dplyr and tibble prepare tables. stringr matches patterns for monetary amounts. The examples are author-created pay strings in the Riverton style. A price here means an amount, a currency, and a unit such as hour, year, or stipend.

library(readr)
library(dplyr)
library(tibble)
library(stringr)

sentences <- read_csv(
  "data/workforce/workforce_sentences.csv",
  na = character(),
  col_types = cols(
    sentence_id = col_character(),
    document_id = col_character(),
    source_line = col_character(),
    text = col_character(),
    reference_label = col_character(),
    uncertainty = col_character(),
    annotator_id = col_character(),
    rationale = col_character(),
    codebook_version = col_character(),
    codebook_hash = col_character(),
    derived = col_character(),
    transformation = col_character()
  )
)

pay_examples <- tibble(
  example_id = sprintf("pay-%02d", 1:13),
  text = c(
    "$18.50 per hour",
    "18.50/hr",
    "$38,000 a year",
    "$17-$19 an hour",
    "competitive pay",
    "$1,250 stipend",
    "$20 hourly",
    "Pay starts at $16 per hour",
    "$42,500 annually",
    "Stipend of 750 dollars",
    "Up to $21/hour",
    "18 dollars an hour",
    "$38000 a year"
  ),
  expected_currency = c(
    "USD", NA, "USD", "USD", NA, "USD",
    "USD", "USD", "USD", "USD", "USD", "USD",
    "USD"
  ),
  expected_unit = c(
    "hour", "hour", "year", "hour", NA, "stipend",
    "hour", "hour", "year", "stipend", "hour", "hour",
    "year"
  ),
  expected_lower = c(
    18.50, 18.50, 38000, 17, NA, 1250,
    20, 16, 42500, 750, 21, 18,
    38000
  ),
  expected_upper = c(
    NA, NA, NA, 19, NA, NA,
    NA, NA, NA, NA, NA, NA,
    NA
  )
)

Author-created price examples with expected answers
Example ID	Text	Expected currency	Expected unit	Expected lower amount	Expected upper amount
pay-01	$18.50 per hour	USD	hour	18.5	NA
pay-02	18.50/hr	NA	hour	18.5	NA
pay-03	$38,000 a year	USD	year	38000.0	NA
pay-04	$17-$19 an hour	USD	hour	17.0	19
pay-05	competitive pay	NA	NA	NA	NA
pay-06	$1,250 stipend	USD	stipend	1250.0	NA
pay-07	$20 hourly	USD	hour	20.0	NA
pay-08	Pay starts at $16 per hour	USD	hour	16.0	NA
pay-09	$42,500 annually	USD	year	42500.0	NA
pay-10	Stipend of 750 dollars	USD	stipend	750.0	NA
pay-11	Up to $21/hour	USD	hour	21.0	NA
pay-12	18 dollars an hour	USD	hour	18.0	NA
pay-13	$38000 a year	USD	year	38000.0	NA

The table contains 13 examples. One writes a five-digit amount without a comma. That case matters because a partial match would look like a real answer.

When an example gives one amount, the expected answers put it in the lower-amount column. That is correct for Pay starts at $16 per hour, because that text gives the minimum pay. In Up to $21/hour, the amount is the maximum pay. A correct answer would put 21 in the upper-amount column and leave the lower amount missing. The parser below has no rule for up to, and a production parser needs one.

Extract the visible components

The pattern below uses the ICU regular expression engine through stringr. A regular expression is a compact search pattern. This pattern matches an optional dollar sign followed by a number that may contain commas or cents. It also matches an optional second number after a hyphen or the word to.

parse_prices <- function(data) {
  number_pattern <- "(?:\\d{1,3}(?:,\\d{3})+|\\d+)(?:\\.\\d+)?"
  amount_pattern <- paste0(
    "(?i)(?:\\$\\s*)?", number_pattern,
    "(?:\\s*(?:-|to)\\s*(?:\\$\\s*)?", number_pattern, ")?"
  )

  data |>
    mutate(
      amount_text = str_extract(text, amount_pattern),
      currency = case_when(
        str_detect(text, fixed("$")) ~ "USD",
        str_detect(str_to_lower(text), "\\bdollars?\\b") ~ "USD",
        TRUE ~ NA_character_
      ),
      unit = case_when(
        str_detect(str_to_lower(text), "per hour|/hr|/hour|an hour|hourly") ~ "hour",
        str_detect(str_to_lower(text), "a year|annually|annual|year") ~ "year",
        str_detect(str_to_lower(text), "stipend") ~ "stipend",
        TRUE ~ NA_character_
      ),
      amount_clean = str_remove_all(amount_text, "[$,]"),
      amount_lower = as.numeric(str_extract(amount_clean, "\\d+(?:\\.\\d+)?")),
      amount_upper = as.numeric(
        str_match(amount_clean, "(?:-|to)\\s*(\\d+(?:\\.\\d+)?)")[, 2]
      ),
      parser_result = case_when(
        is.na(amount_lower) ~ "not a price",
        is.na(unit) ~ "number without unit",
        !is.na(amount_upper) ~ "range",
        TRUE ~ "single price"
      )
    )
}

price_results <- parse_prices(pay_examples)

Parsed amount, currency, and unit
Example ID	Text	Matched amount	Currency	Unit	Lower amount	Upper amount	Parser result
pay-01	$18.50 per hour	$18.50	USD	hour	18.5	NA	single price
pay-02	18.50/hr	18.50	NA	hour	18.5	NA	single price
pay-03	$38,000 a year	$38,000	USD	year	38000.0	NA	single price
pay-04	$17-$19 an hour	$17-$19	USD	hour	17.0	19	range
pay-05	competitive pay	NA	NA	NA	NA	NA	not a price
pay-06	$1,250 stipend	$1,250	USD	stipend	1250.0	NA	single price
pay-07	$20 hourly	$20	USD	hour	20.0	NA	single price
pay-08	Pay starts at $16 per hour	$16	USD	hour	16.0	NA	single price
pay-09	$42,500 annually	$42,500	USD	year	42500.0	NA	single price
pay-10	Stipend of 750 dollars	750	USD	stipend	750.0	NA	single price
pay-11	Up to $21/hour	$21	USD	hour	21.0	NA	single price
pay-12	18 dollars an hour	18	USD	hour	18.0	NA	single price
pay-13	$38000 a year	$38000	USD	year	38000.0	NA	single price

The number and the unit come from different parts of the phrase. 18.50/hr contains an amount and a unit, but the rule marks the currency as missing. The rule treats every dollar sign as US dollars. That assumption fails on a board with US, Canadian, and Australian postings. Units outside this small set, such as monthly pay, require another branch.

Count the special cases

A parser should report unresolved cases. The code below counts rows that require special handling.

case_counts <- tibble(
  case = c(
    "thousands separator",
    "range with two numbers",
    "missing currency",
    "no number",
    "plain four-plus digit number"
  ),
  examples = c(
    sum(str_detect(price_results$amount_text, fixed(",")), na.rm = TRUE),
    sum(!is.na(price_results$amount_upper)),
    sum(is.na(price_results$currency)),
    sum(is.na(price_results$amount_lower)),
    sum(
      str_detect(str_remove_all(price_results$amount_text, "[$,]"), "\\d{4,}") &
        !str_detect(price_results$amount_text, fixed(",")),
      na.rm = TRUE
    )
  )
)

Cases the parser keeps visible
Case	Examples
thousands separator	3
range with two numbers	1
missing currency	2
no number	1
plain four-plus digit number	1

Three examples contain thousands separators. One is a range with two amounts. Two lack a dollar sign or dollar word. One has no number. The categories overlap, so this table is a checklist rather than a count of distinct rows.

Compare the parser with expected answers

The parser and expected answers come from the same author. A high score here does not give an estimate of accuracy. Read it only as a check that the code behaves as this page says it should.

values_match <- function(observed, expected) {
  (is.na(observed) & is.na(expected)) |
    (!is.na(observed) & !is.na(expected) & observed == expected)
}

price_score <- price_results |>
  summarise(
    examples = n(),
    lower_amount_matches = sum(values_match(amount_lower, expected_lower)),
    upper_amount_matches = sum(values_match(amount_upper, expected_upper)),
    unit_matches = sum(values_match(unit, expected_unit)),
    currency_matches = sum(values_match(currency, expected_currency))
  )

Agreement with this page’s expected answers
Examples	Lower amount matches	Upper amount matches	Unit matches	Currency matches
13	13	13	13	13

The parser matches the expected lower amount, upper amount, unit, and currency on all 13 examples. The wider number pattern prevents an incomplete match for the comma-free case. The parser must not read $38000 as $380.

Examine the Riverton stipend line

The Riverton teaching file contains a stipend phrase but no stipend amount. The rule should report a missing amount rather than invent one.

riverton_pay_line <- sentences |>
  filter(sentence_id == "s025") |>
  transmute(
    example_id = sentence_id,
    text,
    expected_currency = NA_character_,
    expected_unit = "stipend",
    expected_lower = NA_real_,
    expected_upper = NA_real_
  )

riverton_price_result <- parse_prices(riverton_pay_line)

The Riverton stipend line has a unit but no amount
Sentence ID	Text	Amount	Currency	Unit	Parser result
s025	Paid training stipend	NA	NA	stipend	not a price

The line tells a reader that a stipend exists. It does not say how much. A number without a unit is not a price. A unit without a number gives the coordinator no amount to compare.

The limits of custom rules

A custom regular expression can find a restricted set of formats. It can fail on more varied data.

failure_examples <- tibble(
  text = c("thirty-eight thousand a year", "1.250,50 / hr", "$40/hr (CAD)")
)

failure_results <- parse_prices(failure_examples)

Representative failures of the custom regex parser
text	amount_lower	currency
thirty-eight thousand a year	NA	NA
1.250,50 / hr	1.25	NA
$40/hr (CAD)	40.00	USD

Value normalization: The pattern misses "thirty-eight thousand" entirely because it only looks for digits.
Locale formats: It reads the European format 1.250,50 incorrectly. The pattern expects commas as thousands separators, so it matches 1.250 without warning and returns 1.25.
Ambiguous currencies: It assumes $ means USD, which is wrong for CAD.

Custom regular expressions alone are a poor choice for general entity parsing. Systems use named entity recognition, such as spaCy MONEY entities, to find relevant text spans. Separate value parsers, such as Duckling, normalize amounts, currencies, and locale-specific formats into structured data.

What to remember

A usable price needs an amount, a currency, and a unit.
Ranges should keep both numbers.
A parser must not return a partial number when the text contains a longer one.
These counts describe 13 teaching examples and one Riverton line.

The parser can put a pay-related mark on the stipend line. The line gives no amount for a wage comparison.

Build examples with expected answers

Extract the visible components

Count the special cases

Compare the parser with expected answers

Examine the Riverton stipend line

The limits of custom rules

What to remember

Sources