Translating a notice from English to French

Generate English-to-French translations. Examine the preserved details.

natural language generation

machine translation

evaluation

Learn how a local machine-translation checkpoint changes language while preserving the source message.

An archive sends short notices to English- and French-speaking visitors. A translation must change the language while preserving the people, quantities, dates, permissions, and limits in the source.

The model in this lesson has one direction: English to French. The checkpoint, or saved model, fixes that direction. The model does not choose a language from each input. Adding a target-language label to the output does not change the translation direction.

What you will learn

This lesson shows how to:

load a fixed revision of the English-to-French model and tokenizer locally
keep constructed source records and generated translations linked to IDs
count source tokens against the checkpoint’s 512-token input limit
compare translation with a source-copy baseline
examine names, numbers, and negation without treating these checks as adequacy scores
reserve meaning and register judgments for a bilingual reviewer.

Register is the level of formality appropriate for an audience.

Load one local translation direction

huggingfaceR::hf_load_tokenizer() and huggingfaceR::hf_load_pipeline() load the prepared local snapshot through the course helper. Offline mode prevents a render-time download. The model and tokenizer files came from the same immutable revision.

suppressPackageStartupMessages({
  library(dplyr)
  library(huggingfaceR)
  library(jsonlite)
  library(purrr)
  library(reticulate)
  library(stringr)
  library(tibble)
  library(tidyr)
})

source("R/use-nlg.R")

translation_model <- load_nlg_pipeline(
  "opus_en_fr",
  "translation"
)

model_record <- translation_model$metadata |>
  select(model_id, revision, license, task_scope)

Pinned local checkpoint for English-to-French translation
Model	Revision	License	Course task
Helsinki-NLP/opus-mt-en-fr	dd7f6540a7a48a7f4db59e5c0b9c42c8eea67f18	Apache-2.0	machine translation

The package also exports hf_translate(), which calls a remote inference service. This lesson does not use that service. Generation uses the local model object from the fixed pipeline.

Read direction from the checkpoint

The local tokenizer configuration names English as the source and French as the target. A different direction requires a different checkpoint.

tokenizer_config_path <- file.path(
  "data-raw",
  ".cache",
  "nlg-models",
  translation_model$metadata$local_directory,
  "tokenizer_config.json"
)
generation_config_path <- file.path(
  "data-raw",
  ".cache",
  "nlg-models",
  translation_model$metadata$local_directory,
  "generation_config.json"
)

tokenizer_config <- fromJSON(tokenizer_config_path)
generation_config <- fromJSON(generation_config_path)

direction <- tibble(
  checkpoint = translation_model$metadata$model_id,
  source_language = tokenizer_config$source_lang,
  target_language = tokenizer_config$target_lang,
  runtime_language_switch = FALSE,
  checkpoint_beams = generation_config$num_beams
)

Translation direction recorded in the tokenizer configuration
Checkpoint	Source	Target	Runtime direction switch	Checkpoint beams
Helsinki-NLP/opus-mt-en-fr	en	fr	FALSE	4

Prepare constructed notices

The author wrote these four English notices for the lesson. They are demonstrations, not human reference translations, a held-out test set, or a population quality benchmark.

translation_inputs <- tribble(
  ~source_id, ~source_text, ~protected_name, ~requires_negation,
  "MT-01",
  "Maya Chen will open Room 5 on 14 September.",
  "Maya Chen",
  FALSE,
  "MT-02",
  "The library did not cancel the workshop.",
  "",
  TRUE,
  "MT-03",
  "Visitors must bring 2 forms.",
  "",
  FALSE,
  "MT-04",
  "The volunteers may use 12 computers on Tuesday.",
  "",
  FALSE
)

Constructed records for the translation demonstration
Source ID	Constructed English source	Protected name	Negation required
MT-01	Maya Chen will open Room 5 on 14 September.	Maya Chen	FALSE
MT-02	The library did not cancel the workshop.		TRUE
MT-03	Visitors must bring 2 forms.		FALSE
MT-04	The volunteers may use 12 computers on Tuesday.		FALSE

Count tokens before calling the model

Marian tokenizes words into subword units, which can represent parts of words. Its configured input limit is 512 tokens. This lesson disables truncation because removing the end of a notice can remove essential meaning.

translation_budget <- translation_inputs |>
  mutate(
    input_tokens = map_int(
      source_text,
      \(text) nlg_token_count(translation_model$tokenizer, text)
    ),
    model_input_limit = 512L,
    fits_without_truncation = input_tokens <= model_input_limit
  )

The workflow compares input counts with the limit before translation
Source ID	Tokenizer units	Input limit	Fits without truncation
MT-01	12	512	TRUE
MT-02	9	512	TRUE
MT-03	7	512	TRUE
MT-04	10	512	TRUE

For a longer source, setting truncation = TRUE would discard source tokens. This setting would cause data loss. A production workflow would divide the document into sections or stop for human checks.

Generate French candidates

The source text goes directly to the encoder. The checkpoint uses four-beam generation by default, which compares four candidate sequences. The course helper instead uses num_beams = 1 and do_sample = FALSE. This deterministic greedy choice selects the highest-scoring next token and keeps the teaching workflow consistent. It does not imply that the four-beam setting is nondeterministic or inferior.

The output limit is 80 newly generated tokens. The returned record shows whether the decoder emitted its end-of-sequence token, or EOS. A separate field records whether generation used the full output allowance. Both fields can be TRUE when the checkpoint forces EOS at the limit. Neither field proves that the translation is complete for a reader.

translation_results <- translation_budget |>
  mutate(
    generation = map(
      source_text,
      \(text) {
        nlg_generate(
          translation_model,
          text,
          max_new_tokens = 80L
        )
      }
    ),
    french_candidate = map_chr(generation, "text"),
    output_tokens = map_int(generation, "output_tokens"),
    ended_by_eos = map_lgl(generation, "ended_by_eos"),
    hit_token_cap = map_lgl(generation, "hit_token_cap"),
    source_copy_baseline = source_text
  )

Real local translations beside a no-translation baseline
Source ID	English source	Generated French candidate	Untranslated copy baseline
MT-01	Maya Chen will open Room 5 on 14 September.	Maya Chen ouvrira la salle 5 le 14 septembre.	Maya Chen will open Room 5 on 14 September.
MT-02	The library did not cancel the workshop.	La bibliothèque n’a pas annulé l’atelier.	The library did not cancel the workshop.
MT-03	Visitors must bring 2 forms.	Les visiteurs doivent apporter 2 formulaires.	Visitors must bring 2 forms.
MT-04	The volunteers may use 12 computers on Tuesday.	Les volontaires peuvent utiliser 12 ordinateurs mardi.	The volunteers may use 12 computers on Tuesday.

The copy baseline preserves the English source exactly and performs no translation. A changed string alone does not establish a successful translation.

Examine visible details

The next checks compare exact Arabic numerals and search for the protected person name. They also search for common French negation markers when the English source contains did not. These checks can find some clear failures. They cannot establish correct grammar or preservation of the meanings of must and may. Nor can they establish an appropriate register for the audience.

extract_numbers <- function(text) {
  str_extract_all(text, "\\b\\d+\\b")[[1]] |>
    sort()
}

translation_checks <- translation_results |>
  mutate(
    numerals_preserved = map2_lgl(
      source_text,
      french_candidate,
      \(source, output) {
        identical(
          extract_numbers(source),
          extract_numbers(output)
        )
      }
    ),
    name_present = map2_lgl(
      protected_name,
      french_candidate,
      \(name, output) {
        if (!nzchar(name)) {
          NA
        } else {
          str_detect(output, fixed(name))
        }
      }
    ),
    negation_visible = map2_lgl(
      requires_negation,
      french_candidate,
      \(required, output) {
        if (!required) {
          NA
        } else {
          str_detect(
            output,
            regex("\\b(pas|jamais|aucun)\\b", ignore_case = TRUE)
          )
        }
      }
    )
  ) |>
  select(
    source_id,
    numerals_preserved,
    name_present,
    negation_visible
  )

translation_check_display <- translation_checks |>
  mutate(
    across(
      c(numerals_preserved, name_present, negation_visible),
      nlg_screen_label
    )
  )

Conservative automatic surface checks for the constructed notices
Source ID	Arabic-numeral screen	Protected-name screen	French-negation screen
MT-01	not flagged by screen	not flagged by screen	not applicable
MT-02	not flagged by screen	not applicable	not flagged by screen
MT-03	not flagged by screen	not applicable	not applicable
MT-04	not flagged by screen	not applicable	not applicable

The lesson reports no BLEU or chrF number. These translation-overlap metrics require a documented reference corpus, preprocessing signature, and corpus-level interpretation. Even then, one reference cannot represent every acceptable French rendering, and automatic overlap does not replace bilingual judgment.

Not flagged by screen means only that this finite string check found no problem. It does not establish translation adequacy. Not applicable means the source contained no protected name or negation condition to examine.

Leave approval to a bilingual reader

translation_screen_counts <- translation_checks |>
  rowwise() |>
  mutate(
    applicable_screens = sum(!is.na(c_across(c(
      numerals_preserved,
      name_present,
      negation_visible
    )))),
    flagged_screens = sum(c_across(c(
      numerals_preserved,
      name_present,
      negation_visible
    )) %in% FALSE)
  ) |>
  ungroup()

translation_review <- tibble(
  review = c(
    "local model execution",
    "source within the 512-token input limit",
    "decoder stopped at EOS or the output cap",
    "surface preservation checks",
    "bilingual meaning and register checks"
  ),
  status = c(
    sprintf(
      "completed: %d candidates",
      sum(nzchar(translation_results$french_candidate))
    ),
    sprintf(
      "within budget: %d/%d candidates",
      sum(translation_budget$fits_without_truncation),
      nrow(translation_budget)
    ),
    sprintf(
      "recorded: %d/%d candidates",
      sum(
        translation_results$ended_by_eos |
          translation_results$hit_token_cap
      ),
      nrow(translation_results)
    ),
    sprintf(
      "scan completed: %d flags across %d applicable checks",
      sum(translation_screen_counts$flagged_screens),
      sum(translation_screen_counts$applicable_screens)
    ),
    "pending"
  )
)

Automatic checks do not grant bilingual approval
Check stage	Status
local model execution	completed: 4 candidates
source within the 512-token input limit	within budget: 4/4 candidates
decoder stopped at EOS or the output cap	recorded: 4/4 candidates
surface preservation checks	scan completed: 0 flags across 6 applicable checks
bilingual meaning and register checks	pending

The lesson contains no human ratings. A bilingual reviewer still has to compare each pair for meaning, tone, register, and natural French.

What to remember

Translation changes language while trying to preserve the source message.
This checkpoint’s English-to-French direction is fixed in its model files.
Count model tokens before generation.
Treat truncation as data loss.
Keep an untranslated copy baseline beside generated candidates.
Names, numbers, and negation checks can reveal errors but cannot prove adequacy.
Human bilingual checks remain necessary before publication.