Skip to main content

Building a document one section at a time

Separate evidence, continuity, generation, and publication approval

Learn a three-section local generation workflow with explicit evidence limits, token budgets, continuity state, and publication decisions.

Kai needs a short public update about a fictional seed-sharing pilot. A request to “write the whole report” combines planning, evidence selection, continuity, and prose generation. One model call would obscure those separate decisions.

This lesson separates those jobs. A human-written plan assigns evidence to three sections. The local model generates each section. Each section receives compact continuity state that matches recorded evidence. This state summarizes relevant evidence without copying the whole draft into the next prompt.

TipWhat you will learn

This lesson shows how to:

• plan a three-section document before generating prose
• give each section an evidence ledger and output limit
• account for prompt and output tokens in a decoder-only context window
• pass compact continuity state from assigned evidence between section calls
• distinguish end-of-sequence stopping from a token-limit stop
• examine repetition, numbers without evidence, and evidence coverage
• reject the assembled draft for publication until human checks are complete.

Load the local causal model

The course helper loads a prepared local snapshot through huggingfaceR. The model is a decoder-only causal language model. It predicts each new token from preceding tokens. Prompt tokens and newly generated tokens share the same context window.

suppressPackageStartupMessages({
  library(dplyr)
  library(huggingfaceR)
  library(jsonlite)
  library(purrr)
  library(reticulate)
  library(stringr)
  library(tibble)
  library(tidyr)
})

source("R/use-nlg.R")

long_model <- load_nlg_pipeline(
  "qwen_1_5b_instruct",
  "text-generation"
)

model_config_path <- file.path(
  "data-raw",
  ".cache",
  "nlg-models",
  long_model$metadata$local_directory,
  "config.json"
)
model_config <- fromJSON(model_config_path)
context_limit <- as.integer(model_config$max_position_embeddings)

model_record <- long_model$metadata |>
  transmute(
    model = model_id,
    revision,
    license,
    architecture = "decoder-only causal language model",
    context_limit
  )

Pinned model and its shared causal context window

Model	Revision	License	Architecture	Context limit
Qwen/Qwen2.5-1.5B-Instruct	989aa7980e4cf806f80c7fef2b1adb7bc71aa306	Apache-2.0	decoder-only causal language model	32768

For this architecture, enforce prompt_tokens + max_new_tokens <= context_limit.

An encoder-decoder model has separate encoder-input and decoder-output limits. That distinction matters when moving the workflow to another checkpoint.

Plan before writing

The evidence below is fictional demonstration material. It is not a survey estimate, a program evaluation, or a claim about a real seed library.

Each evidence row receives an ID. The plan assigns those IDs to one section and states the section’s purpose before prose generation.

evidence_ledger <- tribble(
  ~evidence_id, ~evidence,
  "E01", "The pilot registered 48 residents.",
  "E02", "The pilot completed 36 pickup appointments in April.",
  "E03", "12 volunteers staffed the collection table.",
  "E04", "Volunteers did not guarantee that any seed would grow.",
  "E05", "29 participants requested an evening pickup.",
  "E06",
  paste(
    "Staff will decide whether to add one Thursday evening;",
    "no decision has been made."
  )
)

document_plan <- tribble(
  ~section_id, ~heading, ~purpose, ~evidence_ids, ~max_new_tokens,
  ~continuity_in, ~continuity_out,
  "SEC-01",
  "Pilot activity",
  "Describe participation, completed pickups, and staffing.",
  c("E01", "E02", "E03"),
  120L,
  "This is the opening section.",
  paste(
    "The pilot registered 48 residents, completed 36 April pickups,",
    "and used 12 volunteers."
  ),
  "SEC-02",
  "Participant request",
  "Describe the request for an evening pickup without claiming a decision.",
  c("E05"),
  100L,
  paste(
    "The pilot registered 48 residents, completed 36 April pickups,",
    "and used 12 volunteers."
  ),
  "Twenty-nine participants requested an evening pickup.",
  "SEC-03",
  "Decision and limit",
  "State the pending Thursday decision and the volunteer guarantee limit.",
  c("E04", "E06"),
  120L,
  "Twenty-nine participants requested an evening pickup.",
  paste(
    "A Thursday evening remains undecided, and volunteers did not",
    "guarantee seed growth."
  )
)

A three-section document plan fixes evidence limits before generation

Section ID	Heading	Section purpose	Evidence IDs	Output budget
SEC-01	Pilot activity	Describe participation, completed pickups, and staffing.	E01, E02, E03	120
SEC-02	Participant request	Describe the request for an evening pickup without claiming a decision.	E05	100
SEC-03	Decision and limit	State the pending Thursday decision and the volunteer guarantee limit.	E04, E06	120

The continuity state restates only assigned evidence. It is not a model-generated summary of the previous section. Thus, a generated detail without source evidence cannot become accepted context for the next section.

The context-only policy requires more than evidence for a fact. Incoming continuity can guide transitions between sections. However, a section can restate a continuity fact only if its assigned evidence also contains that fact. Before generation, the author defines the E05 pattern below to examine that policy for SEC-03.

continuity_patterns <- tribble(
  ~section_id, ~context_evidence_id, ~pattern,
  "SEC-03",
  "E05",
  paste(
    "\\b(29|twenty-nine|twenty nine)\\b.*\\bevening",
    "|\\bevening\\b.*\\b(29|twenty-nine|twenty nine)\\b"
  )
)

Predeclared cross-section restatement check

Section examined	Context-only evidence
SEC-03	E05

Build section prompts and budgets

Each prompt contains one section purpose, only its assigned evidence, and the compact incoming state. The workflow does not append the previous model prose.

long_system <- paste(
  "Write one section of a public program update in 2 or 3 sentences.",
  "Use only the listed evidence.",
  "Do not add examples, causes, recommendations, people, places, or outcomes.",
  "Do not write a heading."
)

section_requests <- document_plan |>
  mutate(
    scoped_evidence = map(
      evidence_ids,
      \(ids) {
        evidence_ledger |>
          filter(evidence_id %in% ids) |>
          transmute(line = paste0(evidence_id, ": ", evidence)) |>
          pull(line) |>
          paste(collapse = "\n")
      }
    ),
    user_prompt = pmap_chr(
      list(section_id, purpose, continuity_in, scoped_evidence),
      \(section_id, purpose, continuity_in, scoped_evidence) {
        paste0(
          "SECTION ID: ", section_id,
          "\nPURPOSE: ", purpose,
          "\nCONTINUITY STATE: ", continuity_in,
          "\nSCOPED EVIDENCE:\n", scoped_evidence
        )
      }
    ),
    model_prompt = map_chr(
      user_prompt,
      \(user) {
        nlg_chat_prompt(
          long_model$tokenizer,
          long_system,
          user
        )
      }
    ),
    prompt_tokens = map_int(
      model_prompt,
      \(prompt) nlg_token_count(long_model$tokenizer, prompt)
    ),
    planned_total_tokens = prompt_tokens + max_new_tokens,
    fits_context = planned_total_tokens <= context_limit
  )

Decoder-only token accounting for every section call

Section ID	Prompt tokens	Maximum new tokens	Planned total	Fits context
SEC-01	136	120	256	TRUE
SEC-02	127	100	227	TRUE
SEC-03	133	120	253	TRUE

Generate all three sections

The loop below makes three separate neural generation calls. Greedy decoding selects the highest-scoring next token and uses no random sampling. Small numerical differences between machines can still change a selection. Research documents repetitive output as a risk of likelihood-maximizing decoding in open-ended tasks. This observed tendency does not mean every greedy output contains repetition.

section_generations <- section_requests |>
  mutate(
    generation = map2(
      model_prompt,
      max_new_tokens,
      \(prompt, budget) {
        nlg_generate(
          long_model,
          prompt,
          max_new_tokens = budget,
          repetition_penalty = 1.05
        )
      }
    ),
    generated_text = map_chr(generation, "text"),
    output_tokens = map_int(generation, "output_tokens"),
    ended_by_eos = map_lgl(generation, "ended_by_eos"),
    hit_token_cap = map_lgl(generation, "hit_token_cap")
  )

Three separately generated sections

Section ID	Heading	Generated section
SEC-01	Pilot activity	The pilot successfully registered 48 residents, completed 36 pickup appointments in April, and was staffed by 12 volunteers at the collection table.
SEC-02	Participant request	Residents have expressed interest in an evening pickup service, with 29 participants requesting this option.
SEC-03	Decision and limit	The upcoming Thursday decision involves a volunteer guarantee limit, with twenty-nine participants requesting an evening pickup. The staff will determine whether to add one Thursday evening, but no decision has been finalized as of now.

Examine architectural stopping

A final period is a surface character. It does not reveal why generation stopped. The helper compares the final generated token ID with the model’s end-of-sequence, or EOS, configuration. It separately records whether the output reached the token limit.

stopping_diagnostics <- section_generations |>
  transmute(
    section_id,
    output_tokens,
    output_budget = max_new_tokens,
    ended_by_eos,
    hit_token_cap,
    ends_with_punctuation = str_detect(
      str_trim(generated_text),
      "[.!?]$"
    )
  )

Token-level stopping evidence beside a surface heuristic

Section ID	Output tokens	Output budget	EOS emitted	Hit token cap	Ends with punctuation
SEC-01	33	120	TRUE	FALSE	TRUE
SEC-02	20	100	TRUE	FALSE	TRUE
SEC-03	41	120	TRUE	FALSE	TRUE

EOS says the model selected a termination token. It does not establish that the section is complete, accurate, or ready to publish. The EOS and token-limit flags describe separate conditions. Both may be TRUE when a model forces EOS on the last permitted new token.

Examine repetition and section-specific evidence

The calculation converts text to lowercase and extracts letter and number sequences. The distinct trigram ratio divides unique three-token sequences by all three-token sequences. Values nearer 1 show fewer exact repeated trigrams. The lesson claims no universal publication threshold.

The evidence flags are simpler: each planned fact has a surface pattern. A missing pattern is a reason to revise. A present pattern cannot prove that the fact has the correct relation or limits.

distinct_ngram_ratio <- function(text, n = 3L) {
  tokens <- str_extract_all(
    str_to_lower(text, locale = "en"),
    "[\\p{L}\\p{N}]+"
  )[[1]]

  if (length(tokens) < n) {
    return(1)
  }

  starts <- seq_len(length(tokens) - n + 1L)
  ngrams <- map_chr(
    starts,
    \(start) paste(tokens[start:(start + n - 1L)], collapse = " ")
  )
  length(unique(ngrams)) / length(ngrams)
}

evidence_patterns <- tribble(
  ~evidence_id, ~pattern,
  "E01", "\\b48\\b",
  "E02", "\\b36\\b.*\\bApril\\b|\\bApril\\b.*\\b36\\b",
  "E03", "\\b(12|twelve)\\b.*\\bvolunteer",
  "E04", "\\b(not|no|never|without|did not)\\b.*\\bguarantee",
  "E05",
  paste(
    "\\b(29|twenty-nine|twenty nine)\\b.*\\bevening",
    "|\\bevening\\b.*\\b(29|twenty-nine|twenty nine)\\b"
  ),
  "E06", "\\bThursday\\b.*\\b(no|not|undecided|pending)"
)

section_diagnostics <- section_generations |>
  transmute(
    section_id,
    distinct_trigram_ratio = map_dbl(
      generated_text,
      distinct_ngram_ratio
    ),
    source_numbers = map(
      scoped_evidence,
      \(text) sort(str_extract_all(text, "\\b\\d+\\b")[[1]])
    ),
    output_numbers = map(
      generated_text,
      \(text) sort(str_extract_all(text, "\\b\\d+\\b")[[1]])
    ),
    unsupported_number = map2_lgl(
      source_numbers,
      output_numbers,
      \(source, output) !all(output %in% source)
    ),
    unsupported_evaluation = str_detect(
      generated_text,
      regex(
        "\\b(successfully|effective|improved|successful)\\b",
        ignore_case = TRUE
      )
    )
  )

coverage_diagnostics <- document_plan |>
  select(section_id, evidence_ids) |>
  unnest_longer(evidence_ids, values_to = "evidence_id") |>
  left_join(evidence_patterns, by = "evidence_id") |>
  left_join(
    section_generations |>
      select(section_id, generated_text),
    by = "section_id"
  ) |>
  mutate(
    surface_flag = str_detect(
      generated_text,
      regex(pattern, ignore_case = TRUE)
    )
  ) |>
  select(section_id, evidence_id, surface_flag)

continuity_diagnostics <- continuity_patterns |>
  left_join(
    section_generations |>
      select(section_id, generated_text),
    by = "section_id"
  ) |>
  mutate(
    restated_outside_scope = str_detect(
      generated_text,
      regex(pattern, ignore_case = TRUE)
    )
  ) |>
  select(
    section_id,
    context_evidence_id,
    restated_outside_scope
  )

Repetition and numeral diagnostics

Section ID	Distinct trigram ratio	Numeral without evidence present	Evaluative wording without evidence
SEC-01	1	FALSE	TRUE
SEC-02	1	FALSE	FALSE
SEC-03	1	FALSE	FALSE

Evidence coverage flags for each generated section

Section ID	Evidence ID	Surface flag present
SEC-01	E01	TRUE
SEC-01	E02	TRUE
SEC-01	E03	TRUE
SEC-02	E05	TRUE
SEC-03	E04	FALSE
SEC-03	E06	TRUE

Continuity-fact check against assigned section limits

Section ID	Context-only evidence	Restated outside assigned scope
SEC-03	E05	TRUE

These are limited structural checks. They do not detect an invented plant example, a cause absent from the source, a changed obligation, or a contradiction without a new numeral.

Compare each generated section with the ledger and its assigned evidence. In SEC-01, successfully would add an evaluation absent from the ledger. SEC-03 must retain the statement that volunteers made no guarantee about seed growth. Restating the E05 evening request in SEC-03 would violate the section limits because E05 belongs to SEC-02.

The prompt supplies the number 29 in continuity context, so that number alone is not evidence of invention. Using it to restate E05 in SEC-03 would still require draft revision.

Assemble the draft and make a decision

The headings come from the human plan. The prose under each heading comes from the three model calls above.

assembled_document <- section_generations |>
  transmute(
    section_id,
    section = paste0("## ", heading, "\n\n", generated_text)
  ) |>
  pull(section) |>
  paste(collapse = "\n\n")

cat(assembled_document)

## Pilot activity

The pilot successfully registered 48 residents, completed 36 pickup appointments in April, and was staffed by 12 volunteers at the collection table.

## Participant request

Residents have expressed interest in an evening pickup service, with 29 participants requesting this option.

## Decision and limit

The upcoming Thursday decision involves a volunteer guarantee limit, with twenty-nine participants requesting an evening pickup. The staff will determine whether to add one Thursday evening, but no decision has been finalized as of now.

The automatic findings show whether the draft already needs revision. They still cannot authorize publication. The publication decision remains reject until a person examines every generated claim, continuity across sections, audience, and tone.

automatic_revision_needed <- any(
  stopping_diagnostics$hit_token_cap,
  section_diagnostics$unsupported_number,
  section_diagnostics$unsupported_evaluation,
  !coverage_diagnostics$surface_flag,
  continuity_diagnostics$restated_outside_scope
)

held_sections <- unique(c(
  stopping_diagnostics$section_id[
    stopping_diagnostics$hit_token_cap
  ],
  section_diagnostics$section_id[
    section_diagnostics$unsupported_number |
      section_diagnostics$unsupported_evaluation
  ],
  coverage_diagnostics$section_id[
    !coverage_diagnostics$surface_flag
  ],
  continuity_diagnostics$section_id[
    continuity_diagnostics$restated_outside_scope
  ]
))

long_review <- tibble(
  review = c(
    "local section generation",
    "prompt plus output budget",
    "token-level stopping diagnostics",
    "automatic evidence and continuity screens",
    "human checks of facts, continuity, tone, and audience"
  ),
  status = c(
    sprintf(
      "completed: %d sections",
      sum(nzchar(section_generations$generated_text))
    ),
    sprintf(
      "within budget: %d/%d sections",
      sum(section_requests$fits_context),
      nrow(section_requests)
    ),
    sprintf(
      "recorded: %d/%d sections",
      sum(
        stopping_diagnostics$ended_by_eos |
          stopping_diagnostics$hit_token_cap
      ),
      nrow(stopping_diagnostics)
    ),
    sprintf(
      "held: %d/%d sections flagged",
      length(held_sections),
      nrow(section_generations)
    ),
    "pending"
  )
)

document_decision <- tibble(
  automated_recommendation = if_else(
    automatic_revision_needed,
    "revise before human review",
    "send to human review"
  ),
  publication_decision = "reject",
  reason = paste(
    "Human checks of facts, continuity, tone, and audience are pending.",
    "Automatic checks cannot approve the document."
  )
)

Completed scans do not approve the generated sections

Check stage	Status
local section generation	completed: 3 sections
prompt plus output budget	within budget: 3/3 sections
token-level stopping diagnostics	recorded: 3/3 sections
automatic evidence and continuity screens	held: 2/3 sections flagged
human checks of facts, continuity, tone, and audience	pending

The generated document is not approved for publication

Automated recommendation	Publication decision	Reason
revise before human review	reject	Human checks of facts, continuity, tone, and audience are pending. Automatic checks cannot approve the document.

A future editor may revise and accept the document. This render records no such approval.

What to remember

• Long-text generation starts with document planning and evidence limits for each section.
• Decoder-only prompts and outputs share one context window.
• Pass compact continuity state from assigned evidence instead of unbounded raw history.
• Examine EOS and token-limit status directly. Punctuation alone does not explain stopping.
• Repetition, numeral, and evidence flags are preliminary filters.
• Reject generated prose for publication until a human editor examines the complete document.

Sources

• Reiter and Dale on document planning in applied NLG
• Holtzman et al. on neural text degeneration
• Fan et al. on hierarchical long-form generation
• See et al. on multi-paragraph generation and discourse failures
• Transformers text-generation controls
• Qwen2.5-1.5B-Instruct model card