Span-preserving parse & strictness

You write colmn: when you meant column:. What should happen? A lenient parser shrugs, ignores the key it doesn’t recognise, and runs your pipeline with a silently-missing setting, a bug you discover hours later in the output. Clinker takes the opposite stance: that typo is an error, raised at plan time, pointing at the exact line. Two design choices make that possible, and both are worth stealing.

Prerequisites

• Proof tokens

→

Leads to

• The plan/runtime boundary

What you’ll be able to do

• Read the Spanned<T> wrapper and name what it carries beyond the value itself.
• Explain what #[serde(deny_unknown_fields)] changes about parsing a config, and why a silent-ignore default is dangerous.
• Write a strict, span-aware config reader that rejects an unknown key with a located error message.
• Distinguish span-tracking (locating a value) from strictness (rejecting an unknown key), and name what the single from_str chokepoint buys on top of both.

New terms in this lesson (each is also expanded inline at the point you first need it):

• span: the recorded source location of a value.
• Spanned<T>: a value paired with its span.
• source position: a line/column coordinate into the source text.
• diagnostic: an error rendered against its source span.

Predict first

✋ Predict first

A config struct is declared #[serde(deny_unknown_fields)] with one field, column: String. You feed it YAML that has the line colmn: id (a typo) plus a correct column: id. What does parsing return: the value with the typo silently dropped, or an error? And separately: does plain serde even know which LINE the bad key sat on? Write yes/no for each before revealing.

Decide on your answer before you open the reveal — committing a guess is what makes it stick.

Reveal the answer

Parsing returns an error, not a value, and that is the whole point of deny_unknown_fields. Without it, serde’s default is to ignore colmn: and happily build the struct from column:, leaving you with a config that parsed “successfully” while quietly discarding a key you thought mattered.

On the second question: plain serde, by itself, throws away where each value sat. To answer “which line?” you need to parse into a Spanned<T>, which keeps the value and its span.

The trap is assuming these are one feature. They are two independent choices: strictness (reject unknown keys) decides whether a typo fails; span-tracking decides whether the failure can point at the line. You can have either without the other. Clinker insists on both, behind one door. Hold that split; the rest of the lesson earns it.

Choice one: every value remembers where it came from

When serde turns YAML into a Rust struct, it normally throws away where each value sat in the file. That’s fine until you need to say “the problem is here.” So clinker parses into Spanned<T>, a wrapper that keeps the value and its source position:

clinker-plan ·yaml.rs ·Spanned type @19acdcb4

// re-exported by clinker from the serde-saphyr crate
pub struct Spanned<T> {
    pub value: T,
    pub referenced: Location,   // where this value is used in the source
    pub defined: Location,      // where it was defined (e.g. a YAML anchor)
}

A Spanned<PipelineNode> is a pipeline node that still knows its line and column. The engine holds the whole pipeline as Vec<Spanned<PipelineNode>> precisely so that any later complaint (a bad reference, a security rejection, a type error) can be rendered as a diagnostic that points at the source, the way a good compiler underlines the offending token rather than saying “error somewhere in your file.”

This is the same philosophy as last lesson’s ValidatedPath: carry the extra fact in the type rather than recomputing or guessing it later. There the fact was “screened”; here it’s “came from line N.” It also rhymes with the RecordView<'a, S> you met back in Reading a field without copying: a Spanned<T> is a value bundled with a fact about where it lives: there a lifetime tying a borrow to its storage, here a span tying a value to its source line.

Evidence note

Spanned, Location, and Span are defined in the serde-saphyr dependency and re-exported from crates/clinker-plan/src/yaml.rs; they aren’t clinker’s own types. What’s clinker’s is the decision to parse into them everywhere.

Choice two: unknown fields are rejected, not ignored

The strictness lives on the config structs themselves, via a serde attribute:

clinker-plan ·source.rs ·deny_unknown_fields doc @19acdcb4

#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
pub struct WatermarkConfig {
    pub column: String,
    // ... a stray `colmn:` key here is a PARSE ERROR, not a silent no-op
}

#[serde(deny_unknown_fields)] flips serde from “ignore keys I don’t recognise” to “refuse any key I don’t recognise.” That single attribute is what converts your colmn: typo from a silent misconfiguration into a loud, located failure before the pipeline runs. The attribute appears on config structs across the crate; it’s a house rule, not a one-off.

Notice these two choices are orthogonal: deny_unknown_fields decides whether the typo is an error; Spanned<T> decides whether the error can name the line. The next section builds both, in miniature, so you can see the seam between them.

Choice three: one chokepoint for all of it

Strictness and span-tracking only hold if nothing sneaks around them. So clinker funnels all YAML parsing through a single function. No other code is permitted to call the underlying parser directly:

clinker-plan ·yaml.rs ·from_str fn @19acdcb4

pub fn from_str<'de, T>(yaml: &'de str) -> Result<T, YamlError>
where
    T: Deserialize<'de>,
{
    if yaml.len() > MAX_INPUT_BYTES {
        // pre-parse rejection — cheap, before the parser sees a huge input
        return Err(YamlError(make_oversize_error(yaml.len())));
    }
    serde_saphyr::from_str_with_options(yaml, budget_options()).map_err(YamlError)
}

The module doc states the rule plainly: “This module is the single entry point for YAML parsing in clinker. No other code path is permitted to call serde_saphyr::from_str*.” Why insist on one door? Two reasons, both architectural:

• Defence in depth. budget_options() caps input size (32 MB), nesting depth (256), and node count (100 000), disables !include, and enforces an alias/anchor ratio against “billion laughs” expansion attacks. A single chokepoint means those limits apply to every parse, with no forgotten code path that parses unbounded input.
• Bus-factor containment. serde-saphyr is a pre-1.0, single-maintainer dependency. Routing every call through one wrapper means that if it ever needs replacing, there’s exactly one file to change, not a hundred call sites scattered across the crate.

A chokepoint is the parser-side cousin of the proof token: instead of trusting every caller to remember the limits, you make the one gate enforce them for everyone.

Worked → completion → faded

No serde needed to feel the two choices. You’ll build a tiny config reader that is both strict (rejects unknown keys) and span-aware (knows the line), using only std: first fully worked, then completing one gap, then writing the whole thing.

Worked: strict and located, end to end

Here is the full reader. Run it: the good config prints each field with its remembered line, and the typo is rejected with a located message. Read the two marked branches; they are the entire difference between strictness and leniency.

rust // editable

reset ▶ run

struct Spanned { value: String, line: usize }

fn parse_strict(input: &str) -> Result<Vec<(String, Spanned)>, String> {
  let allowed = ["source", "column", "format"];
  let mut out = Vec::new();
  for (i, raw) in input.lines().enumerate() {
      let line = i + 1; // 1-based, like an editor numbers lines
      let (key, val) = raw
          .split_once(':')
          .ok_or_else(|| format!("line {line}: expected `key: value`"))?;
      let key = key.trim();
      if !allowed.contains(&key) {
          // STRICT branch: an unknown key is an error — and we know exactly where.
          // Swap this for `continue` and you've built the LENIENT parser instead.
          return Err(format!("line {line}: unknown field `{key}`"));
      }
      // pair the value with its remembered source line: that's a Spanned
      out.push((key.to_string(), Spanned { value: val.trim().into(), line }));
  }
  Ok(out)
}

fn main() {
  let good = "source: customers.csv\ncolumn: id\nformat: csv";
  match parse_strict(good) {
      Ok(fields) => {
          for (key, span) in &fields {
              // read the value AND its remembered source line
              println!("line {}: {} = {}", span.line, key, span.value);
          }
      }
      Err(e) => println!("rejected: {e}"),
  }

  // Typo: `colmn` instead of `column`.
  let typo = "source: customers.csv\ncolmn: id";
  match parse_strict(typo) {
      Ok(_) => println!("accepted (a lenient parser would do this — silently wrong)"),
      Err(e) => println!("rejected: {e}"),
  }
}

struct Spanned { value: String, line: usize } fn parse_strict(input: &str) -> Result<Vec<(String, Spanned)>, String> { let allowed = ["source", "column", "format"]; let mut out = Vec::new(); for (i, raw) in input.lines().enumerate() { let line = i + 1; // 1-based, like an editor numbers lines let (key, val) = raw .split_once(':') .ok_or_else(|| format!("line {line}: expected `key: value`"))?; let key = key.trim(); if !allowed.contains(&key) { // STRICT branch: an unknown key is an error — and we know exactly where. // Swap this for `continue` and you've built the LENIENT parser instead. return Err(format!("line {line}: unknown field `{key}`")); } // pair the value with its remembered source line: that's a Spanned out.push((key.to_string(), Spanned { value: val.trim().into(), line })); } Ok(out) } fn main() { let good = "source: customers.csv\ncolumn: id\nformat: csv"; match parse_strict(good) { Ok(fields) => { for (key, span) in &fields { // read the value AND its remembered source line println!("line {}: {} = {}", span.line, key, span.value); } } Err(e) => println!("rejected: {e}"), } // Typo: `colmn` instead of `column`. let typo = "source: customers.csv\ncolmn: id"; match parse_strict(typo) { Ok(_) => println!("accepted (a lenient parser would do this — silently wrong)"), Err(e) => println!("rejected: {e}"), } }

The good config parses; the typo is rejected with line 2: unknown field \colmn`. That one if !allowed.contains(…)` branch is the whole difference between “fails loudly at plan time, here” and “fails mysteriously at run time, somewhere.”

Completion: render the located error

Below, the parse loop already builds a Spanned and detects the unknown key, but the error string is left as a TODO. The reader has the line number (line) and the offending key (key) in scope. Fill in the format! so the message points at the source, the way a diagnostic should. Everything else runs as given.

rust // editable

reset ▶ run

struct Spanned { value: String, line: usize }

fn parse_strict(input: &str) -> Result<Vec<(String, Spanned)>, String> {
  let allowed = ["source", "column", "format"];
  let mut out = Vec::new();
  for (i, raw) in input.lines().enumerate() {
      let line = i + 1;
      let (key, val) = raw
          .split_once(':')
          .ok_or_else(|| format!("line {line}: expected key: value"))?;
      let key = key.trim();
      if !allowed.contains(&key) {
          // The value's span (line) and the bad key are both in scope.
          // Make the error point at the exact spot.
          return Err(format!("TODO: a LOCATED unknown-field error")); // <- complete this
      }
      out.push((key.to_string(), Spanned { value: val.trim().into(), line }));
  }
  Ok(out)
}

fn main() {
  let typo = "source: customers.csv\ncolmn: id";
  match parse_strict(typo) {
      Ok(_) => println!("accepted (silently wrong)"),
      Err(e) => println!("rejected: {e}"),
  }
}

struct Spanned { value: String, line: usize } fn parse_strict(input: &str) -> Result<Vec<(String, Spanned)>, String> { let allowed = ["source", "column", "format"]; let mut out = Vec::new(); for (i, raw) in input.lines().enumerate() { let line = i + 1; let (key, val) = raw .split_once(':') .ok_or_else(|| format!("line {line}: expected key: value"))?; let key = key.trim(); if !allowed.contains(&key) { // The value's span (line) and the bad key are both in scope. // Make the error point at the exact spot. return Err(format!("TODO: a LOCATED unknown-field error")); // <- complete this } out.push((key.to_string(), Spanned { value: val.trim().into(), line })); } Ok(out) } fn main() { let typo = "source: customers.csv\ncolmn: id"; match parse_strict(typo) { Ok(_) => println!("accepted (silently wrong)"), Err(e) => println!("rejected: {e}"), } }

💡 Hint 1

A diagnostic names two things: where (the line) and what (the unknown key). Both are already bound: line and key. Interpolate them into the message.

Show solution

return Err(format!("line {line}: unknown field `{key}`"));

rejected: line 2: unknown field `colmn`

The span (the line) is what turns a generic “unknown field” complaint into a located one. Drop it and the user is left grepping their file. That is exactly why clinker parses into Spanned<T> rather than plain values: so the line is already there when an error needs it.

Faded: write the strictness check yourself

Your turn with much less scaffolding. The Spanned struct, the loop, and the main are given, but the body of the loop that splits the line, enforces strictness, and records the span is yours to write. The rules: split on the first :; an unknown key is a located Err; a known key pushes a Spanned { value, line }.

rust // editable

reset ▶ run

struct Spanned { value: String, line: usize }

fn parse_strict(input: &str) -> Result<Vec<(String, Spanned)>, String> {
  let allowed = ["source", "column", "format"];
  let mut out = Vec::new();
  for (i, raw) in input.lines().enumerate() {
      let line = i + 1; // 1-based source position
      // TODO:
      //   1. split `raw` on the first ':' into (key, val); a line with no ':'
      //      is a located error.
      //   2. trim the key; if it is not in `allowed`, return a located Err.
      //   3. otherwise push a Spanned pairing the trimmed value with `line`.
      todo!()
  }
  Ok(out)
}

fn main() {
  for cfg in ["source: a.csv\ncolumn: id", "source: a.csv\ncolmn: id", "broken-line"] {
      match parse_strict(cfg) {
          Ok(fields) => for (k, s) in &fields { println!("line {}: {} = {}", s.line, k, s.value); },
          Err(e) => println!("rejected: {e}"),
      }
  }
}

struct Spanned { value: String, line: usize } fn parse_strict(input: &str) -> Result<Vec<(String, Spanned)>, String> { let allowed = ["source", "column", "format"]; let mut out = Vec::new(); for (i, raw) in input.lines().enumerate() { let line = i + 1; // 1-based source position // TODO: // 1. split `raw` on the first ':' into (key, val); a line with no ':' // is a located error. // 2. trim the key; if it is not in `allowed`, return a located Err. // 3. otherwise push a Spanned pairing the trimmed value with `line`. todo!() } Ok(out) } fn main() { for cfg in ["source: a.csv\ncolumn: id", "source: a.csv\ncolmn: id", "broken-line"] { match parse_strict(cfg) { Ok(fields) => for (k, s) in &fields { println!("line {}: {} = {}", s.line, k, s.value); }, Err(e) => println!("rejected: {e}"), } } }

Show solution

let (key, val) = raw
    .split_once(':')
    .ok_or_else(|| format!("line {line}: expected `key: value`"))?;
let key = key.trim();
if !allowed.contains(&key) {
    return Err(format!("line {line}: unknown field `{key}`"));
}
out.push((key.to_string(), Spanned { value: val.trim().into(), line }));

Three moves, and they are the same three clinker makes through serde: a strict membership check that rejects unknown keys, a located error built from the line, and a span-preserving push that keeps the line attached to the value. The ? on split_once propagates the malformed-line error the same way every fallible step does. That’s the Result machinery from lesson 12 doing real work here.

Solid on strict parsing and spans? Skip ahead →

Real-source grounding

The miniature mirrors clinker’s real shape. Strictness rides on the serde attribute you read above; the located part rides on parsing into Spanned<T> rather than plain values:

clinker-plan ·yaml.rs ·Spanned type @19acdcb4

What the engine enforces: because the whole pipeline is held as Vec<Spanned<PipelineNode>>, every node carries its line for free. A later pass that finds a bad reference doesn’t have to re-discover where the node came from: the span is already on it, the way RecordView already held its borrow rather than re-deriving it.

What a junior might misread: treating deny_unknown_fields and Spanned as one feature, or assuming the located error “just happens.” It doesn’t: strictness decides whether the typo fails, span-tracking decides whether the failure can name the line, and the single from_str door is what guarantees neither can be bypassed. Three separate decisions, one combined effect.

Strict, located, and behind one door

Two lessons, two flavours of “make the type carry the guarantee”: a proof token for security, spans-and-strictness for config. Both feed the same destination, the validated plan the executor runs.

Retrieval checkpoint

✓ Checkpoint — strict, located parsing

💡 Hint 1

Read the yaml.rs module doc comment at the top first; it states the single-entry-point rule and the two reasons. Then count how many config files carry deny_unknown_fields.

What the tour shows

from_str is the lone public parse entry; its module doc forbids any other call into serde_saphyr, and it applies a shared resource budget plus a pre-parse size check. The deny_unknown_fields grep returns a long list; it’s a crate-wide convention, not one struct. And Spanned is imported and used pervasively so that diagnostics can point at the source line. Together: one strict, bounded, location-preserving door for all config.

// quick check

What does #[serde(deny_unknown_fields)] change about parsing a config?

• It makes parsing faster by skipping validation
• It rejects any key the struct doesn't declare, turning a typo'd or stray field into a parse error instead of a silently-ignored one
• It records the source line of every field

// quick check

Two engineers describe Spanned<T> and deny_unknown_fields. Which statement is the MISCONCEPTION to correct?

• deny_unknown_fields decides whether a typo'd key is an error; Spanned decides whether that error can name the line
• They are two halves of one feature: turning on deny_unknown_fields is what makes the error report the source line
• You could have strictness without spans (a located-less 'unknown field' error) or spans without strictness (a lenient parser that still tracks lines)

// quick check

Why route every YAML parse through a single wrapper function?

• To make the code shorter
• So the resource limits (size, depth, node count, include-disabling) apply to every parse, and so a risky single-maintainer dependency has exactly one place to be swapped
• Because serde requires a single entry point

// quick check

Spaced recall from the lifetimes & zero-copy lesson: RecordView<'a, S> bundles a borrow with a lifetime 'a; Spanned<T> bundles a value with its source span. What is the shared idea, and what does each extra fact prevent?

• Both copy the data to keep it safe; the lifetime and the span are owned duplicates
• Both bundle a value with a fact about it carried IN THE TYPE: the lifetime keeps a borrow from outliving its storage (a dangling reference); the span keeps a value's source line available so an error can point at it
• Both are runtime checks that panic if the fact is wrong

You should be able to:

• You can name what Spanned<T> carries beyond the value, and why that matters for errors
• You can say what deny_unknown_fields does and why a silent-ignore default is dangerous
• You wrote a parse loop that rejects an unknown key with a located error and pushes a Spanned for a known one
• You can explain why all YAML goes through one from_str and name one benefit it adds on top of strictness and spans
• You can distinguish span-tracking from strictness: which decides 'fails or not' and which decides 'names the line'

Verify in the checkout:

grep -n 'pub fn from_str' crates/clinker-plan/src/yaml.rs
grep -rn 'deny_unknown_fields' crates/clinker-plan/src/config/ | head
grep -n 'Spanned' crates/clinker-plan/src/yaml.rs | head

Connections

You can now read the Spanned<T> wrapper, explain what deny_unknown_fields enforces, write a strict and located config reader, and separate span-tracking from strictness from the chokepoint that guarantees both. Next we meet the validated plan itself, and the typed handle that says “this has been fully compiled” the same unforgeable way ValidatedPath said “this was screened.”

Prerequisites

• Proof tokens

→

Leads to

• The plan/runtime boundary

Go deeper on the Rust (optional; the first goes to The Rust Book for first principles, the second to an in-track build):

• Want Result and the ? operator (the machinery a located parse error rides on) from first principles? Read the docs →
• Want to design the error type itself (Display, source span, From) by hand? Skip ahead →

Glossary terms used: span, Spanned<T>, source position, diagnostic.

Mark this lesson complete