Option & Result

Two questions come up constantly in a data engine: is this field even here? and did this step fail? Many languages answer the first with null and the second by throwing. Rust answers both with ordinary types you can’t ignore — Option and Result — and the engine’s source is full of them.

You’ll be able to: read Option and Result in real signatures, and explain how the engine uses them to separate “no more records” and “absent field” from “this failed.”

`Option`: maybe there, maybe not

You already saw one: resolve(&self, name: &str) -> Option<&Value>. The Option is honest about a fact the type system would otherwise hide — a field you ask for might not exist. Some(&value) if it’s there, None if it isn’t. There’s no null to forget to check; to get at the value you must handle the None.

The same shape marks the end of a finite source. A reader’s next_record yields Some(record) for each row and None when the file is exhausted — that None is how a finite job knows to stop:

clinker-format ·traits.rs ·next_record fn @47d2e12

`Result`: maybe it failed

But reading a record can also fail — a malformed line, a broken encoding. That’s not “no more records”; it’s an error. So next_record doesn’t just return an Option — it returns a Result wrapping one:

fn next_record(&mut self) -> Result<Option<Record>, FormatError>;
//                           Ok(Some(record)) → a row
//                           Ok(None)         → end of input (clean)
//                           Err(e)           → something went wrong

Result makes failure a value you handle, not an exception that unwinds the stack. Three outcomes, all in the type: a row, a clean end, or an error.

Recoverable vs fatal — the DLQ

Not every error should kill a job. If one row in a million can’t be coerced — "active".to_int() is nonsense — Clinker can route that single bad record to the dead-letter queue (the dlq count you saw in --dry-run) and keep going, instead of aborting the whole run. The coercion failure itself is a typed error:

clinker-record ·coercion.rs ·CoercionError type @47d2e12

pub enum CoercionError {
    TypeMismatch { from: &'static str, to: &'static str, value: String },
    ParseFailure { input: String, target: &'static str },
}

That’s the difference the type system helps enforce: a data error (one bad row) is recoverable and can be DLQ’d; an invariant error (a bug, an impossible state) is fatal. You’ll meet the engine’s full error vocabulary, PipelineError, in Phase 3.

Handle them, don’t ignore them

rust // editable

#[derive(Debug)]
enum Value { Integer(i64), Text(String) }

// Coerce a text field to an integer — which can fail.
fn to_int(v: &Value) -> Result<i64, String> {
  match v {
      Value::Integer(n) => Ok(*n),
      Value::Text(s) => s.parse::<i64>().map_err(|_| format!("not an integer: {s:?}")),
  }
}

fn main() {
  let cells = [Value::Text(String::from("15200")), Value::Text(String::from("active"))];
  for cell in &cells {
      match to_int(cell) {
          Ok(n) => println!("ok: {n}"),
          Err(e) => println!("would go to the DLQ: {e}"),
      }
  }
}

#[derive(Debug)]
enum Value { Integer(i64), Text(String) }

// Coerce a text field to an integer — which can fail.
fn to_int(v: &Value) -> Result<i64, String> {
  match v {
      Value::Integer(n) => Ok(*n),
      Value::Text(s) => s.parse::<i64>().map_err(|_| format!("not an integer: {s:?}")),
  }
}

fn main() {
  let cells = [Value::Text(String::from("15200")), Value::Text(String::from("active"))];
  for cell in &cells {
      match to_int(cell) {
          Ok(n) => println!("ok: {n}"),
          Err(e) => println!("would go to the DLQ: {e}"),
      }
  }
}

> output appears here — press Run

"15200" coerces; "active" doesn’t, and the Err arm is exactly where the engine decides “DLQ this row and continue.”

// quick check

A reader's next_record returns Result<Option<Record>>. What does Ok(None) mean?

Ok(None) is the clean end-of-input signal: no error (Ok), and no further record (None). An actual failure would be Err(e).

Checkpoint

✓ Checkpoint

// quick check

A single row can't have its lifetime_value coerced to an integer. What does the engine do under a continue strategy?

A data error on one row is recoverable: it goes to the DLQ (the dlq count) and the run continues. Aborting is reserved for fatal, invariant-level errors.

You should be able to:

You can explain what Option<&Value> from resolve() encodes (absent field)
You can read the three outcomes of Result<Option<Record>>
You can give an example of a recoverable (DLQ) error vs a fatal one

Verify in the checkout:

grep -n 'fn next_record' crates/clinker-format/src/traits.rs
grep -n 'CoercionError' crates/clinker-record/src/coercion.rs

You can read absence and failure in the types. Next: the small struct that rides along with every record so the engine always knows where a row came from.