Result & the ? operator — modeling failure

The Rust question for this lesson: the last three lessons were about who owns a value — one owner, a borrow, the heap, many owners. This one is about something every real function faces: what if it fails? Parsing a number, opening a file, writing a byte — any of these can go wrong. Many languages reach for exceptions (invisible in the signature) or a magic null. Rust has neither. A function that might fail says so in its type: it returns Result<T, E> — either Ok(value) with the answer or Err(error) with the reason — and the compiler won’t let you touch the value without acknowledging the error. And when you chain five fallible steps, how do you avoid a staircase of matches? The ? operator. This lesson opens Phase 4 — modeling state & failure.

Know Result, Ok/Err, and how ? lifts errors via From? Skip ahead. Skip ahead →

Failure, written into the type

Result<T, E> is an ordinary enum — enum Result<T, E> { Ok(T), Err(E) } — but it’s the one Rust uses everywhere a thing can fail. A function returning Result<Shape, ShapeError> is telling you, in its signature, “you’ll get a Shape or a ShapeError, and you must deal with both.” The ? operator is the ergonomic half: expr? unwraps an Ok, or returns the Err from the whole function early — converting the error type as it goes.

rust // editable

reset ▶ run

// "This might fail" is written into the return type: a Result.
#[derive(Debug)]
enum Shape { Circle(f64), Square(f64) }

// Our own error type. One variant wraps the std error that parsing a float
// produces, so the detail (which character broke) is carried along.
#[derive(Debug)]
enum ShapeError {
  BadSize(std::num::ParseFloatError),
  UnknownKind(String),
}

// The hinge: ? will use From to turn a ParseFloatError into a ShapeError
// automatically. Without this impl, the "raw.parse()?" line below won't compile.
impl From<std::num::ParseFloatError> for ShapeError {
  fn from(e: std::num::ParseFloatError) -> Self {
      ShapeError::BadSize(e)
  }
}

fn parse_shape(spec: &str) -> Result<Shape, ShapeError> {
  // split_once returns Option; ok_or turns None into an Err, then ? unwraps it.
  let (kind, raw) = spec
      .split_once(' ')
      .ok_or(ShapeError::UnknownKind(spec.to_string()))?;

  // parse() returns Result<f64, ParseFloatError>; ? unwraps the Ok value,
  // or returns early after lifting the error into ShapeError via From.
  let size: f64 = raw.parse()?;

  match kind {
      "circle" => Ok(Shape::Circle(size)),
      "square" => Ok(Shape::Square(size)),
      other => Err(ShapeError::UnknownKind(other.to_string())),
  }
}

fn main() {
  // The first time, handle both arms explicitly with match:
  match parse_shape("circle 2.0") {
      Ok(shape) => println!("parsed: {:?}", shape),   // Ok(Circle(2.0))
      Err(e) => println!("failed: {:?}", e),
  }

  // The failure paths each RETURN an Err — no panic, no exception:
  println!("{:?}", parse_shape("square huh"));  // Err(BadSize(..))      <- lifted by ?
  println!("{:?}", parse_shape("hexagon"));     // Err(UnknownKind("hexagon"))
}

// "This might fail" is written into the return type: a Result. #[derive(Debug)] enum Shape { Circle(f64), Square(f64) } // Our own error type. One variant wraps the std error that parsing a float // produces, so the detail (which character broke) is carried along. #[derive(Debug)] enum ShapeError { BadSize(std::num::ParseFloatError), UnknownKind(String), } // The hinge: ? will use From to turn a ParseFloatError into a ShapeError // automatically. Without this impl, the "raw.parse()?" line below won't compile. impl From<std::num::ParseFloatError> for ShapeError { fn from(e: std::num::ParseFloatError) -> Self { ShapeError::BadSize(e) } } fn parse_shape(spec: &str) -> Result<Shape, ShapeError> { // split_once returns Option; ok_or turns None into an Err, then ? unwraps it. let (kind, raw) = spec .split_once(' ') .ok_or(ShapeError::UnknownKind(spec.to_string()))?; // parse() returns Result<f64, ParseFloatError>; ? unwraps the Ok value, // or returns early after lifting the error into ShapeError via From. let size: f64 = raw.parse()?; match kind { "circle" => Ok(Shape::Circle(size)), "square" => Ok(Shape::Square(size)), other => Err(ShapeError::UnknownKind(other.to_string())), } } fn main() { // The first time, handle both arms explicitly with match: match parse_shape("circle 2.0") { Ok(shape) => println!("parsed: {:?}", shape), // Ok(Circle(2.0)) Err(e) => println!("failed: {:?}", e), } // The failure paths each RETURN an Err — no panic, no exception: println!("{:?}", parse_shape("square huh")); // Err(BadSize(..)) <- lifted by ? println!("{:?}", parse_shape("hexagon")); // Err(UnknownKind("hexagon")) }

Read ? like this: raw.parse()? means if parse() returned Ok(v), evaluate to v and carry on; if it returned Err(e), stop here and return Err(e.into()) from the whole function. That quiet .into() is the load-bearing part — ? converts the error into the function’s error type using a From impl. That is why the From<ParseFloatError> block is not decoration: delete it and raw.parse()? stops compiling, because ? would have no way to turn a ParseFloatError into a ShapeError. One character replaces a five-line match, and it carries the conversion with it.

// quick check

When `expr?` is applied to a value that is Err(e), what happens?

• The function returns early with Err(e.into()) — the error is converted via From and propagated to the caller
• The program panics and unwinds, like .unwrap()
• ? evaluates to a default value and execution continues

The engine coerces cells exactly this way

When a pipeline reads a CSV or HL7 source, every cell arrives as a Value::String. A typed stage then has to turn "42" into a Value::Integer — and that can fail. Clinker’s coercion layer says so in the return type:

clinker-record ·coercion.rs ·coerce_to_int fn @47d2e12

/// Strict coercion: returns Err on failure. Null propagates as Ok(Null).
pub fn coerce_to_int(value: &Value) -> Result<Value, CoercionError> {
    match value {
        Value::Null => Ok(Value::Null),
        Value::Integer(_) => Ok(value.clone()),
        Value::Float(f) => Ok(Value::Integer(*f as i64)),
        Value::String(s) => s
            .parse::<i64>()
            .map(Value::Integer)
            .map_err(|_| CoercionError::ParseFailure {
                input: s.to_string(),
                target: "Integer",
            }),
        other => Err(CoercionError::TypeMismatch {
            from: other.type_name(),
            to: "Integer",
            value: other.to_string(),
        }),
    }
}

The signature Result<Value, CoercionError> declares the failure up front. A "42" string parses to Ok(Value::Integer(42)); an "abc" string fails — and map_err translates the std ParseIntError into a domain CoercionError::ParseFailure. That map_err is the same error-translation ? performs automatically, written out by hand here because the code also wants to attach context: the offending input and the target type, so the diagnostic can say “failed to parse ‘abc’ as Integer” instead of a bare parse error.

Now the bridge back to the previous failure-shaped type you know, Option:

/// Lenient coercion: returns None on failure. Null propagates as Some(Null).
pub fn coerce_to_int_lenient(value: &Value) -> Option<Value> {
    coerce_to_int(value).ok()
}

Every strict coerce_* has a lenient twin that is just .ok(). Result::ok discards the error and converts the outcome to an Option: Ok(v) becomes Some(v), Err(_) becomes None. That single method is the seam between the two “this might not work” types — Result (failure with a reason) and Option (absence, no reason). The engine keeps both functions because the caller decides whether the reason is worth carrying.

The same machinery, at engine scale

The toy’s From<ParseFloatError> for ShapeError impl was tiny. Here is the production version — the error a join or aggregate hits when it outgrows memory and spills batches to disk:

clinker-plan ·runtime_error.rs ·SpillError type @47d2e12

pub enum SpillError {
    Io(std::io::Error),
    Json(serde_json::Error),
    Postcard(postcard::Error),
    // … plus richer DirUnavailable / DiskFull variants
}

impl From<std::io::Error> for SpillError {
    fn from(e: std::io::Error) -> Self { SpillError::Io(e) }
}
impl From<serde_json::Error> for SpillError {
    fn from(e: serde_json::Error) -> Self { SpillError::Json(e) }
}
impl From<postcard::Error> for SpillError {
    fn from(e: postcard::Error) -> Self { SpillError::Postcard(e) }
}

The spill writer touches three failure-prone libraries: the OS (io::Error), a JSON header (serde_json::Error), and a postcard-encoded body (postcard::Error). Each From impl teaches ? how to fold that library’s error into one unified SpillError. So spill code can write writer.write_all(&buf)? and serde_json::to_vec(&schema)? back to back — three different error types collapsing into one through ?, with no match and no manual conversion at the call site. The enum is the union of everything that can go wrong here; the From impls are the on-ramps ? drives onto it. That is the whole pattern: one error type per layer, a From per error source, and ? to lift.

Clinker context

The two examples sit at opposite ends of a run. coerce_to_int is in the record/transform layer (clinker-record): one cell’s worth of “make this the declared type,” where the Result turns “this cell didn’t fit” into a first-class, inspectable outcome instead of a silent 0 or a crash. SpillError lives in the planning/runtime vocabulary (clinker-plan), produced by the executor’s disk-spill subsystem far downstream. Different crates, different failures — but the same Rust spine: name every failure in a type, return it in a Result, and let ? carry it upward.

Why there are two coercion functions — Documented

The strict/lenient split is the smallest version of a decision the whole engine makes: when something fails, do you stop, or absorb it and continue? The doc comments state it outright — strict coerce_to_int “returns Err on failure” (the reason survives, so a caller can route the bad row somewhere), lenient coerce_to_int_lenient “returns None on failure” (the reason is dropped, the value degrades to null, the run continues). Result is the type that holds the reason; Option is the type that discards it. At the pipeline level this same recoverable-vs-fatal choice scales up into an explicit policy — a per-pipeline ErrorStrategy (continue past bad rows, or fail fast), plus a class of internal errors that abort the run no matter what — which the next Phase-4 lessons unpack. The point for this lesson: Result is where Rust forces “can this fail, and does the reason matter?” into a typed, unavoidable question.

Your turn

The outline’s exercise: add a SpillError variant and its From, and show ? lifts it — in a scratch stand-in (never edit Clinker’s source; the contract is scratch-copies only). Use the playground.

(a) Add a new variant to the scratch SpillError below — say Lz4(String) for a compression failure.

(b) Make ? lift a new error source into SpillError: add a From impl for some error type and call something that returns it with ?. (Start by parsing an int: "x".parse::<i64>()? produces a std::num::ParseIntError — add impl From<std::num::ParseIntError> for SpillError and watch ? accept it.)

(c) Comment out the existing impl From<std::io::Error> block and recompile. Read the error on the ? line — ? cannot lift an io::Error into a SpillError without that From. That error message is the lesson.

rust // editable

reset ▶ run

use std::io::Write;

// A scratch stand-in for the real SpillError — same shape, runnable here.
#[derive(Debug)]
enum SpillError {
  Io(std::io::Error),
  // (a) ADD a variant here, e.g.:  Lz4(String),
}

impl From<std::io::Error> for SpillError {
  fn from(e: std::io::Error) -> Self { SpillError::Io(e) }
}
// (b) ADD a matching From impl for your new error source here, e.g.:
// impl From<std::num::ParseIntError> for SpillError { ... }

// A fallible "spill write" that leans on ? to lift io::Error into SpillError.
fn write_spill(buf: &[u8]) -> Result<usize, SpillError> {
  let mut sink = Vec::new();
  let n = sink.write(buf)?;   // io::Error -> SpillError, lifted by ? via From
  Ok(n)
}

fn main() {
  println!("{:?}", write_spill(b"batch-0"));  // Ok(7)
  // (c) Comment out the impl From<std::io::Error> block above and recompile.
}

use std::io::Write; // A scratch stand-in for the real SpillError — same shape, runnable here. #[derive(Debug)] enum SpillError { Io(std::io::Error), // (a) ADD a variant here, e.g.: Lz4(String), } impl From<std::io::Error> for SpillError { fn from(e: std::io::Error) -> Self { SpillError::Io(e) } } // (b) ADD a matching From impl for your new error source here, e.g.: // impl From<std::num::ParseIntError> for SpillError { ... } // A fallible "spill write" that leans on ? to lift io::Error into SpillError. fn write_spill(buf: &[u8]) -> Result<usize, SpillError> { let mut sink = Vec::new(); let n = sink.write(buf)?; // io::Error -> SpillError, lifted by ? via From Ok(n) } fn main() { println!("{:?}", write_spill(b"batch-0")); // Ok(7) // (c) Comment out the impl From<std::io::Error> block above and recompile. }

💡 Hint 1

? desugars to roughly match expr { Ok(v) => v, Err(e) => return Err(SpillError::from(e)) }. So the only thing ? needs is a From<ThatError> for SpillError impl. Adding a variant (step a) gives the error somewhere to land; adding the From (step b) is what actually lets ? perform the conversion.

Show solution

(a) + (b) Add Lz4(String) (or any variant) and a From for a second error source:

enum SpillError {
    Io(std::io::Error),
    Parse(std::num::ParseIntError),   // (a) the new variant
}

impl From<std::num::ParseIntError> for SpillError {   // (b) the on-ramp
    fn from(e: std::num::ParseIntError) -> Self { SpillError::Parse(e) }
}

fn count_from(text: &str) -> Result<i64, SpillError> {
    let n: i64 = text.parse()?;   // ParseIntError -> SpillError, lifted by ?
    Ok(n)
}

With both pieces in place, text.parse()? compiles even though parse yields a ParseIntError, not a SpillError — ? calls SpillError::from(e) for you.

(c) Remove impl From<std::io::Error> for SpillError and the sink.write(buf)? line fails with roughly: the trait `From<std::io::Error>` is not implemented for `SpillError` (often phrased as “the ? operator cannot convert”). That is ? telling you the on-ramp is missing — restore the From impl and it compiles again. The real SpillError carries four such impls (io::Error, serde_json::Error, postcard::Error, and an lz4_flex frame error) for exactly this reason.

Common misconceptions

• “? is just .unwrap() with nicer syntax.” No. .unwrap() panics and crashes the thread on Err; ? returns the Err to the caller, converted via From. One aborts, the other propagates — opposite behaviors. Reach for ? in any function that itself returns a Result/Option.
• “? only works when the error types already match.” No — ? calls .into() on the error, so any error E that has a From into the function’s error type is lifted automatically. That From impl is required, though: without it, ? won’t compile. The whole SpillError pattern (a From per library error) exists to make ? usable.
• “Result and Option are interchangeable.” Both model “there might not be a T,” but Result carries a reason on failure (Err(E)) and Option does not (None). Convert with .ok() (Result → Option, dropping the reason — exactly what coerce_*_lenient does) or .ok_or(e) (Option → Result, supplying one). Use Result when the caller needs to know why it failed.

✓ Checkpoint

// quick check

Why does SpillError implement From for io::Error, serde_json::Error, and postcard::Error separately?

• So the ? operator can lift each library's error into one unified SpillError automatically, with no match at the call site
• Because Rust requires every enum variant to have a From impl
• To make SpillError larger so it never needs boxing

You should be able to:

• Write a function returning Result<T, E> and explain what Ok and Err each carry
• Explain what ? does on Ok vs Err, and what From has to do with it
• Read coerce_to_int and say where the failure is modeled, then explain how .ok() bridges Result to Option
• State why SpillError needs one From impl per error source

Verify in the checkout:

cargo test -p clinker-record test_coerce_string_to_int_invalid
cargo check -p clinker-plan

Verify it for real

The strict-vs-lenient Result claim has a real test in clinker-record — a bad string fails the strict path and yields None on the lenient one:

cargo test -p clinker-record test_coerce_string_to_int_invalid

It builds Value::String("abc") and asserts coerce_to_int(&v).is_err() and coerce_to_int_lenient(&v).is_none() — the two halves of the Result/Option pairing in one test. For the ?/From side, the exercise above compiles against a scratch SpillError; the real SpillError lives in clinker-plan, so a plain cargo check -p clinker-plan confirms its From impls (the on-ramps ? relies on) build at the pin.

Where this leads

You can now read any fallible Clinker signature: Result<T, E> says “this can fail,” and ? threads the failure upward, converting it through From. That is the load-bearing half of Phase 4. The next lessons turn from using a Result to designing its error type: first by hand — the Display + std::error::Error + From boilerplate that a type like FormatError writes out in full — and then with thiserror, which generates that same boilerplate from a derive. After that, Clinker’s PipelineError vocabulary and the recoverable-vs-fatal model this lesson only gestured at.

Prerequisites

• Option — absence without null

→

Leads to

• Designing an error type by hand — Display, Error, From

Mark this lesson complete