Dispatching a node — enum vs trait

Every node in a compiled DAG has to be routed to the operator that runs it, and the executor faces a sharp design question: how do you call into one of many kinds? You met one answer already, in Planning & Expressions. The format layer reached its readers through Box<dyn FormatReader>, dynamic dispatch, because formats are an open-ended, runtime-chosen plug-in seam. The DAG executor faces the same shape of problem (many kinds of node) and deliberately answers it the opposite way: a closed enum and one exhaustive match. This is the first lesson of the deepest pass, where we stop reading plans and watch records actually move. The Rust tools in tension are the enum and the trait; we use just enough of each to read the real dispatch code, and The Rust Book, ch. 10 is the canonical treatment of traits and trait objects if you want them from first principles.

Prerequisites

• CXL: a staged language

→

Leads to

• The worker pool

What you’ll be able to do

• Distinguish static dispatch from dynamic dispatch, and name what each costs: a vtable indirection per call versus none.
• Read the executor’s central match and explain why it has no _ => catch-all and no dyn Operator anywhere.
• Explain when a closed enum + exhaustive match beats Box<dyn Trait>, and why the IO seam made the opposite call.
• Write both strategies in runnable Rust and predict which one the compiler turns into a checklist when a new kind is added.

New terms in this lesson (each is also expanded inline at the point you first need it):

• static dispatch: the call target is fixed at compile time.
• dynamic dispatch: the call target is found at run time, through a table.
• vtable: the per-type method table a trait object follows.
• dyn: pre-taught where it reappears below, the marker of a trait object.
• enum dispatch: one match over a closed enum, the executor’s strategy.

Predict first

✋ Predict first

The executor must route each DAG node to its operator. There are 13 kinds of node, all decided by the engine. One author reaches for Box<dyn Operator> (a trait object per node); another reaches for a single match on a closed enum. A 14th node kind is added next quarter but ONE author forgets to wire up its dispatch. In which design does that omission become a compile error, and in which does it slip through to run time? Commit to an answer before revealing.

Decide on your answer before you open the reveal — committing a guess is what makes it stick.

Reveal the answer

The closed-enum + match design catches it at compile time; the dyn Operator design lets it reach run time. That gap is the whole reason the executor chose the enum, and it is the skill this lesson builds.

A wildcard-free match over a closed enum is exhaustive: add a 14th variant and every match that doesn’t handle it stops compiling, and the compiler names the exact site to fix. With Box<dyn Operator>, the new operator is just one more type satisfying the trait. Nothing forces a central site to acknowledge it, so a forgotten wiring is a run-time surprise (a node that does nothing, or a lookup that returns None), not a build failure.

The trap is assuming “more kinds → trait objects, always.” Trait objects are the right tool when the set is open and chosen by outsiders at run time, which is exactly the IO seam. Node kinds are the opposite: a closed, engine-owned set. Hold that prediction; everything below earns it.

The closed set of node kinds

A compiled plan is a DAG of nodes, and a node is one of a fixed, engine-known set of kinds: a source, a transform, a route, a merge, a sort, an aggregation, an output, and a handful more. That set is a closed enum, the same PlanNode you glimpsed in Planning & Expressions:

clinker-plan ·mod.rs ·PlanNode type @19acdcb4

pub enum PlanNode {
    Source { /* ... */ },
    Transform { /* ... */ },
    Route { /* ... */ },
    Merge { /* ... */ },
    Sort { /* ... */ },
    Aggregation { /* ... */ },
    Output { /* ... */ },
    // ... 13 variants in all — the complete vocabulary of pipeline nodes
}

The key word is closed. The kinds of node a pipeline can contain are decided by the engine, not by users, not at run time. Contrast that with formats: anyone can add a new wire format, and which one a job uses is read from the plan at run time. Formats are open; node kinds are closed. That single difference drives the whole dispatch decision.

Two ways to dispatch over many kinds

“Call into one of many kinds” has two standard answers in Rust, and this lesson is about choosing between them.

• A trait object, Box<dyn Operator>. Each kind is its own type implementing a shared trait; you store them all behind one boxed dyn and call through a vtable. The call target is found at run time, which is dynamic dispatch. The set can be open: outside code can add a new implementor the central code never named.
• A closed enum, one match over a fixed set of variants. Each arm is resolved at compile time, so the call is static dispatch: no vtable, no indirection. The set is closed: every kind is named in one place.

You met the first strategy in the IO seam and the static-dispatch idea again in one reader, every format (generics). This lesson is the deliberate contrast: same problem shape, opposite choice, because the constraints flipped.

One exhaustive match, no dyn

Because the node set is closed, the executor dispatches with a single match over the enum, enum dispatch. Each arm hands the node to its operator module:

clinker-exec ·dispatch.rs ·dispatch_plan_node fn @19acdcb4

pub(crate) fn dispatch_plan_node(
    ctx: &mut ExecutorContext<'_>,
    current_dag: &ExecutionPlanDag,
    node_idx: NodeIndex,
) -> Result<(), PipelineError> {
    let node = current_dag.graph[node_idx].clone();
    match node {
        PlanNode::Source { .. }      => dispatch_source(ctx, current_dag, node_idx, &node)?,
        PlanNode::Transform { .. }   => dispatch_transform(ctx, current_dag, node_idx, &node)?,
        PlanNode::Route { .. }       => dispatch_route(ctx, current_dag, node_idx, &node)?,
        PlanNode::Merge { .. }       => dispatch_merge(ctx, current_dag, node_idx, &node)?,
        // ... one arm per variant — and crucially, NO `_ =>` catch-all
    }
    Ok(())
}

There is no trait object here and none anywhere in the engine: searching the whole codebase for dyn Operator or trait Operator finds nothing. Operators are not boxed behind a vtable; they’re arms of a match. And there is no _ => wildcard: the match spells out every variant. That second detail is doing real architectural work, as the next section shows.

Why closed-enum dispatch wins here

The IO seam argued dynamic dispatch was right for formats. Here the trade flips, for three concrete reasons:

• Exhaustiveness is a feature. Because the match has no catch-all, adding a new PlanNode variant makes every non-exhaustive match a compile error. The compiler hands you the exact list of sites to update (exactly the guarantee from reading a value). With dyn Operator, a forgotten case would be a run-time surprise, not a build failure.
• The set is closed and known at compile time. A dyn seam exists to let outside code plug in types the engine never named. Node kinds are all named by the engine itself, so the open-endedness dyn buys is worthless here, and you’d pay a vtable indirection per node for it.
• Operators want specialized data. Each arm can pull the variant’s own payload (a sort’s keys, a transform’s compiled program) by pattern-matching, with no downcasting.

The mental model: dyn is for open sets chosen by others at run time; a closed enum is for sets the engine owns and wants the compiler to police. The IO seam is the former, the operator set the latter. Same language feature family (traits and enums), opposite tool for opposite jobs.

Evidence note

The source encodes this choice structurally (there is no dyn Operator, and the dispatch match is wildcard-free) but carries no comment stating the rationale, and the planning docs in this checkout reference no ADR for it. The reasoning above is the standard Rust trade-off read off the code, not a quote from it.

Worked → completion → faded

Now you build both strategies in runnable Rust, scaffolded down to your own. One new idea per rung: first read the enum strategy and feel the exhaustiveness; then complete a missing arm; then write the closed-enum dispatcher from a skeleton, and see the trait-object version stay silent where the enum version shouts.

Worked: feel the exhaustiveness

Here are both strategies side by side. The enum version makes the compiler your checklist; the trait-object version quietly accepts a new kind with no nudge. Run it, then read the comments.

rust // editable

reset ▶ run

// CLOSED ENUM + exhaustive match — the executor's strategy (static dispatch).
enum Node { Source, Transform, Output }

fn dispatch(n: &Node) -> &'static str {
  match n {
      Node::Source => "read",
      Node::Transform => "evaluate",
      Node::Output => "write",
      // no `_ =>` — add a 4th variant and THIS match stops compiling
  }
}

// TRAIT OBJECT + dyn — the IO-seam strategy (dynamic dispatch through a vtable).
trait Operator { fn run(&self) -> &'static str; }
struct Read; struct Eval; struct Write;
impl Operator for Read  { fn run(&self) -> &'static str { "read" } }
impl Operator for Eval  { fn run(&self) -> &'static str { "evaluate" } }
impl Operator for Write { fn run(&self) -> &'static str { "write" } }

fn main() {
  // enum dispatch: one match, every kind named in one place.
  for n in [Node::Source, Node::Transform, Node::Output] {
      println!("enum:  {}", dispatch(&n));
  }
  // dyn dispatch: a Vec of boxed trait objects, each call goes through a vtable.
  let ops: Vec<Box<dyn Operator>> = vec![Box::new(Read), Box::new(Eval), Box::new(Write)];
  for op in &ops {
      println!("dyn:   {}", op.run());
  }
  // Try this: add a `Sort` variant to Node. The compiler immediately flags
  // `dispatch` as non-exhaustive. Now add a `Sort` operator struct instead —
  // NOTHING forces you to push it into `ops`; the omission is silent.
}

// CLOSED ENUM + exhaustive match — the executor's strategy (static dispatch). enum Node { Source, Transform, Output } fn dispatch(n: &Node) -> &'static str { match n { Node::Source => "read", Node::Transform => "evaluate", Node::Output => "write", // no `_ =>` — add a 4th variant and THIS match stops compiling } } // TRAIT OBJECT + dyn — the IO-seam strategy (dynamic dispatch through a vtable). trait Operator { fn run(&self) -> &'static str; } struct Read; struct Eval; struct Write; impl Operator for Read { fn run(&self) -> &'static str { "read" } } impl Operator for Eval { fn run(&self) -> &'static str { "evaluate" } } impl Operator for Write { fn run(&self) -> &'static str { "write" } } fn main() { // enum dispatch: one match, every kind named in one place. for n in [Node::Source, Node::Transform, Node::Output] { println!("enum: {}", dispatch(&n)); } // dyn dispatch: a Vec of boxed trait objects, each call goes through a vtable. let ops: Vec<Box<dyn Operator>> = vec![Box::new(Read), Box::new(Eval), Box::new(Write)]; for op in &ops { println!("dyn: {}", op.run()); } // Try this: add a `Sort` variant to Node. The compiler immediately flags // `dispatch` as non-exhaustive. Now add a `Sort` operator struct instead — // NOTHING forces you to push it into `ops`; the omission is silent. }

Add a Sort variant to Node and the build breaks at dispatch, pointing at the exact code that doesn’t yet handle it. That is the compiler acting as a complete, always-current checklist of “every place a new operator must be wired in.” The dyn Operator side has no such site: a new Sort struct compiles fine and simply never appears in ops if you forget, and the failure waits for run time.

Completion: write the missing arm

Below, the closed-enum dispatcher is written for you except one arm. A Merge variant exists but its arm was dropped, so the match is non-exhaustive and won’t build. Fill in the arm so every variant is handled, and notice you can’t paper over it with a _ => and keep the lesson’s guarantee.

rust // editable

reset ▶ run

enum Node { Source, Transform, Merge, Output }

fn dispatch(n: &Node) -> &'static str {
  match n {
      Node::Source => "read",
      Node::Transform => "evaluate",
      // TODO: add the Merge arm here — what should a merge node do?
      Node::Output => "write",
  }
}

fn main() {
  for n in [Node::Source, Node::Transform, Node::Merge, Node::Output] {
      println!("{}", dispatch(&n));
  }
}

enum Node { Source, Transform, Merge, Output } fn dispatch(n: &Node) -> &'static str { match n { Node::Source => "read", Node::Transform => "evaluate", // TODO: add the Merge arm here — what should a merge node do? Node::Output => "write", } } fn main() { for n in [Node::Source, Node::Transform, Node::Merge, Node::Output] { println!("{}", dispatch(&n)); } }

💡 Hint 1

The other three arms each return a one-word action for that kind. A merge node combines several inputs into one stream, so its action word is "combine" (or similar). Add Node::Merge => "combine",, a real arm, not a _ => wildcard, so the match stays exhaustive.

Show solution

Node::Merge => "combine",

A named arm, not a _ => catch-all. The difference is the whole point: with the explicit arm, the next variant someone adds breaks the build here and demands attention. A _ => wildcard would have compiled today but silently swallowed every future kind, throwing away the compiler-as-checklist guarantee the executor relies on.

Faded: write the dispatcher from a skeleton

Your turn with much less scaffolding. Given the closed Stage enum, write dispatch so it maps each variant to its action word, with no _ => arm. Then confirm the claim for yourself: after it compiles, mentally add a Validate variant and predict exactly what the compiler says.

rust // editable

reset ▶ run

enum Stage { Parse, Typecheck, Eval }

// TODO: write dispatch(s: &Stage) -> &'static str
//   Parse     => "parse"
//   Typecheck => "typecheck"
//   Eval      => "eval"
// Use one match, no `_ =>` catch-all.
fn dispatch(s: &Stage) -> &'static str {
  todo!()
}

fn main() {
  for s in [Stage::Parse, Stage::Typecheck, Stage::Eval] {
      println!("{}", dispatch(&s));
  }
}

enum Stage { Parse, Typecheck, Eval } // TODO: write dispatch(s: &Stage) -> &'static str // Parse => "parse" // Typecheck => "typecheck" // Eval => "eval" // Use one match, no `_ =>` catch-all. fn dispatch(s: &Stage) -> &'static str { todo!() } fn main() { for s in [Stage::Parse, Stage::Typecheck, Stage::Eval] { println!("{}", dispatch(&s)); } }

Show solution

fn dispatch(s: &Stage) -> &'static str {
    match s {
        Stage::Parse => "parse",
        Stage::Typecheck => "typecheck",
        Stage::Eval => "eval",
    }
}

This is dispatch_plan_node in miniature: one match, one arm per variant, no wildcard. Add a Validate variant and the compiler reports non-exhaustive patterns: Stage::Validate not covered, pointing straight at this function. That error is not an annoyance; it is the engine’s guarantee that no node kind can ever silently fall through dispatch.

Retrieval checkpoint

✓ Checkpoint — enum vs trait dispatch

💡 Hint 1

Read the match node block in dispatch_plan_node to the end. Is there a _ => arm? Then run the last grep: does any dyn Operator exist to compare against?

What the tour establishes

PlanNode is a closed enum of node kinds. dispatch_plan_node matches every variant with no catch-all, so dispatch is exhaustive, and grep dyn Operator finds nothing, confirming operators are never trait objects. Adding an operator means a new variant and a new arm; the wildcard-free match makes the arm compiler-mandatory. The engine reserves dyn for the open IO seam and uses closed enums where it wants the compiler to enforce completeness.

// quick check

Why does the executor dispatch DAG nodes through a closed enum + exhaustive match instead of Box<dyn Operator>?

• Trait objects can't be stored in a Vec
• The node kinds are a closed set the engine owns, so a wildcard-free match gives compile-time exhaustiveness (every new operator forces a build error at each site) with no vtable cost; the open-endedness of dyn buys nothing here
• Enums are always faster than traits in every situation

// quick check

A new Dedup operator is added as a PlanNode variant but its match arm is forgotten. What happens?

• It runs but does nothing at run time
• The build fails: dispatch_plan_node's wildcard-free match is now non-exhaustive, and the compiler points at it
• It falls through to the Source arm

// quick check

MISCONCEPTION CHECK: an engineer says 'closed-enum dispatch is just the better design, so the engine should rip out Box<dyn FormatReader> in the format layer and use an enum there too.' What's wrong with that?

• Nothing: enums beat trait objects everywhere, so the format layer is a leftover mistake
• The format set is OPEN and runtime-chosen (outside code can add a wire format the engine never named), so a closed enum would defeat the seam's whole purpose; dyn is correct there and enum dispatch is correct for the closed node set
• Trait objects are deprecated in modern Rust, so both layers should use enums

// quick check

Spaced recall from the IO seam lesson: the engine stores format readers as Box<dyn FormatReader>. Why a trait object THERE, when the node executor avoids dyn entirely?

• Because formats are slower than nodes, so they need the vtable
• Because the format is chosen from the plan at run time and outside code can add new formats; an open, runtime-chosen set is exactly what dyn is for, and the per-call vtable cost is invisible next to file IO
• Because Box<dyn> is required whenever you have more than three implementors

// quick check

Spaced recall from the Value cell lesson: what does it mean that Value (and PlanNode) are CLOSED enums?

• Their variants are private and can't be pattern-matched
• The full set of variants is fixed and named by the engine in one place; outside code cannot add a new variant, which is exactly what lets a wildcard-free match be an exhaustive checklist
• They can only ever hold one value at a time across the whole program

You should be able to:

• You can explain the difference between an open, runtime-chosen set (formats) and a closed, engine-owned set (node kinds)
• You can distinguish static dispatch (enum match) from dynamic dispatch (Box<dyn Trait> + vtable) and name what each costs
• You can explain what the missing _ => arm in dispatch_plan_node guarantees
• You can state the two changes needed to add a new operator and why the compiler forces the second

Verify in the checkout:

grep -n 'pub enum PlanNode' crates/clinker-plan/src/plan/execution/mod.rs
grep -n 'fn dispatch_plan_node' crates/clinker-exec/src/executor/dispatch.rs
grep -rn 'dyn Operator' crates/ || echo 'no dyn Operator anywhere — closed-enum dispatch only'

Connections

You can now distinguish static from dynamic dispatch, read the executor’s wildcard-free match, and say why the engine has no dyn Operator anywhere, reserving trait objects for the open IO seam and closed enums for the node set it owns. Next: how the executor runs operators at once, the threads that drive each source and the bounded channels that connect them.

Prerequisites

• CXL: a staged language

→

Leads to

• The worker pool

Go deeper on the Rust (optional, one-directional, for the same concepts taught from first principles in The Rust Book):

• Want traits, trait objects, and dyn from first principles, with no engine framing? Read the docs →

Glossary terms used: static dispatch, dynamic dispatch, vtable, dyn, enum dispatch.

Mark this lesson complete