Enum vs trait dispatch

Welcome to Phase 4 — Execution & Memory, the deepest pass. Phase 3 ended at the plan/runtime boundary; now we cross it and watch records actually move. This first lesson revisits a question from Phase 3 — “how do you call into one of many kinds?” — and finds the engine answering it the opposite way from the IO seam. Back in lesson 3.1 the format layer used Box<dyn FormatReader>: dynamic dispatch, because formats are an open-ended, runtime-chosen plug-in seam. The DAG executor faces the same shape of problem — many kinds of node — and deliberately chooses a closed enum and one exhaustive match instead.

Prerequisites

• CXL: a staged language

→

Leads to

• Threads & channels

You’ll be able to: explain when closed-enum dispatch beats trait-object dispatch, read the executor’s central match, and say why the engine has no dyn Operator anywhere.

The closed set of node kinds

A compiled plan is a DAG of nodes, and a node is one of a fixed, engine-known set of kinds: a source, a transform, a route, a merge, a sort, an aggregation, an output, and a handful more. That set is a closed enum — the same PlanNode you glimpsed in Phase 3:

clinker-plan ·mod.rs ·PlanNode type @47d2e12

pub enum PlanNode {
    Source { /* ... */ },
    Transform { /* ... */ },
    Route { /* ... */ },
    Merge { /* ... */ },
    Sort { /* ... */ },
    Aggregation { /* ... */ },
    Output { /* ... */ },
    // ... 13 variants in all — the complete vocabulary of pipeline nodes
}

The key word is closed. The kinds of node a pipeline can contain are decided by the engine, not by users, not at run time. Contrast that with formats: anyone can add a new wire format, and which one a job uses is read from the plan at run time. Formats are open; node kinds are closed. That single difference drives the whole dispatch decision.

One exhaustive match, no dyn

Because the set is closed, the executor dispatches with a single match over the enum. Each arm hands the node to its operator module:

clinker-exec ·dispatch.rs ·dispatch_plan_node fn @47d2e12

pub(crate) fn dispatch_plan_node(
    ctx: &mut ExecutorContext<'_>,
    current_dag: &ExecutionPlanDag,
    node_idx: NodeIndex,
) -> Result<(), PipelineError> {
    let node = current_dag.graph[node_idx].clone();
    match node {
        PlanNode::Source { .. }      => dispatch_source(ctx, current_dag, node_idx, &node)?,
        PlanNode::Transform { .. }   => dispatch_transform(ctx, current_dag, node_idx, &node)?,
        PlanNode::Route { .. }       => dispatch_route(ctx, current_dag, node_idx, &node)?,
        PlanNode::Merge { .. }       => dispatch_merge(ctx, current_dag, node_idx, &node)?,
        // ... one arm per variant — and crucially, NO `_ =>` catch-all
    }
    Ok(())
}

There is no trait object here and none anywhere in the engine: searching the whole codebase for dyn Operator or trait Operator finds nothing. Operators are not boxed behind a vtable; they’re arms of a match. And there is no _ => wildcard — the match spells out every variant. That second detail is doing real architectural work, as the next section shows.

Why closed-enum dispatch wins here

Lesson 3.1 argued dynamic dispatch was right for the IO seam. Here the trade flips, for three concrete reasons:

• Exhaustiveness is a feature. Because the match has no catch-all, adding a new PlanNode variant makes every non-exhaustive match a compile error — the compiler hands you the exact list of sites to update (exactly the guarantee from lesson 2.2). With dyn Operator, a forgotten case would be a run-time surprise, not a build failure.
• The set is closed and known at compile time. A dyn seam exists to let outside code plug in types the engine never named. Node kinds are all named by the engine itself, so the open-endedness dyn buys is worthless here — and you’d pay a vtable indirection per node for it.
• Operators want specialized data. Each arm can pull the variant’s own payload (a sort’s keys, a transform’s compiled program) by pattern-matching, with no downcasting.

The mental model: dyn is for open sets chosen by others at run time; a closed enum is for sets the engine owns and wants the compiler to police. The IO seam is the former; the operator set is the latter. Same language feature family (traits and enums), opposite tool for opposite jobs.

Evidence note

The source encodes this choice structurally — there is no dyn Operator, and the dispatch match is wildcard-free — but carries no comment stating the rationale, and the planning docs in this checkout reference no ADR for it. The reasoning above is the standard Rust trade-off read off the code, not a quote from it.

Feel the exhaustiveness

Here are both strategies side by side. The enum version makes the compiler your checklist; the trait-object version quietly accepts a new kind with no nudge:

rust // editable

reset ▶ run

// CLOSED ENUM + exhaustive match — the executor's strategy.
enum Node { Source, Transform, Output }

fn dispatch(n: &Node) -> &'static str {
  match n {
      Node::Source => "read",
      Node::Transform => "evaluate",
      Node::Output => "write",
      // no `_ =>` — add a 4th variant and THIS match stops compiling
  }
}

fn main() {
  for n in [Node::Source, Node::Transform, Node::Output] {
      println!("{}", dispatch(&n));
  }
  // Try this: add `Sort` to the enum. The compiler immediately flags
  // `dispatch` as non-exhaustive — it hands you the site to update.
}

// CLOSED ENUM + exhaustive match — the executor's strategy. enum Node { Source, Transform, Output } fn dispatch(n: &Node) -> &'static str { match n { Node::Source => "read", Node::Transform => "evaluate", Node::Output => "write", // no `_ =>` — add a 4th variant and THIS match stops compiling } } fn main() { for n in [Node::Source, Node::Transform, Node::Output] { println!("{}", dispatch(&n)); } // Try this: add `Sort` to the enum. The compiler immediately flags // `dispatch` as non-exhaustive — it hands you the site to update. }

Add a Sort variant and the build breaks at dispatch, pointing at the exact code that doesn’t yet handle it. That is the compiler acting as a complete, always-current checklist of “every place a new operator must be wired in.” A Box<dyn Operator> design would have compiled fine and failed later, at run time, when an unhandled node showed up.

Solid on enum vs trait-object dispatch? Skip ahead →

// quick check

Why does the executor dispatch DAG nodes through a closed enum + exhaustive match instead of Box<dyn Operator>?

• Trait objects can't be stored in a Vec
• The node kinds are a closed set the engine owns, so a wildcard-free match gives compile-time exhaustiveness (every new operator forces a build error at each site) with no vtable cost — the open-endedness of dyn buys nothing here
• Enums are always faster than traits in every situation

Read the dispatcher

✓ Checkpoint — enum vs trait dispatch

💡 Hint 1

Read the match node block in dispatch_plan_node to the end — is there a _ => arm? Then run the last grep: does any dyn Operator exist to compare against?

What the tour establishes

PlanNode is a closed enum of node kinds. dispatch_plan_node matches every variant with no catch-all, so dispatch is exhaustive — and grep dyn Operator finds nothing, confirming operators are never trait objects. Adding an operator means a new variant and a new arm; the wildcard-free match makes the arm compiler-mandatory. The engine reserves dyn for the open IO seam and uses closed enums where it wants the compiler to enforce completeness.

// quick check

A new Dedup operator is added as a PlanNode variant but its match arm is forgotten. What happens?

• It runs but does nothing at run time
• The build fails: dispatch_plan_node's wildcard-free match is now non-exhaustive, and the compiler points at it
• It falls through to the Source arm

You should be able to:

• You can explain the difference between an open, runtime-chosen set (formats) and a closed, engine-owned set (node kinds)
• You can explain what the missing _ => arm in dispatch_plan_node guarantees
• You can state the two changes needed to add a new operator and why the compiler forces the second

Verify in the checkout:

grep -n 'pub enum PlanNode' crates/clinker-plan/src/plan/execution/mod.rs
grep -n 'fn dispatch_plan_node' crates/clinker-exec/src/executor/dispatch.rs
grep -rn 'dyn Operator' crates/ || echo 'no dyn Operator anywhere — closed-enum dispatch only'

You’ve seen how the executor decides which operator to run. Next: how it runs them at once — the threads that drive each source and the bounded channels that connect them.

Mark this lesson complete