Skip to content
Clinker Academy

Engine Track — overview

The Engine Track is clinker-first. The Clinker engine is the textbook, and each Rust concept is introduced exactly where the engine forces it: you learn just enough Rust because the engine needs it, read the real source, work through the architectural reasoning, and exercise it in a live checkout. The Rust mechanism stays a light touch. When you want a concept from first principles, each lesson links out to the official Rust documentation (The Rust Book, the standard library, the Rustonomicon) rather than re-teaching it here.

The shape of the curriculum

Section titled “The shape of the curriculum”

A hybrid end-to-end + concentric progression: a shallow pass through the whole engine first, then concentric passes at increasing depth. Full design in the planning corpus under docs/curriculum-planning/.

  • Orientation: build & run Clinker, the compiler loop, workspace navigation.
  • A Record’s Journey: follow one record end-to-end, shallow.
  • Data & Representation: Value, Record, ownership, lifetimes, zero-copy.
  • Planning & Expressions: the plan→runtime boundary, traits, error taxonomy, CXL.
  • Execution & Memory: enum dispatch, the DAG executor, memory arbitration, spill.
  • Extending & Contributing: add an operator / reader / CXL builtin; review; capstones.

The architectural spine is grounded in real, revision-pinned source. Take the plan→runtime proof boundary, the typed handle the executor accepts instead of raw YAML:

clinker-plan ·compiled.rs ·CompiledPlan type @19acdcb4 
pub struct CompiledPlan {
    dag: ExecutionPlanDag,        // the lowered execution graph
    config: PipelineConfig,       // the validated configuration
    // …plus the validated compile outputs the runtime reads (typechecked CXL, bound schemas)
    // ...
}

Start here

Section titled “Start here”

All six modules are written and ready. Begin with Build & run Clinker:

  • Orientation: build the engine, run a real pipeline, work the compiler loop, navigate the crates, read an execution plan.
  • A Record’s Journey follows one record end to end: what a record is, how YAML becomes a plan, how a record moves through the DAG, and a full traced journey.
  • Data & Representation, the first deep pass: the 32-byte Value, exhaustive match, ownership and borrowing, Option/Result, provenance, collections, smart pointers, and the lifetimes behind zero-copy reads.
  • Planning & Expressions goes up a layer: the two dispatch strategies (trait objects vs generics), the typed proof handles (ValidatedPath, the strict spanned parse, CompiledPlan), the PipelineError taxonomy, and CXL as a staged parse→typecheck→eval language.
  • Execution & Memory, the deepest pass: closed-enum node dispatch, one-thread-per-source ingest with bounded channels, buffers that spill, the MemoryArbitrator’s interior mutability, bounded-memory policy, the unsafe FieldStr, and how the engine benchmarks and measures it all.
  • Extending & Contributing turns from reading to contributing: the test strategy that proves a change, then adding a CXL builtin, a reader/writer format, and a DAG operator, followed by the review gauntlet and how to plan a change without breaking an architectural boundary.
  • Capstones, the graduation: C1 adds a self-contained extension end to end with tests against a fixed rubric, and C2 scopes, plans, and delivers a real, supervised contribution through the project’s Readiness-Review → Decision-Gate → review workflow.