Navigate the workspace

Clinker isn’t one file, or even one crate. It’s thirteen crates in a single workspace. That sounds like a lot to hold in your head. It isn’t, once you see that the crates are layered, and the layering itself is the map you navigate by.

Prerequisites

• The compiler loop

→

Leads to

• Read a plan with --explain

What you’ll be able to do

• Read the crate layout and name what job each layer does, from the shared vocabulary up to the command-line edge.
• Locate a symbol’s home crate by following the dependency direction as a compass, then confirm it instead of trusting it.
• Trace one pub mod line in a lib.rs to the file it names, and one crate’s dependency upward to the layer above it.
• Distinguish “the layout predicts where this lives” from “the layout guarantees where this lives,” and say which command settles the difference.

New terms in this lesson (each is named here before it appears below):

• crate: Rust’s unit of compilation, one library or one binary.
• workspace, the set of crates that build together as one project.
• module: a namespace inside a crate, usually one file, wired in by pub mod.
• dependency direction, which crate may use which; here it points one way only, upward.

Predict first

✋ Predict first

You need to find where the type Value is defined, the enum for a single record cell. You know the workspace has a clinker-record crate ("the engine's vocabulary layer") whose lib.rs lists its modules. Before reading any source: which file would you open first, and what one shell command would confirm the definition is there? Write both down.

Decide on your answer before you open the reveal — committing a guess is what makes it stick.

Reveal the answer

The layout makes this a short hop, not a hunt. Value is vocabulary, the kind of thing the lowest layer owns, so you start at crates/clinker-record/src/lib.rs, the crate’s table of contents. There you’d expect a line like pub mod value;, which says “the module value lives in value.rs.” So the file to open is crates/clinker-record/src/value.rs, and one command settles it:

grep -rn 'pub enum Value' crates/clinker-record/src/value.rs

The wrong mental model is “I’ll search the whole tree for Value”: a blind grep -rn Value crates/ over thirteen crates returns hundreds of hits (every use of the type, not its one definition). The layout lets you predict the one crate to look in, so you search a single directory and read the table of contents instead of drowning in matches. That predict-then-confirm move is the entire skill of this lesson: the layout narrows the search; a command confirms the answer.

A workspace is a set of crates

A crate is Rust’s unit of compilation, a library or a binary. A workspace groups related crates that build together and depend on one another; the top-level Cargo.toml lists the members. This is exactly why, back in the compiler loop, you could write cargo check -p clinker-record: the -p picks one crate out of the workspace to check, instead of rebuilding all thirteen.

Inside a crate, lib.rs is the table of contents: a stack of pub mod ...; lines, each naming a module (a file) and exposing it. Here’s the real top of the clinker-record crate, the engine’s vocabulary layer:

clinker-record ·lib.rs module @19acdcb4

crates/clinker-record/src/lib.rs

pub mod value;        // the Value type lives in value.rs
pub mod schema;       // Schema in schema.rs
pub mod resolver;     // FieldResolver in resolver.rs
// ... one line per module, then re-exports of the public vocabulary

pub mod value; means “there is a module value, its code is in value.rs, and it is public.” So clinker_record::value::Value is exactly where the table of contents told you it would be: the prediction you made up top, now confirmed by the source.

The layering is the map

The crates stack from low-level vocabulary up to the command-line edge. The dependency direction only ever points upward: a lower crate never depends on a higher one.

   clinker        cxl-cli            ← the binaries (edge)
   clinker-net    clinker-channel    clinker-schema
   clinker-exec                      ← runtime: executor, operators, memory, spill
   clinker-plan                      ← planning: config, validation, the DAG
   clinker-format                    ← streaming readers / writers
   cxl                               ← the CXL expression language
   clinker-record / clinker-core-types   ← the vocabulary everything shares

This direction is your compass. Reading records? That’s vocabulary → clinker-record. Parsing a pipeline? clinker-plan. Running it? clinker-exec. Decoding a CSV? clinker-format. The AI onboarding docs spell out every crate’s job; start there, then verify against the source:

clinker ·20_CRATE_MAP.md doc @19acdcb4

Read → locate → trace: three rungs on real paths

You have the map; now use it three ways, each one harder than the last. Every path below is real, so open it in your clinker checkout as you go.

Read: name the layer

Look at the stack above and read it top to bottom without opening a file. The bottom row, clinker-record / clinker-core-types, is vocabulary: the Value, Schema, and Record types everything else speaks in. The top row, clinker and cxl-cli, are binaries, the command-line edge you ran in Build & run Clinker. Everything in between is a stage of the same journey: vocabulary → expressions (cxl) → reading bytes (clinker-format) → planning (clinker-plan) → running (clinker-exec) → the edge.

That single read answers a surprising number of “where does X live?” questions before you touch a file. The job names are the index.

Locate: predict the crate, then confirm

Now a task: you want the Schema type, the named, typed columns of a record. Which crate? Schema is vocabulary, so the layout points at the bottom row, clinker-record. Open its table of contents and confirm the module is declared:

grep -n 'pub mod schema' crates/clinker-record/src/lib.rs
#   → pub mod schema;        // Schema in schema.rs

The layout predicted the crate; the grep against the real lib.rs confirmed the module. That’s the loop: predict from the map, then spend one command to be sure. Notice this is the same instinct from the compiler loop: reach for the smallest command that answers your question, not a workspace-wide sweep.

Trace: follow a line to its file, and a crate to its layer

The hardest rung is following a thread two hops. First, trace a pub mod line down to the file it names: pub mod value; in lib.rs points at crates/clinker-record/src/value.rs. Open that file and you’re looking at the Value definition itself.

Second, trace a crate up one layer. clinker-exec (runtime) sits above clinker-plan (planning), so the executor is allowed to depend on the planner, and you can see the direction in the dependency list:

grep -n 'clinker-plan' crates/clinker-exec/Cargo.toml
#   → clinker-plan = { path = "../clinker-plan" }   (exec depends on plan: upward)

There is no matching line the other way: clinker-plan/Cargo.toml does not list clinker-exec, because dependencies point upward only. Tracing that one line is how you prove the compass, rather than taking the diagram on faith.

Look it up, don’t assume

The layering predicts most locations, but not all. The clearest example: the top-level runtime error type, PipelineError, is the error the executor returns, so you’d reasonably look in clinker-exec. It actually lives in clinker-plan:

clinker-plan ·error.rs ·PipelineError type @19acdcb4

pub enum PipelineError {
    Config(ConfigError),
    Format(FormatError),
    Eval(EvalError),
    Io(std::io::Error),
    // ... ~25 variants in all — the planning layer's full failure vocabulary
}

There’s a reason (the planning layer owns the vocabulary of what can go wrong, and the executor consumes it), but the lesson for now is procedural: find the symbol, don’t guess its home. cargo doc --workspace --no-deps builds browsable docs, and a quick grep -rn "enum PipelineError" crates/ settles it in a second. The layout is a strong hint; a command is the proof.

Retrieval checkpoint

✓ Lab

Open crates/clinker-record/src/lib.rs and trace one pub mod line to the file it names.

You should be able to:

• You can name, for a given task, which crate to open, and which layer it sits in
• You located `Value`'s definition by following the table of contents, not a blind search
• You traced one dependency line upward and confirmed there's no line pointing back down
• You confirmed where `PipelineError` is defined rather than guessing

Verify in the checkout:

grep -rn 'pub enum Value' crates/clinker-record/src/value.rs
grep -n 'clinker-plan' crates/clinker-exec/Cargo.toml
grep -rln 'enum PipelineError' crates/
cargo doc --workspace --no-deps --offline

// quick check

You need the Schema type. Using only the layout as your compass, which crate do you open first, and what do you do next?

• clinker-exec, because Schema is used at runtime, and trust that it's there
• clinker-record (the vocabulary layer), then confirm with grep against its lib.rs
• Run a workspace-wide grep for Schema and read whichever file has the most hits

// quick check

PipelineError is the error type the executor returns. Which crate defines it?

• clinker-exec, the runtime crate
• clinker-plan, the planning crate
• clinker-core-types

// quick check

Spaced recall from the compiler loop: you changed one file in clinker-record and want the fastest signal that it still type-checks. Which command?

• cargo build --workspace, to rebuild all thirteen crates to be safe
• cargo check -p clinker-record, to type-check just the crate you touched
• cargo doc --workspace --no-deps, to generate the docs and read them

You can now navigate the engine by its layers: read the map, locate a symbol’s crate, and trace a line to its file or a crate to its layer.

Go deeper on the Rust (optional, one-directional): the Value type you located here is taught from first principles, as a Rust enum, in The Rust Book.

Want Value as a plain Rust enum, with no engine framing? Read the docs →

Next, the last orientation skill: reading what a pipeline will do, straight from its plan.

Mark this lesson complete