Glossary

These definitions describe the current engine and the Rust concepts used in the lessons.

For runnable pipeline syntax, follow the Pipeline track; standalone Rust exercises may use simplified types.

adapter

An iterator method that returns another iterator (.filter, .map, .take). Adapters are lazy: they wrap the chain without running it. Contrast with a consumer.

agent-ready

An implementation issue with bounded scope, observable acceptance criteria, resolved design decisions, and verification grounded in source and tests.

arbitration

The plan’s memory policy: the order of strategies (e.g. BackPressurePreferred then Priority) the engine would use to stay inside its memory budget under pressure.

arbitrator

Clinker’s MemoryArbitrator: one Arc-shared object every operator thread charges bytes against; it tracks the budget and decides who must spill, pause, or abort when memory runs tight.

arc

Shared ownership of one heap value across many owners (and threads); Arc::clone copies the handle and bumps a count, not the value.

atomic

An integer or bool (AtomicU64, AtomicBool, …) whose reads and writes are indivisible at the hardware level, so many threads can update it through &self with no lock and no data race. fetch_add and friends are single lock-free CPU instructions.

backpressure

When a downstream node is slower than its upstream, a streaming engine lets the slow node set the pace rather than piling up unread records.

binding

A pattern that names the payload it matches so the arm’s body can use it — like Integer(n) capturing the i64 as n. Writing (_) matches the payload but binds nothing.

blast radius

The full set of crates, boundaries, and behaviors a change reaches. Scoping a change means tracing its blast radius before writing code, so a surprise dependency or a crossed boundary is found on paper, not in review.

borrow

Safety first: a reference lets you use a value without taking ownership; the borrow checker enforces either many readers or one writer, never both. Performance is the consequence: because borrows don’t copy, they’re also cheap.

bounded-memory

Resource limits enforced through admission, accounting, backpressure, spill, and explicit refusal. Logical allocation bounds and sampled process RSS are different measurements; the configured limit is not an instantaneous OS-enforced RSS cap.

box

An owning pointer to heap storage. Box<T> for a sized T is one pointer wide; a box of a dynamically sized type also carries metadata.

buffer

A blocking operator’s held set of records — the rows it must keep around while it works, before it can emit anything.

cargo-check

A cargo command that type-checks the code without generating a runnable binary, so it returns far faster than a full build. The every-few-seconds signal.

cell-refcell

Single-threaded interior mutability. A Cell swaps a whole value in and out through &self; a RefCell hands out &mut at run time and panics if you break the one-writer rule. Neither is thread-safe — across threads you reach for an atomic or a Mutex instead.

change scoping

The pre-implementation step of mapping a change’s blast radius against the invariants and the Decision-Gate triggers, to decide whether it’s a direct edit or an escalation.

channel

In pipeline authoring, a cataloged deployment overlay selected by ID; declared variables, config, and supported structural operations customize a target. In runtime Rust code, a channel is a queue between producers and consumers. These are different concepts.

ci-gate

One pass/fail check inside CI (fmt, a clippy pass, the test suite, a license audit). A red gate blocks the merge.

clippy

Rust’s lint tool. Beyond compile errors, it flags style and correctness smells; Clinker runs it as a required gate.

clone

An explicit, possibly-expensive deep copy you opt into with .clone(). Sometimes the right call; over-cloning to silence the borrow checker is a known anti-pattern.

collect

The consumer that drains an iterator into a collection (often a Vec). Materializes every element at once — the opposite of streaming, so the engine avoids it on the hot path.

compile-time validation

Checking a fact (the plan is fully validated and lowered) by encoding it in the type system, so a violation is a program that does not compile — not a runtime guard.

condvar

A condition variable: a thread parks on it (releasing a paired Mutex) until another thread signals, instead of busy-spinning. Used to block a worker until a state change — here, to pause a source until the arbitrator resumes it.

crate

Rust’s unit of compilation: one library or one binary, built as a whole. The smallest thing cargo compiles on its own.

cxl

Clinker’s small expression language for computing/reshaping fields inside a pipeline (emit, let, filter, distinct, …).

dag

The directed acyclic graph the nodes form: records flow forward along the wires, never in a cycle.

Decision Gate

A recorded decision point for choices an agent cannot validate from existing source/tests/docs — new dependencies, public-API or schema changes, cross-crate boundaries — with chosen vs rejected options, evidence, and consequences.

deny-warnings

The flag -D warnings, which promotes every compiler/clippy warning to a hard error so warnings can’t accumulate unnoticed.

dependency-direction

Which crate is allowed to use which: in a layered workspace, dependencies point upward only — a lower crate never depends on a higher one. This direction is what makes the layout a reliable map.

derive

An attribute (#[derive(…)]) that asks the compiler to auto-generate a trait implementation — Debug for printing, Clone for copying — instead of writing it by hand.

derive-macro

A procedural macro invoked with #[derive(…)] that generates code at compile time from the type it is attached to. thiserror::Error generates the Display, Error, and From impls for an error enum.

diagnostic

The compiler’s structured report on a problem: an error code like E0425, a precise source location, and usually a suggested fix.

DLQ

The dead-letter queue: the side-channel where rejected records go under a non-fail-fast strategy, so one bad row doesn’t sink the whole job.

dry-run

With —dry-run alone, plan and typecheck without reading records. Add -n N to execute at most N records per Source into an explicit preview destination instead of configured Sink paths.

dyn

The keyword that forms a trait object: a value typed by the trait it satisfies, not its concrete type. Written boxed as a Box of dyn Trait, it pairs the data with a vtable for run-time dispatch.

dynamic-dispatch

Calls resolved at run time through a vtable: the data carries a pointer to the table of methods for its actual type, and the call follows it. What a boxed dyn Trait gives you.

enum

A type that is exactly one of a fixed, named set of kinds. In Rust each kind (variant) may carry its own typed data, so an enum is a sum type, not merely named integers.

enum-dispatch

Routing over a fixed, named set of kinds with one match on a closed enum. Each arm is resolved statically, and a wildcard-free match makes a new variant a compile error at every site.

error-strategy

Some errors can be routed to a dead-letter queue and processing continues; others always abort. Whether a given error is recoverable is a policy decision (fail_fast vs continue), not an inherent property of the error type — except Internal, which always aborts.

error strategy / recoverable vs fatal

Some errors can be routed to a dead-letter queue and processing continues; others always abort. Whether a given error is recoverable is a policy decision (fail_fast vs continue), not a property of the error type — except Internal, which always aborts.

error-trait

std::error::Error is the standard trait every error type implements. Its one notable method, source(), returns the lower-level error that caused this one, so tools can walk a chain of causes.

exhaustiveness

A match over a closed enum must cover every variant; leaving one out is a compile error (non-exhaustive patterns), not a runtime surprise.

exit-code

The integer a process returns on exit: 0 means success, and distinct non-zero codes name distinct failure kinds a caller can branch on.

field

One named, typed component of a struct, accessed with dot syntax (value.field). Every value of the struct has every field.

fieldstr

Clinker’s 24-byte string type for record fields: a union of an inline arm (short bytes stored in place) and a heap arm (an Arc when shared, a Box when unique), discriminated by a trailing tag byte.

fixture

A committed test input or expected output (e.g. an .expected.csv under tests/fixtures/) that a test loads rather than building inline.

gauntlet

The full set of CI checks every change must pass before it can merge — run as parallel jobs, each with its own gate.

generic

Code written once against an unknown type and reused for many concrete types. A generic struct or function names a type parameter and works for every type that satisfies its bounds.

golden-snapshot

A committed reference output (a snapshot or golden file). The test re-runs the operation and compares byte-for-byte against the committed reference; an intentional change is reviewed and re-accepted.

group

Rows bucketed by a key for aggregation or windowing.

heap

The run-time memory pool you allocate from for values whose size or lifetime doesn’t fit the stack’s strict in-and-out discipline. Reached through a pointer; freed when its owner is dropped.

impl

An impl block (impl Trait for Type) supplies the method bodies that make a concrete type satisfy a trait.

integration-test

A test compiled as a separate crate under tests/, so it can only call the crate’s public API — forcing it to drive the engine the way a user does.

interior-mutability

A type that lets you mutate the value behind a shared & reference, moving the borrow checker’s one-writer rule from compile time to a run-time or hardware-enforced check. The basis of Cell, RefCell, Atomic*, Mutex, RwLock, and ArcSwap.

invariant

A condition that must always hold for code to be correct — here, a fact an unsafe block relies on (e.g. these bytes are valid UTF-8). The SAFETY comment writes it down; the constructor or a layout assertion establishes it.

iterator

A value that produces a sequence one element at a time on demand via next(), yielding Some(item) until it returns None at the end. The engine’s readers are iterators over records.

lazy

An iterator computes nothing until a consumer pulls from it. Adapters like .filter/.map only describe work; no element flows until something asks for one.

lifetime

A name for how long a borrow is valid — the compiler’s bookkeeping that a reference never outlives the data it points to.

match

A control-flow expression that compares a value against a sequence of patterns and runs the first arm that matches. Over a closed enum, the compiler checks that the patterns cover every variant.

module

A namespace inside a crate, usually one file; pub mod name; in lib.rs declares that the file name.rs is part of the crate and exposes it.

monomorphization

The compile-time step that stamps out a separate, specialised machine-code copy of generic code for each concrete type it is used with. The reason generic calls cost nothing extra at run time.

move

Assigning or passing a non-Copy value transfers ownership; the source can no longer be used.

mutex

A lock guarding shared data: lock() blocks until this thread has exclusive access and returns a guard whose &mut deref is the protected value; dropping the guard unlocks. Serializes every accessor, so it fits cold paths, not per-record hot ones.

newtype

A struct that wraps a single existing type to give it a new, distinct identity — e.g. struct ValidatedPath(PathBuf). The wrapper is a different type from what it wraps, even though it holds exactly one value.

node

One step in a pipeline DAG. Sources read, intermediate nodes transform or combine records, and Sinks write. Composition calls lower into a sub-pipeline.

notional machine

The predict-with-it model of how a pipeline runs.

option

A value that is either Some(T) or None — absence encoded in the type, so there is no null.

oracle

A second, trusted-but-slow implementation a fast implementation is checked against: they must always agree. The slow one is ‘self-evidently correct’; the fast one is the optimization under test.

ownership

Every value has exactly one owner; when the owner goes away, the value is freed. This is how Rust guarantees memory safety without a garbage collector.

pipeline

A Clinker job described in YAML: a set of nodes wired into a graph that reads from sources, transforms records, and writes outputs.

plan

The compiled, validated execution plan Clinker produces from a pipeline before running it — proof the job is well-formed, printed by —explain.

plan/runtime boundary

The typed boundary between planning and execution. CompiledPlan proves planning succeeded; current executor entry points still recompile its retained config before dispatch, so the handle alone does not establish stored-DAG reuse.

proof token

A value whose very existence is compile-time evidence that some check passed. Possible only when the type’s constructor is the sole gate (a private inner field), so holding the type proves the check ran.

property-test

A test that generates many random inputs and checks an invariant holds on every one, rather than asserting a single fixed example.

provenance

The origin metadata a record carries — which file and row it came from, in which batch, ingested when — so errors and dead-letter entries can name the exact source.

Readiness Review

The project’s grounding step: before writing code you state what the work is, the source/tests/docs you inspected, current-vs-desired behavior, and a recommended route.

record

One row: positional values owned by OwnedValues and a shared schema held through SharedStorage<Schema>, plus provenance and document context.

reference count

The integer Arc (or Rc) keeps alongside the shared value, counting how many owning handles exist. Every clone adds one; every drop subtracts one; the value is freed when it reaches zero.

reference-validity

A reference is valid only while the value it points to is still alive; reading through it after the value is dropped is a dangling reference, which Rust rejects at compile time.

result

Success (Ok) or failure (Err) encoded in the type; ? propagates errors, converting via From.

result-option

A composed type read as three outcomes: Ok(Some(x)) = a value; Ok(None) = clean end-of-input; Err(e) = something went wrong. Build it from Option and Result separately first.

retire-gate

A source-text test that fails CI if a deliberately retired identifier (an old module name, a withdrawn diagnostic code) reappears in the sources — so removed code stays removed.

review-checklist

The list a contributor self-applies to their own diff before requesting review — scope, hidden API/schema/auth changes, weakened tests, captured follow-ups — to catch what CI cannot.

runtime / executor

The hot, parallel, memory-bounded core that pulls records and runs the plan per record. It accepts only a CompiledPlan — never raw config.

schema

Named, typed columns. The declared source schema controls parsing, validation, and downstream CXL types; runtime ingestion does not guess types.

seam

A boundary where one component talks only to a contract, so a new implementation can plug in without the component changing. Traits are how Clinker builds seams.

Send

An auto-trait meaning ownership of this value may move to another thread. A non-Send type cannot be handed to thread::spawn — the compiler rejects it.

sender/receiver

The two endpoints of a channel: the sender (tx) puts values in, the receiver (rx) takes them out. Splitting the channel into two handles is what lets the producer and consumer live on different threads.

shared ownership

One value with several genuine, independent owners at once — the opposite of the single-owner rule. Rc and Arc are how Rust grants it safely.

soundness

A piece of unsafe code is sound when there is no possible input or program state that makes it trigger undefined behaviour. Soundness is a property of the whole module, not one line: the constructors and assertions must establish every invariant the unsafe blocks assume.

source

The Error::source() method returns the underlying error that caused this one (or None). Overriding it links your error to its cause, building a chain a logger can print top to bottom.

source-position

A line-and-column coordinate into the original text; the engine keeps it 1-based, the way an editor numbers lines.

span

The source location attached to a parsed value — its line and column in the original file — kept so an error can point back at the exact spot.

spanned

A wrapper that pairs a parsed value with its source span, so the value still knows the line and column it came from.

spill

Pushing a buffer’s records out of RAM to disk so they no longer count against the memory budget; the records drain back in order later.

static-dispatch

Calls resolved at compile time to a concrete method, so they are direct and inlinable with no vtable. A match arm on a closed enum is resolved this way.

streaming

Processing records incrementally without retaining the complete input at that boundary. A pipeline can also contain blocking or materialized boundaries, whose retained state must be budgeted.

streaming-mode

A run mode the plan reports: records can flow straight through node by node, without the engine collecting them all before emitting.

struct

A type that bundles several named fields that always travel together — a product type. A struct value holds all of its fields at once, where an enum value is exactly one variant.

sum type

A type formed as a choice between alternatives — a value is one variant or another, never several at once. Rust enums are sum types; this is what makes them more than named integers.

Sync

Rust auto-trait: T is Sync when &T can be sent between threads. Arc<T> is thread-safe only when the contained type satisfies the required Send and Sync bounds.

thiserror

A widely used crate providing a derive macro for error types. #[derive(thiserror::Error)] generates the Display, std::error::Error, and From impls for an enum from attributes, at compile time, with no runtime cost.

thread

An OS-scheduled line of execution within a process. Clinker spawns one per source so reading and decoding run in parallel with the dispatcher.

tier

A customer-tier field a transform adds to each record (e.g. gold/silver); used here as a concrete example of a node enriching the record as it passes.

trait

A named contract of method signatures a type promises to provide; it carries no data of its own. Many unrelated types can satisfy the same trait.

trait-bound

A constraint on a type parameter (S: RecordStorage) that says S must implement that trait. Inside the generic code you may call only what the bound promises.

trait-method

One method declared in a trait. A required trait method has no body — every implementor must supply one; a default-bodied one may be overridden.

trait-object

A value typed by the trait it satisfies, not its concrete type — written as a boxed dyn Trait. It pairs the data with a vtable so the right method is found at run time.

type-parameter

A placeholder type named in a type’s or function’s angle brackets (the S in RecordView) that the compiler fills in with a concrete type at each use site.

type-state

Encoding a fact about a value’s state into its type, so the compiler tracks the state for you. A ValidatedPath is the ‘screened’ state of a path; an unscreened path has a different type.

unit-test

A test that exercises one small unit of behavior in isolation. In clinker these live inline in a #[cfg(test)] mod tests block and can reach the file’s private functions.

unsafe

A block or fn where you may perform operations the compiler cannot verify (deref a raw pointer, read a union field, call from_utf8_unchecked); in exchange you take on responsibility for an invariant the compiler would otherwise check.

value

One typed cell. The ten variants are Null, Bool, Integer, Float, Decimal, String, Date, DateTime, Array, and Map.

variant

One of the named kinds an enum can be. A unit variant carries no data; a tuple variant carries one or more typed payloads.

vec

A growable, contiguous, heap-owning array — Rust’s default list. Indexed by position in O(1); owns its elements and frees them when dropped.

vtable

A hidden table of function pointers — one per method — that a trait object carries so a call can find the right concrete method for the value’s actual type at run time.

watermark

A threshold or progress marker whose meaning depends on context. Memory watermarks drive pressure responses; source/document progress markers describe input progress. Neither implies unbounded event streaming.

workspace

A set of related crates that build together and share one lockfile; the top-level Cargo.toml lists the members.

zero-copy

Reading a value through a borrow without duplicating it; the data stays where it is and the reader looks through a reference. A copy is paid only where a value must outlive its source.

match arm

One pattern and its result expression in a Rust match. Exhaustive matches cover every possible variant.

from attribute

The thiserror field attribute that derives a From conversion from an underlying error and marks that error as the source.

From trait

Rust’s infallible conversion trait. Implementing From<A> for B provides conversion from A to B, including error conversion used by the question-mark operator.