Benchmark & measure memory

All of Execution & Memory has rested on a claim: this is cheaper. A 24-byte FieldStr saves an allocation; the arbitrator keeps memory bounded; spilling trades RAM for disk. None of that is worth anything unless you can measure it. This final lesson is about clinker’s measurement tools, and they come in two flavours that are often conflated: a fast estimate the runtime uses to budget memory, and an exact count the benchmarks use to verify it.

Prerequisites

• FieldStr

→

Leads to

• Extending & Contributing (Overview)

What you’ll be able to do

• Explain what heap_size estimates, why scalars and short inline strings cost 0, and why an estimate (not a real allocator probe) is the right tool for the live budget.
• Read a criterion benchmark and name what black_box does and why omitting it would time nothing.
• Distinguish the two measurement tools by job: a cheap estimate for the live budget versus an exact count for offline verification.
• Write a custom GlobalAlloc that counts allocated bytes and forwards to the system allocator, and predict the byte delta a single allocation produces.

New terms in this lesson (each is also expanded inline at the point you first need it):

• benchmark: a repeatable timing experiment, run many times for a stable number.
• criterion: clinker’s benchmarking library; reports a timing distribution, not one noisy number.
• black_box: the fence that stops the optimizer deleting the work you are timing.
• peak RSS: the high-water mark of physical RAM the process holds.
• throughput: records processed per unit time; the thing the budget must not wreck.

Predict first

✋ Predict first

Two ways to size a record. heap_size walks a Value and sums an estimate of owned heap bytes; AccountingAlloc intercepts the real allocator and counts every byte the process actually requests. For one Value holding a 12-byte inline FieldStr (a string short enough to live inside the 24-byte field type), which number is larger, the heap_size estimate or the exact allocated-byte count, and why? Commit to one before revealing.

Decide on your answer before you open the reveal — committing a guess is what makes it stick.

Reveal the answer

Trick framing: for that value, both can be 0, but they get there for different reasons, and in general the exact count is the larger, noisier number.

• heap_size for a short inline FieldStr returns 0: the bytes live inside the 24-byte field type, so the value owns no separate heap allocation. The estimate is reading a structural fact.
• The exact allocator count, measured around constructing that value, can also be 0 if no heap request happened. But in a real run it sweeps up everything: temporary buffers, capacity over-allocation, allocations from code you didn’t write. It measures events, not ownership.

The wrong mental model is “heap_size and the allocator count are two ways to get the same number, so use whichever is handy.” They answer different questions. heap_size asks how much heap does this value own right now?, a cheap, deterministic structural estimate. The allocator count asks how many bytes did this region of code actually request from the OS?, which is exact, but it includes transient allocations and costs real time to collect. Confusing them is the misconception this whole lesson exists to prevent.

The cost model: heap_size

Recall the arbitrator and bounded memory in action: operators charge bytes against the memory budget. What is a record’s byte cost? The engine estimates it with heap_size: owned heap bytes, accounted per Value variant:

clinker-record ·value.rs ·heap_size fn @19acdcb4

/// Estimated heap bytes owned by this value (excludes the enum itself).
pub fn heap_size(&self) -> usize {
    match self {
        Value::String(s) => s.heap_size(),                 // 0 if inline, else byte length
        Value::Array(arr) => {
            arr.capacity() * std::mem::size_of::<Value>()
                + arr.iter().map(Value::heap_size).sum::<usize>()
        }
        Value::Map(m) => m.iter().map(|(k, v)| k.len() + v.heap_size()).sum(),
        _ => 0,                                            // scalars live inline — no heap
    }
}

Two things to read here. Scalars (Integer, Bool, dates) return 0: they live inline in the 32-byte Value, owning no heap. And a short inline FieldStr also returns 0, which is the 24-byte type from unsafe & FieldStr paying off in the cost model directly. Crucially, this is an estimate, computed by walking the value, not a real allocator measurement. It has to be: charging runs on the hot path, per record, per operator, and you cannot attach an allocator probe to every value without wrecking the throughput you’re trying to bound. A cheap, consistent estimate is the right tool for a runtime budget.

The benchmark suite: criterion

For the other job, proving a change actually made things faster or smaller, clinker uses criterion benchmarks. They live in each crate’s benches/ directory: record_ops (record create/get/set/clone, value_heap_size), arbitration_poll, and more.

clinker-exec ·arbitration_poll.rs ·bench_should_spill bench @19acdcb4

// crates/clinker-record/benches/record_ops.rs — the criterion shape
fn bench_value_heap_size(c: &mut Criterion) {
    let string = Value::String("a medium length string".into());
    c.bench_function("value_heap_size/string", |b| {
        b.iter(|| black_box(string.heap_size()));   // black_box stops the optimizer
    });
}

black_box is the load-bearing detail: it hides its argument from the optimizer, so the compiler can’t “see through” the benchmark and delete the work you’re trying to time. You run these with cargo bench -p clinker-record --bench record_ops (or --bench arbitration_poll for the arbitrator-poll cost). The arbitration_poll suite, for instance, times should_spill across consumer-registry sizes to keep the arbitrator’s per-poll cost from regressing as pipelines deepen.

Evidence note

There is no automated timing-threshold gate in this checkout. For instance, arbitration_poll’s “~5µs/call” budget is a documented expectation a human reads off the criterion report, not an asserting test. CI checks that the benches still compile and run (a smoke matrix), not that they hit a number.

The exact count: a custom global allocator

An estimate budgets; a count verifies. To measure the real bytes a section allocates, clinker ships a counting global allocator behind the bench-alloc feature:

clinker-bench-support ·alloc.rs ·AccountingAlloc type @19acdcb4

pub struct AccountingAlloc { allocs: AtomicUsize, bytes_alloc: AtomicUsize, /* ... */ }

// SAFETY: every call forwards to the System allocator after counting.
unsafe impl GlobalAlloc for AccountingAlloc {
    unsafe fn alloc(&self, layout: Layout) -> *mut u8 {
        self.allocs.fetch_add(1, Ordering::Relaxed);
        self.bytes_alloc.fetch_add(layout.size(), Ordering::Relaxed);
        unsafe { System.alloc(layout) }     // delegate the real work
    }
    unsafe fn dealloc(&self, ptr: *mut u8, layout: Layout) {
        /* count, then */ unsafe { System.dealloc(ptr, layout) }
    }
}

Install it with #[global_allocator] and every allocation in the process flows through it, bumping atomic counters (the interior-mutability pattern from the arbitrator, and the unsafe impl discipline from unsafe & FieldStr). A scoped Region snapshots the counters before and after a block to report exactly how many bytes it allocated; that’s how the executor’s per-stage heap_delta_bytes metric is captured under the feature. The cost is real (it adds contention per allocation), so it’s a measurement build, not the production path.

So the two tools divide the labour cleanly: heap_size estimates, cheaply, for the live budget; AccountingAlloc counts, exactly, for offline verification. Confusing the two, whether by budgeting off real allocation or benchmarking off the estimate, would get you the worst of each.

Worked → completion → faded

You now build the counting allocator yourself, scaffolded down to your own. One new idea per rung: first read a complete one, then close a single gap, then write the counting from scratch.

Worked: a counting allocator you can run

The Rust playground lets you install a global allocator, so you can build a miniature AccountingAlloc right here: same unsafe impl GlobalAlloc, same forward-to-System pattern. Run it and watch the byte delta land on exactly the capacity you asked for.

rust // editable

reset ▶ run

use std::alloc::{GlobalAlloc, Layout, System};
use std::sync::atomic::{AtomicUsize, Ordering};

struct Counter;
static ALLOCS: AtomicUsize = AtomicUsize::new(0);
static BYTES: AtomicUsize = AtomicUsize::new(0);

// SAFETY: forwards every call to System after bumping the counters.
unsafe impl GlobalAlloc for Counter {
  unsafe fn alloc(&self, layout: Layout) -> *mut u8 {
      ALLOCS.fetch_add(1, Ordering::Relaxed);
      BYTES.fetch_add(layout.size(), Ordering::Relaxed);
      System.alloc(layout)
  }
  unsafe fn dealloc(&self, ptr: *mut u8, layout: Layout) {
      System.dealloc(ptr, layout)
  }
}

#[global_allocator]
static A: Counter = Counter;

fn main() {
  let before = BYTES.load(Ordering::Relaxed);
  let v: Vec<u8> = Vec::with_capacity(1024);   // exactly one heap allocation
  let after = BYTES.load(Ordering::Relaxed);
  println!("bytes the Vec allocated: {}", after - before);
  println!("total allocations so far: {}", ALLOCS.load(Ordering::Relaxed));
  std::hint::black_box(&v);
}

use std::alloc::{GlobalAlloc, Layout, System}; use std::sync::atomic::{AtomicUsize, Ordering}; struct Counter; static ALLOCS: AtomicUsize = AtomicUsize::new(0); static BYTES: AtomicUsize = AtomicUsize::new(0); // SAFETY: forwards every call to System after bumping the counters. unsafe impl GlobalAlloc for Counter { unsafe fn alloc(&self, layout: Layout) -> *mut u8 { ALLOCS.fetch_add(1, Ordering::Relaxed); BYTES.fetch_add(layout.size(), Ordering::Relaxed); System.alloc(layout) } unsafe fn dealloc(&self, ptr: *mut u8, layout: Layout) { System.dealloc(ptr, layout) } } #[global_allocator] static A: Counter = Counter; fn main() { let before = BYTES.load(Ordering::Relaxed); let v: Vec<u8> = Vec::with_capacity(1024); // exactly one heap allocation let after = BYTES.load(Ordering::Relaxed); println!("bytes the Vec allocated: {}", after - before); println!("total allocations so far: {}", ALLOCS.load(Ordering::Relaxed)); std::hint::black_box(&v); }

The Vec::with_capacity(1024) shows up as a 1024-byte allocation in the delta, an exact count, not an estimate. This is clinker’s AccountingAlloc in miniature: a GlobalAlloc impl that counts and forwards.

Completion: close the counting gap

Here the dealloc side is written; the alloc side counts allocations but forgets to add the bytes. Fill the one missing line so BYTES tracks the real allocated size. Predict the printed delta before you run it.

rust // editable

reset ▶ run

use std::alloc::{GlobalAlloc, Layout, System};
use std::sync::atomic::{AtomicUsize, Ordering};

struct Counter;
static ALLOCS: AtomicUsize = AtomicUsize::new(0);
static BYTES: AtomicUsize = AtomicUsize::new(0);

// SAFETY: forwards every call to System after bumping the counters.
unsafe impl GlobalAlloc for Counter {
  unsafe fn alloc(&self, layout: Layout) -> *mut u8 {
      ALLOCS.fetch_add(1, Ordering::Relaxed);
      // TODO: add layout.size() to BYTES so the byte count is real
      System.alloc(layout)
  }
  unsafe fn dealloc(&self, ptr: *mut u8, layout: Layout) {
      System.dealloc(ptr, layout)
  }
}

#[global_allocator]
static A: Counter = Counter;

fn main() {
  let before = BYTES.load(Ordering::Relaxed);
  let s = String::from("0123456789");          // 10 bytes — one heap allocation
  let after = BYTES.load(Ordering::Relaxed);
  println!("bytes this String allocated: {}", after - before);
  std::hint::black_box(&s);
}

use std::alloc::{GlobalAlloc, Layout, System}; use std::sync::atomic::{AtomicUsize, Ordering}; struct Counter; static ALLOCS: AtomicUsize = AtomicUsize::new(0); static BYTES: AtomicUsize = AtomicUsize::new(0); // SAFETY: forwards every call to System after bumping the counters. unsafe impl GlobalAlloc for Counter { unsafe fn alloc(&self, layout: Layout) -> *mut u8 { ALLOCS.fetch_add(1, Ordering::Relaxed); // TODO: add layout.size() to BYTES so the byte count is real System.alloc(layout) } unsafe fn dealloc(&self, ptr: *mut u8, layout: Layout) { System.dealloc(ptr, layout) } } #[global_allocator] static A: Counter = Counter; fn main() { let before = BYTES.load(Ordering::Relaxed); let s = String::from("0123456789"); // 10 bytes — one heap allocation let after = BYTES.load(Ordering::Relaxed); println!("bytes this String allocated: {}", after - before); std::hint::black_box(&s); }

💡 Hint 1

The pattern is already next door in dealloc’s sibling: a fetch_add on the atomic. You want BYTES.fetch_add(layout.size(), Ordering::Relaxed);, where layout.size() is the byte count of this request.

Show solution

unsafe fn alloc(&self, layout: Layout) -> *mut u8 {
    ALLOCS.fetch_add(1, Ordering::Relaxed);
    BYTES.fetch_add(layout.size(), Ordering::Relaxed);   // the missing line
    System.alloc(layout)
}

String::from("0123456789") requests at least 10 bytes, so the delta prints 10 (it can be larger if the allocator rounds the request up; the count is exact for what was requested, which is the point). Without the missing line, BYTES never moves and the delta is 0 even though a real allocation happened: a silent measurement that always reports zero. That is the failure mode the fetch_add exists to prevent.

Faded: write the counting yourself

Now the scaffolding is gone from the body. The dealloc side is given; write the whole alloc body so it counts both the allocation event and the byte size, then forwards to System. The #[global_allocator] wiring and main are done; only the alloc body is yours.

rust // editable

reset ▶ run

use std::alloc::{GlobalAlloc, Layout, System};
use std::sync::atomic::{AtomicUsize, Ordering};

struct Counter;
static ALLOCS: AtomicUsize = AtomicUsize::new(0);
static BYTES: AtomicUsize = AtomicUsize::new(0);

// SAFETY: this impl must forward to System; it may only COUNT before delegating.
unsafe impl GlobalAlloc for Counter {
  unsafe fn alloc(&self, layout: Layout) -> *mut u8 {
      // TODO: bump ALLOCS by 1, add layout.size() to BYTES, then return System.alloc(layout)
      todo!()
  }
  unsafe fn dealloc(&self, ptr: *mut u8, layout: Layout) {
      System.dealloc(ptr, layout)
  }
}

#[global_allocator]
static A: Counter = Counter;

fn main() {
  let before = BYTES.load(Ordering::Relaxed);
  let v: Vec<u8> = vec![0u8; 100];             // 100 bytes — one heap allocation
  let after = BYTES.load(Ordering::Relaxed);
  println!("bytes allocated: {}", after - before);
  println!("allocations:     {}", ALLOCS.load(Ordering::Relaxed));
  std::hint::black_box(&v);
}

use std::alloc::{GlobalAlloc, Layout, System}; use std::sync::atomic::{AtomicUsize, Ordering}; struct Counter; static ALLOCS: AtomicUsize = AtomicUsize::new(0); static BYTES: AtomicUsize = AtomicUsize::new(0); // SAFETY: this impl must forward to System; it may only COUNT before delegating. unsafe impl GlobalAlloc for Counter { unsafe fn alloc(&self, layout: Layout) -> *mut u8 { // TODO: bump ALLOCS by 1, add layout.size() to BYTES, then return System.alloc(layout) todo!() } unsafe fn dealloc(&self, ptr: *mut u8, layout: Layout) { System.dealloc(ptr, layout) } } #[global_allocator] static A: Counter = Counter; fn main() { let before = BYTES.load(Ordering::Relaxed); let v: Vec<u8> = vec![0u8; 100]; // 100 bytes — one heap allocation let after = BYTES.load(Ordering::Relaxed); println!("bytes allocated: {}", after - before); println!("allocations: {}", ALLOCS.load(Ordering::Relaxed)); std::hint::black_box(&v); }

💡 Hint 1

Three statements, in order: ALLOCS.fetch_add(1, ...), BYTES.fetch_add(layout.size(), ...), then System.alloc(layout) as the final expression (no semicolon, so it’s the return value). Count before you delegate so the counters never lag.

Show solution

unsafe fn alloc(&self, layout: Layout) -> *mut u8 {
    ALLOCS.fetch_add(1, Ordering::Relaxed);
    BYTES.fetch_add(layout.size(), Ordering::Relaxed);
    System.alloc(layout)            // delegate the real work, return its pointer
}

vec![0u8; 100] is one heap request of 100 bytes, so the delta prints 100 (the ALLOCS total is higher, because println! and friends allocate too, and that is exactly why a real measurement uses a scoped Region to subtract the surrounding noise). You’ve now written clinker’s AccountingAlloc in miniature: count, then forward. Swap vec![0u8; 100] for String::from("...") or Vec::with_capacity(512) and watch the bytes track precisely, an exact count, never an estimate.

Solid on the cost model, criterion, and the counting allocator? Skip ahead →

Retrieval checkpoint

✓ Checkpoint: benchmarking & measuring

💡 Hint 1

Compare the two: heap_size walks a value and sums an estimate; AccountingAlloc intercepts the real allocator. Which one runs on every record during a job, and which only under a bench build?

What the tour establishes

heap_size is a per-variant estimate of owned heap bytes (scalars and short inline strings cost 0), cheap enough to charge against the budget per record. Criterion benches (record_ops, arbitration_poll) time real operations, with black_box defeating the optimizer; you run them with cargo bench. AccountingAlloc is an unsafe impl GlobalAlloc that counts every allocation and forwards to System, used under the bench-alloc feature for exact measurement. Estimate for the live budget, exact count for verification: different tools, different jobs.

// quick check

Why does the runtime budget charge memory using heap_size (an estimate) instead of the exact AccountingAlloc count?

• AccountingAlloc is less accurate
• heap_size is a cheap per-value estimate suited to the hot path, where charging runs per record per operator; measuring real allocation per value would wreck the throughput the budget exists to protect, so the exact count is reserved for offline benchmarking
• heap_size and AccountingAlloc always return the same number

// quick check

What is black_box used for in a criterion benchmark?

• To hide the benchmark's results from the report
• To stop the optimizer from seeing through the benchmarked expression and deleting the work, which would make the measurement meaningless
• To run the benchmark on a background thread

// quick check

Two engineers describe heap_size and AccountingAlloc. Which statement is the MISCONCEPTION to correct?

• heap_size estimates owned heap bytes by walking a value; AccountingAlloc counts real allocation events through a GlobalAlloc impl
• They are two ways to get the same number, so for a record's memory you can use whichever is handier; they're interchangeable
• heap_size runs per record on the hot path; AccountingAlloc only runs under a bench-feature measurement build

// quick check

Spaced recall from bounded memory in action: when should_spill trips on peak RSS or summed charged bytes, what does the arbitrator do, and where does the byte figure it charges come from?

• It immediately aborts the run; the figure is the exact AccountingAlloc count
• It runs one arbitration round (poll_arbitration) that asks the policy for a victim to pause or spill; the per-record bytes charged come from the cheap heap_size estimate, not a real allocator probe
• It pauses every operator permanently; the figure is read from the criterion report

You should be able to:

• You can explain what heap_size estimates and why an estimate (not real allocation) is right for the live budget
• You can read a criterion bench and say what black_box is for
• You can distinguish heap_size (cheap estimate for the budget) from AccountingAlloc (exact count for verification) by job
• You wrote a GlobalAlloc that counts bytes and forwards to System, and predicted the byte delta of a single allocation

Verify in the checkout:

grep -n 'fn heap_size' crates/clinker-record/src/value.rs
grep -n 'struct AccountingAlloc\|impl GlobalAlloc' crates/clinker-bench-support/src/alloc.rs
grep -n 'fn bench_should_spill' crates/clinker-exec/benches/arbitration_poll.rs
cargo bench -p clinker-record --bench record_ops -- value_heap_size

Connections

That’s the end of Execution & Memory. You’ve gone from “the plan is validated” to the live machine that runs it: closed-enum dispatch, one thread per source feeding bounded channels, buffers that spill, an arbitrator that bounds memory through interior mutability, the unsafe core of the field string, and the tools that measure all of it. The thread tying Planning & Expressions and Execution & Memory together: push proof to the boundary, then run hard inside it, with the type system, the budget, and the benchmarks each enforcing a different guarantee. Extending & Contributing turns you from reader to contributor: adding an operator, a format, a CXL builtin, and passing the review gauntlet.

Mark this lesson complete