Errors & the dead-letter queue

The question for this lesson: your file has 10,000 rows and three of them are malformed. Should the whole job stop, or should it process the 9,997 good rows and set the bad ones aside? You decide. Here’s how.

What you’ll be able to do

• Choose an error strategy: fail_fast vs continue.
• Send rejected records to a dead-letter queue so good data still flows.
• Read Clinker’s exit codes to know what happened.

Two strategies

The pipeline-level error_handling: block sets the strategy:

• fail_fast (the default) stops on the first bad record. Best while you’re developing, or when any bad row means the whole batch is untrustworthy.
• continue keeps going and sends each bad record to the dead-letter queue. Best for large feeds where a few bad rows are expected and you want the good data now.

✋ Predict first

A 10,000-row file has 3 malformed rows. Under `strategy: continue` with a DLQ configured, how many rows reach the output, and where do the 3 bad ones go?

Decide on your answer before you open the reveal — committing a guess is what makes it stick.

Reveal the answer

9,997 good rows reach the output; the 3 bad rows go to the DLQ file. Under fail_fast instead, the job would stop at the first bad row and write nothing, which is useful when “3 bad rows” means something is wrong upstream that you must fix first.

Configuring the dead-letter queue

error_handling:
  strategy: continue
  dlq:
    path: ./output/errors.csv     # rejected records land here
    include_reason: true          # add a column explaining why each was rejected
    include_source_row: true      # keep the original row for debugging

🌱 New here? — why keep the bad rows?

A dead-letter queue is a to-do list, not a trash can. With include_reason and include_source_row, each rejected row arrives with what failed and the original data, so someone can fix the source and re-run just those rows. Discarding bad rows silently hides data-quality problems; the DLQ surfaces them.

Use: continue + DLQ

This pipeline keeps going past bad rows. Predict the outcome, then run it.

✋ Predict first

5 orders come in; 1 has a non-numeric amount that fails the transform. Under strategy: continue with a DLQ, how many rows reach results, and how many reach errors.csv?

Reveal the expected result

Pipeline complete: 5 total, 4 ok, 4 written, 1 dlq
exit code: 2

results.csv:
id,amount,cents
A,250,25000
B,80,8000
D,100,10000
E,40,4000

errors.csv:
_cxl_dlq_id,_cxl_dlq_timestamp,_cxl_dlq_source_file,_cxl_dlq_source_name,_cxl_dlq_source_row,_cxl_dlq_triggering_field,_cxl_dlq_triggering_value,_cxl_dlq_error_category,_cxl_dlq_error_detail,_cxl_dlq_stage,_cxl_dlq_route,_cxl_dlq_trigger,id,amount
<uuid>,<timestamp>,./data/orders_raw.csv,orders,3,cents,oops,type_coercion_failure,row 3: field 'cents': conversion failed: cannot convert 'oops' to Int,transform:total,,true,C,oops

pipeline.yaml // editable

reset ⌖ explain ✓ dry-run ▶ run

pipeline: name: orders nodes: - type: source name: orders config: name: orders type: csv path: ./data/orders_raw.csv options: { has_header: true } schema: - { name: id, type: string } - { name: amount, type: string } - type: transform name: total input: orders config: cxl: | emit cents = amount.to_int() * 100 - type: output name: results input: total config: name: results type: csv path: ./output/results.csv error_handling: strategy: continue dlq: path: ./output/errors.csv include_reason: true include_source_row: true

Modify: switch the strategy to fail_fast

Same five orders, same one bad row, but now flip the dial. In the runner above, find strategy: continue in the error_handling: block and change it to strategy: fail_fast. Leave everything else as-is, then predict what changes before you run it again.

✋ Predict first

With `strategy: fail_fast` (instead of `continue`), the run hits the same row C with the non-numeric `oops`. Does an errors.csv file get written this time? How many rows reach results.csv, and what exit code do you get?

Decide on your answer before you open the reveal — committing a guess is what makes it stick.

Reveal the answer

Under fail_fast, the run stops at the first bad row instead of setting it aside. There’s no DLQ file at all: fail_fast aborts rather than dead-letters, so nothing is routed to errors.csv. The output is partial or empty (the job stops the moment it hits row C), and the exit code is a failure of 3. The run started, then a row’s data broke the transform during evaluation, which is where a fail_fast stop lands. Compare that to continue, where the same bad row goes to the DLQ, the other four orders reach results.csv, and the job finishes with exit code 2 (completed, with rows in the DLQ). One dial, two very different outcomes: an abort versus a completed-with-caveats run.

Exit codes: what happened?

When clinker run finishes, its exit code tells you the outcome at a glance:

Code	Meaning
0	success, no rejected rows
1	config, schema, or CXL compile error (a bad type, a missing field); the job never started
2	completed, but some rows went to the DLQ
3	a CXL evaluation error, or the DLQ rate ceiling was hit; this is where a fail_fast stop on a bad row lands
4	an I/O, format, or spill error
130	interrupted (Ctrl-C)

Watch the line between 1 and 3. Code 1 means the pipeline couldn’t even be built: a type that doesn’t exist, a field you forgot. Code 3 means it built fine and started running, then a row’s data broke an expression. Under fail_fast, the first bad row stops the job and you get exit 3 (not 1). Exit 2 is the one to watch in automation: the job succeeded, but rows were rejected, so check the DLQ.

// quick check

Under `strategy: continue`, a row fails to convert `amount` to a number. What happens?

• The row goes to the DLQ with a reason; the job keeps processing other rows
• The whole job stops immediately
• The row is silently dropped with no record

Create: an error policy from scratch

Now author the policy yourself. You’re building the nightly invoices pipeline: a few malformed rows are expected, they must be quarantined for review (never silently dropped), and the job must still finish so the downstream reports run. Write the error_handling block, then predict the exit code.

error_handling:
  strategy: continue            # don't abort on a bad row...
  dlq:
    path: ./output/invoice_errors.csv
    include_reason: true        # ...record WHY each row was rejected...
    include_source_row: true    # ...and keep the original row to fix and re-run

Predict: a run processes 500 invoices and 2 fail to coerce. How many rows reach the output, how many reach invoice_errors.csv, and what exit code does clinker run return?

498 rows reach the output, 2 reach invoice_errors.csv, and the exit code is 2: completed, but some rows went to the DLQ. Exit 2 is the signal automation watches: the job succeeded, but rows need attention. (Had you chosen fail_fast, the first bad invoice would abort with exit 3 and write no DLQ file, which is wrong for this job.) Check your config first with clinker run invoices.yaml --dry-run, and the plan with --explain, before the real run.

Retrieval checkpoint

✓ Checkpoint

// quick check

A DLQ row failed at `amount.to_int()` on the text `oops`. Why does that conversion fail, and which earlier idea from lesson 02 does it exercise?

• `oops` isn't a number, so the string-to-number coercion can't succeed; it's the CXL type-coercion idea from lesson 02
• The DLQ corrupted the value, so it can't be read back
• `.to_int()` only works on the first row of a file

Common mistakes

• Using continue while developing. Early on, prefer fail_fast so you see problems immediately.
• Ignoring exit code 2 in automation. It’s a success with caveats; check the DLQ.

You should be able to:

• Choose fail_fast vs continue for a given situation and justify it
• Configure a DLQ with include_reason and include_source_row
• Read an exit code of 2 and say what to check

Verify in the checkout:

clinker run order_fulfillment.yaml --dry-run
clinker run order_fulfillment.yaml; echo "exit: $?"

Go deeper (optional)

Curious how the engine decides which errors can be recovered and which always abort? That’s the Engine Track’s Error handling and the Rust Track’s PipelineError. Optional, since your error_handling: config is all you need here.