10  Debugging and Error Handling

In the previous chapters, you learned functional programming fundamentals and how to build reproducible pipelines with {rixpress}. Now we’ll add another layer of robustness: error handling and debugging.

When a pipeline runs, things inevitably go wrong. A dataset might have unexpected missing values. A computation might divide by zero. A join might drop more rows than expected. When these issues occur, the first question is always: where did it happen, and what do we do about it?

T answers that question at three levels:

By the end of this chapter, you’ll know how to write pipelines that fail gracefully, how to read the diagnostics T produces, and how to use the interactive debugger to get inside a failing node.

10.1 Errors Are Values

In most languages, an exception is an interruption: it unwinds the call stack, and unless someone catches it, the program dies. T takes a different view. An error in T is just another value — a first-class citizen, like a number, a string, or a DataFrame — and, like any other value, it can be passed around, inspected, and transformed.

Consider the most classic failure of all:

err = 1 / 0

In a strict language this would crash. In T, the expression evaluates to an error value, and the program continues. You can ask the error value questions:

err = 1 / 0
print(is_error(err))
print(error_code(err))
print(error_msg(err))
print(error_context(err))
true
DivisionByZero
Division by zero.
{}
  • is_error(x) returns true if x is an error value, false otherwise.
  • error_code(x) returns the machine-readable code, here DivisionByZero.
  • error_msg(x) returns the human-readable message.
  • error_context(x) returns a dict of context attached to the error. It is empty for a bare arithmetic failure, but errors raised inside functions can carry stack information.

This is what makes T resilient by default: a failing expression does not stop the world, it produces a value that flows downstream, and every function downstream gets a chance to notice it and react.

10.2 Resilient Builds and Fail-Fast

That resilience extends to whole pipelines. When a node’s code raises an error, T captures it as a first-class value, marks the node accordingly, and — by default — keeps building every branch of the pipeline that does not depend on the failed node.

Every node in a build ends up in one of four states:

Status What it means
Completed The node ran successfully and serialized its output artifact.
Completed with error The node’s code ran but raised a user-space exception. T captured the error as a value, and independent branches kept building.
Errored The sandbox itself failed — a syntax error, a missing package, running out of memory. This is a hard failure that aborts the build.
Skipped The node was never run because an upstream dependency suffered a hard failure.

The important distinction is between the first two rows. A Completed with error node is a soft failure: the code ran, the error is a value, and the build carries on. An Errored node is a hard failure: Nix’s build machinery itself gave up, and everything downstream is marked Skipped.

If you want the opposite behaviour — stop at the first error — T has a fail-fast mode:

t run --failfast src/pipeline.t

From the REPL you can set t_run(failfast = true, ...) for a single run, or declare t_make(failfast = true) when defining a pipeline. For most data pipelines the default resilient mode is what you want: you get as many diagnostic outcomes as possible from a single pass, and you decide afterwards which failures actually matter.

10.3 Pipes: Propagate or Recover

T has two pipe operators, and the difference between them is the error-handling story.

The ordinary pipe |> is short-circuiting: if the value on the left is an error, the function on the right is not applied, and the error flows through untouched. The maybe-pipe ?|> always forwards the value — errors included — to the function on the right, giving that function a chance to look at the error and recover.

doubled = 5 |> \(x: Int -> Int) x * 2
print(doubled)
e = 10 / 0
propagated = e |> \(x: Int -> Int) x + 1
print(is_error(propagated))
print(error_code(propagated))
recover = \(x: Any -> Int) if (is_error(x)) 0 else x
print(e ?|> recover)
print(5 ?|> recover)
10
true
DivisionByZero
0
5

Three things to notice. First, |> on an error passes the error straight through: propagated is still the original DivisionByZero error, which means the x + 1 function was never applied. Second, ?|> hands the error to recover, which inspects it with is_error and substitutes a default. Third, ?|> is not only for errors: when the value is a normal one, the function runs exactly as it would with |>.

Use |> for the majority of your pipelines, where you expect success and want any error to keep propagating until something explicitly handles it. Use ?|> at the specific points where you want to act on an error — log it, replace it with a fallback, or transform it.

10.4 Recovering from Errors

The most declarative way to recover is pattern matching. match lets you destructure an error value and branch on what went wrong:

caught = match(1 / 0) { Error { m } => str_join(["Caught: ", m], ""), _ => "no error" }
print(caught)
safe = match(1 / 0) { Error { _ } => 0, _ => 1 }
print(safe)
fallback = match(42) { Error { m } => str_join(["Caught: ", m], ""), _ => 99 }
print(fallback)
Caught: Division by zero.
0
99

The Error { m } pattern binds the error’s message to m, so you can build a diagnostic string, write it to a log, or feed it into a fallback computation. The _ pattern catches everything else — in the last example, the successful value 42 — and returns it (or a default) unchanged.

From there, the recovery patterns fall out naturally:

  • Default values: ?|> \(x: Any -> Int) if (is_error(x)) 0 else x replaces any failure with a sensible default.
  • Fallbacks: try the expensive computation first, and if it fails, fall back to a cheaper approximation — the match above is exactly this shape.
  • Log and continue: print error_msg(x) inside the error branch, then return a placeholder so the rest of the pipeline can run and you get a full diagnostic report from one pass.
  • Log and rethrow: print the message, then error(error_code(x), error_msg(x)) to keep the failure visible to whoever is upstream of you.

For a single, uniform recovery, a ?|> with an if (is_error(x)) is the lightest tool. When different error types need different responses, or when the recovery logic has more than one branch, match is clearer.

10.5 When a Pipeline Node Fails

Let’s put it together. The pipeline below has three nodes: one that succeeds, one that divides by zero, and one more that succeeds.

p = pipeline {
  nums = [10, 0, 30]
  bad = 1 / 0
  ok = 42
}
build_pipeline(p)
print(is_error(p.bad))
print(error_code(p.bad))
print(error_msg(p.bad))
print(is_error(read_node(p.bad)))
T execution failed:
Error running t (error code 1): true 
DivisionByZero 
Division by zero. 
{} 
10 
true 
DivisionByZero 
0 
5 
Caught: Division by zero. 
0 
99 

false
DivisionByZero
Division by zero.
true

Because the build is resilient, build_pipeline(p) completes: nums and ok are built, and bad is marked Completed with error. The build summary you see on the console names the failing node and even suggests the next command to run.

The four print statements show how you interrogate the result. Note the subtlety: is_error(p.bad) is false, because the node is not an error — it is a node that contains an error. The error lives in the node’s artifact. That is why error_code(p.bad) and error_msg(p.bad) work directly on the node (they look inside it), and why is_error(read_node(p.bad)) is the way to check the node’s value: read_node deserializes the artifact, and that value is indeed an error.

10.6 Inspecting a Failed Build

Beyond per-node checks, T gives you build-wide views of what went wrong.

collect_exceptions(p) gathers every captured error in the pipeline into a DataFrame with a row per failing node:

collect_exceptions(p)
T execution failed:
Error running t (error code 1): true 
DivisionByZero 
Division by zero. 
{} 
10 
true 
DivisionByZero 
0 
5 
Caught: Division by zero. 
0 
99 

  node  status  code            message
  ----  ------  --------------  -----------------
  bad   Error   DivisionByZero  Division by zero.
DataFrame: 1 rows x 4 cols

In a long pipeline with several independent failures, this is the fastest way to see the full damage report in one place.

Chapter 8 introduced build_log(p) and build_log_to_frame(), which turn the most recent build into a DataFrame with one row per node — name, status, duration, and artifact path:

bl = build_log(p)
build_log_to_frame(bl)
T execution failed:
Error running t (error code 1): true 
DivisionByZero 
Division by zero. 
{} 
10 
true 
DivisionByZero 
0 
5 
Caught: Division by zero. 
0 
99 

  name  status                duration  path
  ----  --------------------  --------  -----------------------------------
  nums  Completed             0.8792    /nix/store/ds36srq9gb2q49bhsri1z...
  bad   Completed with error  0.8896    /nix/store/ds36srq9gb2q49bhsri1z...
  ok    Completed             0.8686    /nix/store/ds36srq9gb2q49bhsri1z...
DataFrame: 3 rows x 4 cols

The status column is exactly the four-state vocabulary from earlier in this chapter, and the duration column tells you which nodes are slow even when they succeed. (One gotcha: assign the log to a variable that is not already a T builtin — log is the natural logarithm — before passing it to build_log_to_frame.)

Finally, inspect_node reports the metadata T recorded for a single node:

inspect_node(p.bad)
T execution failed:
Error running t (error code 1): true 
DivisionByZero 
Division by zero. 
{} 
10 
true 
DivisionByZero 
0 
5 
Caught: Division by zero. 
0 
99 

dict
├── name: "bad"
├── runtime: "T"
├── path: "/nix/store/lvxlc8a6bll2hlwnbxyj7vbvxyw3f3wf-pipeline_output/bad/artifact"
├── serializer: "default"
├── class: "Error"
├── dependencies: []
└── warnings: []

The class field is the node’s value type as T sees it: Int for a healthy node, Error for a soft failure. Combined with the build log, this is usually all you need to triage a failed build without ever opening a debugger.

10.7 Interactive Debugging

Sometimes the build log is not enough. The classic case: a Python node fails with a cryptic traceback, the real inputs are locked in the Nix store, and writing mock inputs by hand is tedious and error-prone. For that, T has an interactive debugger that drops you inside the failing node’s runtime, with all of its upstream dependencies already materialized.

10.7.1 The t debug Command

From your shell, run:

t debug <node_name>

By default this looks for the pipeline in src/pipeline.t; if your pipeline lives elsewhere, pass the file first:

t debug src/my_custom_pipeline.t data_cleanup

T evaluates the script up to the requested node, resolves every upstream dependency, locates their latest materialized /nix/store paths, sets up the environment, and launches the runtime’s interactive REPL. Targeting a Python node, for example, produces something like:

================================================
Debugging Node: Y (Runtime: Python)
================================================
Environment variables set for dependencies:
  - dataset_np = /nix/store/5fcfj6wfh...-pipeline_output/dataset_np

Starting interactive Python REPL...
Tip: Load upstream dependencies in Python using:
  import tlang
  dataset_np = tlang.read_node("dataset_np")
Press Ctrl+D or exit to return to T REPL.
================================================

10.7.2 debug_node() from the REPL

If you are already inside a T REPL session with a built pipeline, the same debugger is one call away:

debug_node(p.data_cleanup)

This spawns the same subshell environment. Inside it, a tlang companion library lets you load upstream node data live:

py> import tlang
py> df = tlang.read_node("raw_df")
py> df.dtypes

Now you can inspect the exact data that crashed your node, dry-run the offending function line by line, and confirm the fix. Press Ctrl+D (or exit(), or q() in R) to leave the subshell; control returns cleanly to your T session.

The subshells are customized so you always know where you are: Python starts with a py> prompt, R starts quietly (no welcome banner) with r>, and Julia shows jl>. Interactive debugging is supported for the three REPL-capable runtimes — Python, R, and Julia; debugging a Quarto or Bash node raises a descriptive ValueError instead of dropping you into a raw shell.

10.7.3 Keeping the Debug Environment Reproducible

The debugger runs inside the project environment declared in tproject.toml. If a runtime package needed to deserialize an upstream artifact is missing, the debugger will start without it. Use t doctor to spot missing packages and add them to the right section:

  • [r-dependencies].packages for R packages
  • [py-dependencies].packages for Python packages
  • [jl-dependencies].packages for Julia packages

For example, a Julia node that reads a CSV ancestor needs CSV and DataFrames; one that reads Arrow or Parquet needs Arrow (or Parquet2) and DataFrames; a Python node reading Arrow data needs pandas and pyarrow. After editing tproject.toml, run t update and re-enter nix develop so the debug environment is rebuilt. Do not install these packages ad hoc with pip, install.packages(), or Pkg.add() — those commands are intercepted inside the debugger precisely to keep your debugging reproducible.

10.8 Polyglot Error Handling

The same rules apply to nodes written in R, Python, or Julia. When a foreign script raises a native exception — an R stop(), a Python ValueError, a Julia UndefVarError — T captures it exactly as it captures a T error: the node is marked Completed with error, and error_code, error_msg, collect_exceptions, and the build log all work on it as before.

p2 = pipeline {
  data = node(
    command = to_dataframe([
      x = [1.0, 2.0, 3.0],
      y = [0L, 0L, 0L]
    ])
  )
  model = rn(
    command = <{
      data$y <- as.factor(data$y)
      glm(y ~ x, data = data, family = binomial(link = "logit"))
    }>,
    deserializer = ^ipc,
    serializer = ^pmml
  )
}
build_pipeline(p2)

Here the response variable has only one level, so the logistic regression fails inside the R sandbox. The build does not abort; model is a soft failure, and error_msg(p2.model) carries R’s original message. And because model is an R node, debug_node(p2.model) drops you into an r> subshell with the exact data frame that caused the trouble.

The one polyglot-specific habit worth forming: keep error messages in your foreign code informative. A bare stop() in R or a bare raise in Python gives you an Error with little to go on; a message like stop("no rows left after filtering") is what error_msg will hand you later, and what you will thank yourself for at 2 a.m.

10.9 Common Errors

A few failures show up in nearly every pipeline. Knowing the idiomatic fix saves a lot of debugging:

  • Missing values: mean([1, 2, NA, 4]) raises an NAError. Pass na_rm = true when NAs are expected, or filter them deliberately.
  • Type mismatches: "Age: " + 25 raises a TypeError because T does not implicitly coerce. Convert explicitly with to_string() (or str_join).
  • Empty collections: mean([]) raises a ValueError. Check length(data) == 0 first and return a default.
  • Division by zero: guard the denominator — if (count == 0) 0.0 else total / count — or let it fail and recover downstream with ?|> or match.
  • Missing columns: df.nonexistent raises a NameError. Check colnames(df) before accessing, and raise a descriptive error of your own when a required column is absent.

10.10 Best Practices

  • Fail fast, fail explicitly. Validate inputs early and raise descriptive errors (error("ValidationError", "Empty dataset")) at the boundary where you can say why it is invalid, rather than letting a cryptic failure surface ten nodes later.
  • Let soft failures stay soft. In a long pipeline, prefer the default resilient build so one bad node does not hide three others; use collect_exceptions to review everything at once.
  • Keep recovery at the edges. Use |> in the middle of your pipelines and concentrate ?|>/match recovery where a fallback is genuinely meaningful.
  • Name your errors. A good error_msg is a unit of documentation: it should tell the next reader what was expected and what they found.
  • Debug with real data. When in doubt, debug_node beats a mock: the subshell gives you the exact upstream artifacts your node saw.

10.11 Summary

T treats errors as first-class values, and that single decision shapes everything in this chapter. is_error, error_code, error_msg, and error_context let you interrogate a failure; |> propagates errors untouched while ?|> hands them to a recovery function; and match gives you declarative branching over error and success alike. At the pipeline level, resilient builds capture node failures as Completed with error without aborting, and collect_exceptions, build_log, and inspect_node turn any build — healthy or not — into data you can query. When the data is not enough, t debug and debug_node drop you into the failing node’s own runtime, with its real inputs, ready to step through the code that broke.

Next, we turn to distribution: how to package all of this into containers so that others can run your pipelines without installing anything at all.