Core concepts

How the engine works underneath the editor and the library. Each section is the docstring of the module that implements it.

Physical quantities as socket types, with Pint.

Two ways to give a socket a unit, which work together:

  • Quantity["km/h"]: the value is a Pint quantity. Any quantity of the same dimension connects, and arrives converted to the unit the input declares, so a node taking Quantity["m"] always receives metres. A bare Quantity accepts any dimension (checked when the graph runs).

  • A NewType with a registered unit: the value stays a plain float (or array), and the unit is part of the type:

    Seconds = unit_type("Seconds", "s")        # NewType("Seconds", float) + its unit
register_unit(Days, "day")                 # for a NewType defined elsewhere

    Seconds then connects to Days (converted on the link), to Quantity["h"] (wrapped as a quantity) and back, but never to a length. A NewType without a unit stays strict: it needs an explicit converter.

Conversions happen on the link, in runs, checks, previews and exported scripts alike, the way Blender converts between socket types. Mixing dimensions is refused while editing, before anything runs.

Quantity inputs get a text widget that reads values like "9.81 m/s^2". A mixed number reads as a recipe means it ("2 1/4 cup" is 2.25 cups), and a date ("2011-03-01") is refused rather than read as 2011 - 3 - 1. Pint comes with noodlelab; it is only imported by graphs that use quantities.

noodlelab adds a few units to Pint’s registry: currencies (USD, EUR, GBP, JPY, CHF, CAD, AUD, CNY, INR), each its own dimension so that money in different currencies never mixes (convert with a rate entered as a value, 1.08 USD/EUR); individual, a dimension, so that a head count keeps its unit where Pint’s dimensionless count would vanish; and sabin / metric_sabin for acoustic absorption. Years print as yr and cups as cup rather than Pint’s a and cp.

A workspace defines its own units in units.toml, next to constants.toml, one name = "definition" per line in Pint’s syntax:

widget = "[widget]"        # a new dimension
crate = "12 * widget"
person = "individual"

Pint’s registry is shared by the whole process and cannot forget a unit, so defining a unit again the same way does nothing, while a name Pint already has, or a unit defined differently before, is an error that stops the run (restart noodlelab after changing a definition).

Measurement uncertainty, following the GUM (JCGM 100), with uncertainties.

An uncertain value is a number (or a Pint quantity) with a standard uncertainty: 9.81 ± 0.02 m/s². It is a property of the value, not of the socket, so it flows wherever the number would:

  • Every node propagates it. An uncertain value arriving at an input that takes a plain number or a quantity is handled by the framework (lift()): the node runs on the nominal values, and again with each uncertain input nudged, which gives the sensitivity coefficients (∂f/∂xᵢ) of the GUM’s law of propagation. The results carry the combined uncertainty, and correlations: two results computed from the same measurement stay correlated, so a - a is exactly 0. Node authors write nothing.

  • Uncertain in a signature asks for the uncertain value itself (to show it, or to report its budget); Uncertain["m"] asks for one in metres. Such an input may also receive an exact number, which has no uncertainty.

  • Widgets for Uncertain and Quantity inputs read 9.81 ± 0.02 m/s^2, 9.81 +/- 0.02 and the concise 9.81(2).

  • Each value made by Make Uncertain has a name, so a result can list how much each input contributed to its uncertainty (budget()).

  • Arrays carry uncertainty element by element. An uncertain array is a NumPy object array whose every element is an uncertain number (an exact element has u = 0), optionally with a unit; its elements stay correlated with the inputs they came from, as scalars do. lift() propagates into and out of arrays: a vectorized node, which works element by element, is nudged with all the elements at once (two calls, whatever the length), any other node element by element (a Jacobian, up to MAX_JACOBIAN elements; beyond that the result’s uncertainty is unknown, NaN, and Monte Carlo is the way). Arrays of more than MAX_UNCERTAIN_ELEMENTS elements are too costly to carry this way and go on at their nominal values, with a note.

Linear propagation is exact for linear models and a first-order approximation otherwise (GUM 5.1.2); results are reported with the uncertainty rounded to two significant digits and the value to the same decimal place (GUM 7.2.6).

Named constants: π, the speed of light, standard gravity, and your own.

A constant is a number someone else already fixed, so it should be named, not retyped. 299792.458 typed into a graph cannot be told from a measurement, and 9.81 might be standard gravity or a local survey. A constant carries its unit, its source and, where it was measured rather than defined, its standard uncertainty. That uncertainty then propagates like any other (see uncertainty):

  • Mathematical constants (pi, tau, e) are the math module’s, plain numbers without a unit.

  • Exact constants are defined, not measured: the SI’s since 2019 (c, h, k_B, N_A, q_e…) and conventions such as g0 and atm. They are quantities with no uncertainty.

  • Measured constants (G, m_e, alpha…) come from CODATA 2022 with their standard uncertainty. They are uncertain values named after the constant, so an uncertainty budget can say how much G contributed.

Pint knows most of these too, as units. That is why h in a unit is an hour, G is a gauss and e is an elementary charge. The names here follow the physics instead, and e stays Euler’s number as in math and in symbolic expressions. The elementary charge is q_e.

Your own constants come in layers. Each layer can add constants and replace those of the layers before it:

  1. the built-in table (BUILTIN);

  2. constants registered from Python with define() (a node pack can do this when it is imported);

  3. the workspace’s constants.toml, shared by every graph in it;

  4. the graph’s own (ExecGraph.constants), which travel with the file.

Replacing a built-in constant is allowed, since a lab may use a local value, but it is flagged unless the entry says override = true. pi, tau and e can never be replaced, because symbolic expressions fix their meaning. In constants.toml, an entry is either text or a table:

rho_water = "998.2 kg/m^3"
k_spring = "1520 ± 12 N/m"

[g_local]
value = 9.8123
unit = "m/s^2"
uncertainty = 0.0005
title = "local acceleration of gravity"
source = "gravity survey 2024, station 12"

The layers that apply to a graph make up a Scope. The executor enters it (scope()) while it checks, probes, runs or exports that graph, so nodes read it with get() and need no extra input. A node that reads constants tells the executor which ones (@my_node.uses_constants). Their definitions then go into its cache key, and editing constants.toml reruns exactly the nodes that use what changed. The scope lives in a ContextVar. asyncio.to_thread and tasks carry it along, but a bare threading.Thread does not: start threads that run node code through contextvars.copy_context().run.

Monte Carlo propagation of distributions (JCGM 101, GUM Supplement 1).

The Monte Carlo node asks the executor to run what feeds it again and again (executor.Executor._monte_carlo()): the nodes upstream of its input that depend on an uncertainty source (a Measurement, a Type A or B evaluation, a value typed with ±). In each trial every source draws a value from its own distribution (uncertainty.make() does it while a Sampler is active): normal, rectangular, triangular, or Student’s t for a mean of few readings. Everything downstream runs on plain numbers, so non-linear models and arrays are propagated exactly, which the linear (GUM) propagation only approximates. When every node re-run is vectorized (see @node(vectorized=True)), the trials run in batches, each source drawing an array, with the same samples (see Sampler).

The node then reports the mean, the standard uncertainty, a probabilistically symmetric coverage interval, and whether the GUM result is validated by the Monte Carlo one (JCGM 101, 8).

Requirements: what a design must achieve, as values the graph can use and verify.

A Requirement says that a quantity (a metric such as link_margin or dc_power) must be at least, at most, equal to or between limits, in a unit. Requirements travel through the graph as a RequirementSet, so the same statement of need is used three ways:

  • as an input: a requirement’s limit drives the design (the required bit error rate sets the required signal-to-noise ratio);

  • as a check: Requirement.verify() compares a result with it and returns a Check with the margin, which the executor reports like any check (Problems tab, noodlelab test, provenance). A result with an uncertainty is judged with its expanded uncertainty U = 2u: when the nominal value meets the limit but by no more than U, the verdict is inconclusive, which is not a pass;

  • as a filter: Requirement.margins() scores many candidates at once, for trade studies and selection.

The Requirements pack has the nodes; Verifications collects checks into a compliance matrix for a report.

Requirements are written one per line:

COM-001 link_margin >= 3 dB [Analysis]  # The link shall close with 3 dB margin
PWR-001 dc_power <= 40 W
THM-001 temperature between -20 and 60 degC
ORB-001 altitude == 550 ± 5 km

> and < are strict: a value exactly at the limit meets >= but not >. A value without a unit is taken to be in the requirement’s unit (decibels and bit rates are usually plain numbers); a quantity is converted to it. A dimensionless quantity against a requirement in decibels is refused: it could be a ratio (4, which is 6.02 dB) or a level already in dB. A temperature difference (delta_degC, as degC - degC gives it) against a limit in degC is compared as a difference; a value in K is an absolute temperature, so write limits on differences in K or delta_degC.

Checks: expectations about results that are reported without stopping the run.

A node reports a check by returning a Check (usually as one of the outputs of a NamedTuple, next to the value it checked). The executor turns each one into a node_check event, from a fresh result and from one reused from the cache alike, so a check never goes quiet; noodlelab test fails when any check fails. The Checks pack has the nodes (Expect Value, Expect In Range, Expect Table).

A check that verifies a requirement (see noodlelab.core.requirements) also says which one, what was required and achieved, and the margin: how far the value is inside the limit (negative: outside), in the requirement’s unit.

A check of an uncertain value can also be inconclusive: its nominal value meets the requirement, but not by more than its expanded uncertainty U = 2u, so the true value may well not. Such a check has passed=False and inconclusive=True, so everything that only asks whether a check passed (noodlelab test, bool(check)) treats it as not passed; Check.status tells the three verdicts apart.

Run provenance: what produced the files in a run folder.

Every run that writes outputs (into runs/<time>-<run id>/, see RunContext.path()) gets a provenance.json next to them, with everything needed to tell later how they came about:

  • the graph exactly as it was run, and who ran it, when, and with what result;

  • the environment: noodlelab, Python and platform, and the version of every installed distribution (including the node packs);

  • per node: its type, the hash of its source code, its settings and links, its Merkle key, and whether it was computed in this run or reused (with the run that originally computed it, for results restored from a checkpoint);

  • fingerprints of the files each node read: size, modification time and, for local files up to a size limit, a SHA-256 of the content;

  • the files the run wrote, with their sizes and hashes.

The record is plain JSON, meant to be read by people and tools alike. It is written when the run ends, whether it succeeded, failed or was cancelled; the files are fingerprinted then too, in a worker thread.

Graph documents.

The editor keeps its own layout (litegraph’s serialisation: positions, sizes, groups, colours) but executes a small, editor-agnostic description:

{"nodes": [
    {"id": "1", "type": "core.number", "inputs": {"value": {"value": 2.5}}},
    {"id": "2", "type": "core.math",
     "inputs": {"a": {"link": {"node": "1", "output": "result"}},
                "b": {"value": 3}, "operation": {"value": "multiply"}}}
]}

A saved GraphDocument contains both, so a graph can be reopened in the editor and run headless with noodlelab run graph.json.

Subgraphs. A node of type "subgraph" holds a graph of its own, like a Blender node group. Its inputs and outputs are ports: inside, a link to {"node": "..", "output": "signal"} reads the subgraph’s input signal, and each output names the inner output it forwards. Outside, it is linked like any node:

{"id": "5", "type": "subgraph", "title": "Clean signal",
 "inputs": {"signal": {"link": {"node": "1", "output": "result"}}},
 "subgraph": {
    "inputs": [{"name": "signal"}],
    "outputs": [{"name": "clean", "link": {"node": "2", "output": "result"}}],
    "nodes": [{"id": "2", "type": "science.savgol_filter",
               "inputs": {"y": {"link": {"node": "..", "output": "signal"}}}}]}}

Before anything checks or runs a graph, flatten() dissolves subgraphs into ordinary nodes with path-like ids ("5/2": node 2 inside node 5), so the executor, health checks, probes, checkpoints and the exporter see a flat graph, and problems and results are reported against the inner nodes.

The report. The editor shows the report in a tab of its own, and a graph keeps it apart as well: report holds the report’s nodes, whose links point at other report nodes, or, with "tab": "processing", at the output of a processing node:

{"nodes": [{"id": "5", "type": "maths.xy_plot", ...}],
 "report": [
    {"id": "1", "type": "report.new_report", "inputs": {...}},
    {"id": "2", "type": "report.add_figure",
     "inputs": {"report": {"link": {"node": "1", "output": "result"}},
                "figure": {"link": {"node": "5", "output": "result",
                                    "tab": "processing"}}}}]}

Data flows one way: processing nodes cannot link into the report. flatten() merges the report in with ids like "report/2".

Repeat zones. A node of type "repeat" runs its body again and again, like Blender’s repeat zone. Its state items are fed back: each pass starts from what the previous one produced, next naming the body output that gives the next value. The zone’s inputs of the same names give the first values, and iterations (or mode: "until" with a boolean until output, up to max_iterations) says how often it runs:

{"id": "7", "type": "repeat", "title": "Time steps",
 "inputs": {"iterations": {"value": 100},
            "T": {"link": {"node": "3", "output": "result"}}},
 "repeat": {
    "state": [{"name": "T", "next": {"node": "5", "output": "result"}, "collect": true}],
    "nodes": [{"id": "5", "type": "science.cool",
               "inputs": {"T": {"link": {"node": "..", "output": "T"}},
                          "k": {"link": {"node": "2", "output": "result"}}}}]}}

Unlike a subgraph, a zone’s body shares the ids of the graph around it, so body nodes read outer nodes directly ("2" above), and {"node": "..", "output": "iteration"} is the pass number, from 0. Outside, {"node": "7", "output": "T"} is the final value, and iterations, converged and history (the collected values of every pass, by state name) describe the run. flatten() turns a zone into ordinary nodes: the driver 7 (core.repeat), a pass-through 7:T (core.repeat_state) per state item, 7:iteration (core.repeat_iteration) and the body, wired as for the first pass, and describes it in ExecGraph.zones for the executor.

Tracked values: node outputs followed from run to run.

A graph lists the outputs it tracks in its metadata (tracked: node, output and optionally a field path, with a label). After each run of the graph, TrackLog.record() appends one line per run to .noodlelab/tracked/<graph>.jsonl in the workspace: for every tracked output its value (for a single number, with its unit and uncertainty, and for a complex one its real and imaginary parts too; see noodlelab.core.provenance.scalar_value()), a short summary, whether the run computed it or reused it, and which run computed it when. The editor’s Tracked tab reads the history back with TrackLog.history().

The history describes runs, so it goes with them: deleting a run removes its line (JsonlLog.forget_runs()), and JsonlLog.clear() starts a graph’s history again.

Requirements followed from run to run.

After each run of a graph, RequirementLog.record() appends one line to .noodlelab/requirements/<graph>.jsonl in the workspace. The line lists every requirement the run knew about, by id:

  • the requirements a node produced (a Requirements or Load Requirements node’s set), which are not verified until some node checks them;

  • the verdict of every check of a requirement (see checks): passed, inconclusive or failed, what was required and achieved, the margin and uncertainty, and where it came from. That is the node and output that checked it, whether the run computed the check or reused it, and which run computed it when.

The editor’s Requirements tab reads it back with RequirementLog.latest() (the state of each requirement now) and RequirementLog.history().

Files: the workspace folder plus pluggable remote storage (“mounts”).

A mount gives remote storage a short name: lab-s3 → s3://my-bucket/raw with its credentials. Any fsspec backend works: s3:// (s3fs), gs:// (gcsfs), az:// (adlfs), sftp:// (paramiko), smb://, http(s)://, file:// for a network share mounted on the server, or your own fsspec implementation. noodlelab needs only fsspec itself (the remote extra); install the backends you use.

Nodes take files with the FileRef type, a string that names a file:

"data/run1.csv"               relative: inside the workspace
"/mnt/nas/run1.csv"           absolute: on the server (local mode only)
"lab-s3://2026/run1.csv"      on the mount named lab-s3

from noodlelab import FileRef, node

@node(category="Tables")
def load_csv(path: FileRef) -> pd.DataFrame:
    with path.open() as f:              # works for every backend
        return pd.read_csv(f)

path.local_path() gives a real local file for libraries that insist on one (downloaded into a cache for remote mounts). pathlib.Path inputs keep working, but can only hold local paths. Both get a browse button in the editor.

The framework treats file inputs specially, so nodes need not: the editor checks that the file exists (and has an extension from Param(accept=...)), probes are re-run when the file changes, and cached results are keyed by the file’s size and modification time, so readers stay cacheable. See file_token().

Mount names and credentials are configured by an administrator; users see mount names, never the underlying URLs or secrets, and the browser API can only list the workspace and configured mounts. Outside the server (scripts, exported graphs) a FileRef also accepts plain fsspec URLs.