# Remote storage

Data rarely lives next to the graphs. A server can give remote storage a short
name, a **mount**, so that graphs read `lab-s3://2026/run1.csv` instead of a
bucket URL with credentials in it. People see the mount's name and browse it in
file pickers; they never see the URL or the secrets.

## Adding a mount

Open **Admin → Storage → Add storage** and fill in:

| Field | What it is |
|---|---|
| **Name** | The short name used in file references, `name://path`: lowercase letters, digits and `-`. |
| **Label** | What the file picker shows (optional). |
| **URL** | The root of the storage, as any [fsspec](https://filesystem-spec.readthedocs.io) URL. |
| **Storage options** | A JSON object passed to the backend: credentials, endpoint, region. Secrets are never shown again once saved. |
| **Writable** | Whether nodes may save files there. Mounts are read-only unless this is on. |
| **Workspaces** | The workspaces that can use it. None ticked means all of them. |

**Test connection** lists the mount's root with the settings as they are on
screen, before anything is saved.

Some examples:

| Storage | URL | Storage options |
|---|---|---|
| S3, MinIO, Ceph, R2 | `s3://bucket/prefix` | `{"key": "...", "secret": "...", "endpoint_url": "https://minio.example.org"}` |
| Google Cloud Storage | `gs://bucket/prefix` | `{"token": "/secrets/gcs.json"}` |
| Azure Blob | `az://container/prefix` | `{"account_name": "...", "account_key": "..."}` |
| SFTP | `sftp://host/home/data` | `{"username": "...", "password": "..."}` |
| A share mounted on the server | `file:///mnt/nas/lab` | `{}` |

## Installing the backends

noodlelab needs only `fsspec` itself (the `remote` extra, included in `full`).
Each kind of storage needs its fsspec backend installed where runs happen:

```bash
pip install "noodlelab[full,s3]"      # AWS S3 and compatible stores (s3fs)
pip install "noodlelab[full,gcs]"     # Google Cloud Storage (gcsfs)
pip install "noodlelab[full,azure]"   # Azure Blob and Data Lake (adlfs)
pip install "noodlelab[full,sftp]"    # SFTP (sshfs)
```

Any other fsspec implementation works too (`smb://`, `http(s)://`, your own):
install it, and use its URL.

## Mounts from the environment

For containers that get their credentials from the environment, set
`NOODLELAB_MOUNTS` to a JSON list of mounts with the same fields:

```bash
NOODLELAB_MOUNTS='[{"name": "lab-s3", "url": "s3://lab-data/raw",
  "options": {"key": "...", "secret": "..."}, "workspaces": []}]'
```

These show up in the admin portal marked *from NOODLELAB_MOUNTS* and can only be
changed there. Mounts added in the portal are saved in the server's
`config.json` (in `NOODLELAB_HOME`), credentials included, so keep that folder
readable only by the server's user.

## Using files in nodes

A node takes a file with the `FileRef` type, a string that names it:

```text
data/run1.csv               relative: inside the workspace
/mnt/nas/run1.csv           absolute: on the server (local mode only)
lab-s3://2026/run1.csv      on the mount named lab-s3
```

Absolute paths and paths that leave the workspace (`../other`) work only on your
own machine, without logins. On a shared server (`--auth users`) and the demo,
graphs reach files inside their workspace and, on a shared server, on the mounts
an admin configured: add a folder on the server as a mount with the URL
`local:///mnt/nas/lab` (or `file:///mnt/nas/lab`) and refer to it as
`nas://run1.csv`.

```python
from noodlelab import FileRef, node


@node(category="Tables")
def load_csv(path: FileRef) -> pd.DataFrame:
    with path.open() as f:  # works for every backend
        return pd.read_csv(f)
```

`path.local_path()` gives a real local file for libraries that insist on one,
downloaded into a cache for remote mounts. The editor checks that the file
exists, previews re-run when it changes, and cached results are keyed by the
file's size and modification time, so nodes that read files stay cacheable.
A run's provenance fingerprints every file it read: size, modification time
and, for local files up to a size limit, a SHA-256 of the content.

Outside the server (scripts, exported graphs), a `FileRef` also accepts plain
fsspec URLs such as `s3://bucket/key`. On a Slurm cluster, jobs use the mounts
of the server's `config.json` when `NOODLELAB_HOME` is on the shared file system
(see [Slurm clusters](slurm.md)).