# nenudata — data management

`nenudata` manages the lifecycle of NenuFAR-CD data: registering observations,
querying paths, transferring data between sites, and running the L1→L2 and
L2→L3 pipeline steps.

All commands read a TOML data-handler configuration file (default:
`data_handler.toml`) that describes the cluster layout, data levels, and
spectral windows.

## Naming conventions — N1 and N2

NenuFAR data uses two naming conventions, one on each side of the L1→L2 step.

**N1** is the telescope's native naming convention: the full observation obs_id
— start and end timestamps, field, and project — for example
`20231208_213000_20231209_070000_NT04_COSMIC_DAWN`.  Raw **L1** data is stored
under this name, so the data handler registers observations under it in
`[obs_ids]`, with one node per spectral window:

```toml
[obs_ids]
"20231208_213000_20231209_070000_NT04_COSMIC_DAWN" = ["nancep5", "nancep5", "nancep5", "nancep5"]
#                                                       one entry per spectral window
```

That full name is precise but long and impractical to handle, so the processed
**L2** data uses a shorter convention, **N2**.  An N2 observation is split along
the *time* axis into chunks, one spectral window per MS
(`SW03_T000.MS`, `SW03_T001.MS`, …) — the frequency coverage is unchanged, only
the time axis is divided.  N2 entries live under `[n2_obs_ids]` with a short,
practical key and carry the originating N1 name(s) so the raw L1 path can still
be resolved.  The `nodes` field is the pool of nodes across which the time
chunks are distributed in round-robin order:

```toml
[n2_obs_ids.20231208_NT04]
n1_obs_ids = ["20231208_213000_20231209_070000_NT04_COSMIC_DAWN"]
nodes      = "node101,node102,node103"
```

The chunk size and merge tolerance for N2 processing are controlled by
`--n_slots_per_chunk` and `--tolerance` on the `l1_to_l2` command.

```{note}
An N1 observation can also be processed straight to L2 under its N1 name
(one MS per spectral window, not split in time) — this is convenient for short
calibrator scans.  Target observations, however, should always go through the
N2 convention so the long integration is split into manageable time chunks.
```

```{toctree}
:maxdepth: 1

config
observe
data
transfer
pipeline
quality
bad_stations
```