Observation registry — list, get_obs_ids

These commands manage the list of obs_ids in a data-handler configuration file.

list — show presence of data per obs_id

nenudata list "202312*_NT04"

Prints a table with one row per obs_id and one column per data level. Each cell shows the number of MSs found on disk for that level (green = all present, yellow = some present, red = none).

nenudata list "all" --config my_data_handler.toml

The obs_ids argument accepts the same glob patterns as get_obs_ids.

get_obs_ids — list matching obs_ids

nenudata get_obs_ids "202312*_NT04"

Prints one obs_id per line. Accepts comma-separated patterns and wildcards (*, ?). This is useful for scripting:

for obs in $(nenudata get_obs_ids "202312*_NT04" -c data_handler.toml); do
    echo "$obs"
done

info — show obs_id metadata

nenudata info OBS_IDS

Prints a human-readable table of parsed obs_id metadata: date, start/end times, duration, and target field name. Obs_ids whose name cannot be parsed as YYYYMMDD_HHMMSS_YYYYMMDD_HHMMSS_FIELD are shown with dashes.

nenudata info "202312*_NT04"

Date        Start  End    Duration  Target  Obs ID
----------  -----  -----  --------  ------  ----------------------------
2023-12-08  21:30  07:00  9.50h     NT04    20231208_213000_..._NT04
2023-12-10  21:30  07:00  9.50h     NT04    20231210_213000_..._NT04
-           -      -      -         -       20231212_NT04

update_n2_obs_ids — register long N1 obs_ids as N2 entries

Groups long-format N1 obs_ids (those whose name can be parsed as YYYYMMDD_HHMMSS_YYYYMMDD_HHMMSS_FIELD) into N2 entries grouped by UT12-to-UT12 night date. Short-format obs_ids are silently skipped.

nenudata update_n2_obs_ids "202312*_NT04" --node nancep5

For each (night, field) group it creates a key YYYYMMDD_FIELD under [n2_obs_ids]. If multiple obs_ids fall on the same night each gets a numbered key: 20231208-1_NT04, 20231208-2_NT04, etc. Existing N2 entries are never modified.

Use --dry_run to preview the changes without writing the config:

nenudata update_n2_obs_ids "202312*_NT04" --node nancep5 --dry_run

Options

Option

Default

Description

--node / -n

(required)

Storage node for N2 output data

--dry_run

false

Print planned entries without modifying the config

--config / -c

data_handler.toml

Data-handler config file

Reference

nenudata list

List all obs_ids

Usage

nenudata list [OPTIONS] OBS_IDS

Options

-c, --config <config>

Data handler configuration file

Arguments

OBS_IDS

Required argument

nenudata get_obs_ids

List all obs_ids (without existence check)

Usage

nenudata get_obs_ids [OPTIONS] OBS_IDS

Options

-c, --config <config>

Data handler configuration file

Arguments

OBS_IDS

Required argument

nenudata info

Print a summary table of obs_id metadata.

OBS_IDS is a pattern (fnmatch) or ‘all’. Obs_ids whose name cannot be parsed as YYYYMMDD_HHMMSS_YYYYMMDD_HHMMSS_FIELD are shown with dashes in the metadata columns.

Usage

nenudata info [OPTIONS] OBS_IDS

Options

-c, --config <config>

Data handler configuration file

Arguments

OBS_IDS

Required argument

nenudata update_n2_obs_ids

Group N1 obs_ids into N2 entries by UT12-to-UT12 night date.

OBS_IDS is a pattern (fnmatch) or ‘all’ matching N1 obs_ids in the data handler. Obs_ids whose name cannot be parsed as YYYYMMDD_HHMMSS_YYYYMMDD_HHMMSS_FIELD are skipped. A N2 key YYYYMMDD_FIELD is created for each (night, field) group; if several N1 obs_ids fall on the same night, they each get a numbered key YYYYMMDD-INDEX_FIELD. Existing N2 entries are never modified.

NODE is the target storage node for the N2 output (may differ from the N1 source node).

Usage

nenudata update_n2_obs_ids [OPTIONS] OBS_IDS

Options

-n, --node <node>

Required Node where N2 output data will be stored

-c, --config <config>

Data handler configuration file

--dry_run

Print planned changes without writing them

Arguments

OBS_IDS

Required argument

See also

  • data — query MS paths and remove data