Junhyeok Lee

Applies to v0.1.0.

Validate the dataset

validate_dataset.py checks a collected dataset without needing the simulator, so run it right after collection (or in CI) before you train.

cd examples/collect_dataset
python validate_dataset.py --ds dataset

Run it inside your virtual environment. See Environmentif you haven't set one up.

What it verifies

  • Counts agree — rgb/, mask/, and manifest.jsonl have the same non-zero number of entries.
  • RGB decodes to H×W×3 and matches the shape recorded in the manifest.
  • Mask is lossless — every colour in the inspected mask maps to a class in classes.json; zero unknown colours. Same guarantee as rgb_to_index's n_unmapped == 0, but checked against the legend on disk.
  • Sensor sanity (only if a sensors.npzis present) — LiDAR / IMU / GNSS shapes plus basic physics (IMU gravity ≈ 9.81, ranges ≥ 0, point counts > 0).

A seg-only dataset from collect_seg_dataset.py has no sensors.npz, so those checks are skipped (it prints (no sensors.npz)); that's expected, not a failure. Pass --sample N to inspect a specific frame instead of the middle one.

A healthy run

dataset: dataset
counts: rgb=6000 mask=6000 manifest=6000
  PASS rgb/mask/manifest counts agree
inspecting frame index 3000
  PASS rgb decodes to HxWx3  -> (1080, 1920, 3)
  PASS rgb shape matches manifest  -> [1080, 1920, 3] vs [1080, 1920, 3]
mask colours: 7 known, 0 unknown
    road             (128, 64, 128)  (1512340 px)
    background       (0, 0, 0)  (498211 px)
    lane             (0, 255, 0)  (61044 px)
  PASS all mask colours are in classes.json (lossless PNG)
  (no sensors.npz)

RESULT: ALL PASS ✅

How the validator works

Every check runs through a small check(name, cond) helper that prints PASS / FAIL and folds the result into a single ok flag, so the script ends with one clear verdict.

Counts & legend

It globs rgb/*.jpg and mask/*.png, reads manifest.jsonl, and asserts the three counts are equal and non-zero. The legend is loaded from classes.json into a {(r,g,b): name}lookup, the dataset's own on-disk truth, not the library's.

Mask colours vs. legend

The lossless check is a fast set difference: take the unique colours of the sampled mask and split them into known / unknown against the legend. Any unknown colour fails the check.

cols, counts = np.unique(mask.reshape(-1, 3), axis=0, return_counts=True)
known, unknown = [], []
for c, n in zip(cols, counts):
    t = tuple(int(v) for v in c)
    (known if t in legend else unknown).append((t, int(n)))
check("all mask colours are in classes.json (lossless PNG)", len(unknown) == 0)

Optional sensor checks

If a sensors.npz exists, each stream is validated against the frame count and basic physics:

  • lidar_ranges — one row per frame, all finite and ≥ 0.
  • imu — shape (n, 10), with gravity showing up as accel ≈ 9.81.
  • gnss — shape (n, 3), and the position actually moves over the run.
  • pcd_* — per-frame structured point clouds with x/y/z/intensity fields and > 0 points each.

The run ends with RESULT: ALL PASS or FAILURES ABOVE, a single line you can gate CI on.