longitudinal/scripts
Furen Xiao 33bc3c603f refactor(core): improve data pipeline robustness and manifest generation
Refactor the data loading and preprocessing pipeline to handle edge cases in
medical imaging data, including NaN/Inf values, shape mismatches, and
numerical instability during training.

- Update `01_build_ntuh_manifest.py` with improved regex for T1c detection,
  spine exclusion, and deduplication logic based on acquisition timestamps.
- Enhance `PatchDataset` in `src/dataset.py` to handle NaN/Inf values,
  clip intensity ranges, and ensure label/image shape alignment via
  padding/trimming.
- Add a zero-gradient fallback in `src/training.py` to prevent DDP
  synchronization failures when encountering NaN/Inf losses.
- Add `scripts/scan_procs.py` for process monitoring.
- Increase DataLoader timeout to prevent hangs during heavy I/O.
2026-09-25 22:12:32 +08:00
..
01_build_ntuh_manifest.py refactor(core): improve data pipeline robustness and manifest generation 2026-09-25 22:12:32 +08:00
02_build_m6_dataset.py feat: initial project structure 2026-09-25 16:00:37 +08:00
03_scan_lee_t1c.py feat: initial project structure 2026-09-25 16:00:37 +08:00
04_reconstruct_lee.py feat: initial project structure 2026-09-25 16:00:37 +08:00
05_build_splits.py feat: initial project structure 2026-09-25 16:00:37 +08:00
06_pseudo_label.py feat: initial project structure 2026-09-25 16:00:37 +08:00
07_train.py feat: initial project structure 2026-09-25 16:00:37 +08:00
08_eval.py feat: initial project structure 2026-09-25 16:00:37 +08:00
09_run_iterative.py feat: initial project structure 2026-09-25 16:00:37 +08:00
preprocess.py feat: initial project structure 2026-09-25 16:00:37 +08:00
scan_procs.py refactor(core): improve data pipeline robustness and manifest generation 2026-09-25 22:12:32 +08:00