Research data infrastructure
nanotrack
A single-molecule video-tracking pipeline rebuilt from MATLAB into a reproducible Python system with serving, orchestration, observability, and CI.
ScopeResearch software. The original methodology supports published work; the open-source Python pipeline is a 2026 engineering reimplementation.
- cluster processing
- 15 h → 30 min
- peer-reviewed publications
- 6
- validation frames
- 10k
- Python
- NumPy
- OpenCV
- scikit-image
- Dask
- Airflow
- FastAPI
- Prometheus / Grafana
01 / Context
The problem
Scientific video analysis was trapped in a research workflow that was difficult to reproduce, parallelize, monitor, and hand to another engineer.
02 / Architecture
How the system fits together
- 01Video inputsynthetic, TIFF, ND2
- 02PreprocessNumPy / OpenCV / skimage
- 03Detectthreshold + connected components
- 04Tracknearest-neighbor + gap fill
- 05FeaturesMSD, MSAD, orientation
- 06Validatequality checks
- 07ExportCSV, JSON, QC, overlay
Supporting systems
- Airflowpipeline orchestration
- Dask / SLURMparallel chunk processing
- Prometheus / Grafanaruntime and error metrics
03 / Engineering
What I built
- Reimplemented detection, tracking, orientation unwrapping, feature extraction, and validation in a Python package.
- Added interchangeable NumPy, OpenCV, and scikit-image preprocessing backends.
- Parallelized chunked processing with Dask and documented a SLURM path for long cluster runs.
- Built a FastAPI service, Airflow DAG, Docker Compose stack, Prometheus metrics, and a provisioned Grafana dashboard.
- Added immutable run artifacts, checkpoint resume, self-contained QC reports, tracking overlays, and GitHub Actions CI.
04 / Evidence
Measured outcomes
processing time
Cluster parallelization mirrors the original research workflow.
nanotrack README — overview ↗peer-reviewed publications
Google Scholar lists six journal articles; additional conference abstracts, posters, and presentations are not counted as publications.
Google Scholar profile ↗validation frames
Synthetic single-molecule run with anisotropic diffusion.
nanotrack README — example outputs ↗data consistency
Automated validation in the research workflow.
Approved resume evidence ↗05 / Demo
Product evidence
06 / Boundaries
Trade-offs and limitations
- Synthetic data is the default demo path; real TIFF and ND2 loaders exist but are not part of every CI run.
- The Python package is an engineering reimplementation, not a claim that every historical publication was produced with this exact codebase.
07 / Sources