SLURM Pipelines#

Run PhenoTypic batch processing on SLURM-managed clusters.

Automatic SLURM Detection#

When SLURM is available, the CLI automatically submits jobs to the scheduler. To force local execution instead:

python -m phenotypic --pipeline pipeline.json --input /plates/ -o /output/ --force-local

SLURM Arguments#

Pass SLURM parameters with repeated --slurm flags:

python -m phenotypic --pipeline pipeline.json --input /plates/ -o /output/ \
    --slurm time=240 \
    --slurm slurm_partition=gpu \
    --slurm mem_gb=16

Check the Profile Before Submitting#

Add --dry-run to the same command to check it without submitting anything. The run preflight tests each profile the run would submit with sbatch --test-only (the CPU profile, and the GPU profile of a staged GPU run), compares time with the partition’s MaxTime, checks that a GPU stage’s partition has GPUs, and warns when --output, --input, the pipeline, or the metadata CSV is on node-local storage that other nodes cannot see. The same checks run before a real submission, so a rejected profile stops the run before any job is queued. The dry-run preview also prints the #SBATCH lines the profile produces.

Wait for Completion#

By default, the CLI returns immediately after submitting SLURM jobs. To wait:

python -m phenotypic --pipeline pipeline.json --input /plates/ -o /output/ --wait

Continue on SLURM#

Automatic continuation works the same way as local execution. Run the same command again:

python -m phenotypic --pipeline pipeline.json --input /plates/ -o /output/