phenotypic.phenotypicCLI module#
PhenoTypic CLI (v2.0)#
A command-line interface for executing PhenoTypic ImagePipelines on images or directories of images with support for local parallel processing and autonomous SLURM cluster execution.
- Features:
Recursive directory support (1 level deep)
Dry-run mode for previewing processing plans
Sample processing mode for testing pipelines
Resume capability with state tracking
Local parallel execution (joblib)
Autonomous SLURM execution with bash scripts
HTML failure reports with tracebacks
Progress monitoring tools
- Usage:
python -m phenotypic –mode full –pipeline pipeline.json –input ./images –output ./out [OPTIONS]
Examples
# Basic usage uv run python -m phenotypic –mode full –pipeline pipeline.json –input ./images -o ./results
# Dry-run to preview processing plan uv run python -m phenotypic –mode full –pipeline pipeline.json –input ./images -o ./results –dry-run
# Sample 5 images per dataset for testing uv run python -m phenotypic –mode full –pipeline pipeline.json –input ./images -o ./results –sample 5
# Resume interrupted processing uv run python -m phenotypic –mode full –pipeline pipeline.json –input ./images -o ./results –resume
# Restart processing from beginning (clears previous state) uv run python -m phenotypic –mode full –pipeline pipeline.json –input ./images -o ./results –restart
# SLURM execution (autonomous) uv run python -m phenotypic –mode full –pipeline pipeline.json –input ./images -o ./results –slurm slurm_partition=compute –slurm slurm_account=proj –slurm mem_gb=16
# SLURM with progress monitoring uv run python -m phenotypic –mode full –pipeline pipeline.json –input ./images -o ./results –slurm slurm_partition=compute –slurm slurm_account=proj –wait
# GridImage with custom dimensions uv run python -m phenotypic –mode full –pipeline pipeline.json –input ./plates -o ./results –image-type GridImage –nrows 16 –ncols 24
# Rerun measurements on a previous forward run without re-detecting # (reads HDFs from <previous-output-dir>/results/*/hdf/, rewrites # parquet measurements + master CSV, skips detection, does NOT # regenerate overlays, does NOT touch processing state): uv run python -m phenotypic –mode measure –pipeline pipeline.json –output <previous-output-dir>
# Recompile a previous output directory: re-aggregate the master # measurements CSV, fill in any overlay PNGs missing under # results/<ds>/overlays/ by reloading their HDFs (threaded across # –njobs workers, alpha from –overlay-alpha), rerun analysis # plugins, rebuild the manifest, and regenerate the HTML dashboard. # Existing overlays are left untouched. Pipeline JSON is NOT # required: uv run python -m phenotypic –mode recompile –output <previous-output-dir>
- Outputs:
Forward runs write a single HDF5 per input image under <output>/results/<dataset>/hdf/<stem>.h5 (layers + metadata + grid state, reloadable via Image.load_hdf5 / GridImage.load_hdf5). Overlay PNGs are always written under <output>/results/<dataset>/overlays/<stem>.png for forward runs; –mode measure reruns reuse existing overlays and do not regenerate them. –mode recompile fills in only-missing overlay PNGs from HDFs but leaves existing ones untouched.
- SLURM Execution (Autonomous HPC Cluster Processing):
Use –slurm to submit jobs to an HPC cluster via SLURM. The CLI will: 1. Generate SBATCH scripts for each dataset 2. Create array jobs for parallel image processing 3. Automatically handle dependencies and chunking 4. Support optional job monitoring with –wait
- Common Academic HPC SLURM Parameters:
slurm_partition Partition/queue name (e.g., compute, gpu, highmem) slurm_account Account for billing/fairshare (required on most clusters) slurm_qos Quality of Service tier (e.g., normal, high) time Wall time in minutes (auto-converts to HH:MM:SS) mem_gb Memory per node in GB (convenience param, adds “G” suffix) slurm_cpus_per_task CPUs per task (useful for joblib parallelism) slurm_constraint Node features/constraints (e.g., gpu_type, cpu_generation) slurm_mail_type Email notifications (e.g., END, FAIL, ALL) slurm_mail_user Email address for notifications
- Advanced SLURM Parameters:
slurm_nodes Number of nodes (default: 1) slurm_mem Memory with custom units (e.g., “32G”, “1024M”) slurm_mem_per_cpu Memory per CPU instead of per node slurm_gpus_per_node GPUs per node for GPU-accelerated operations
- Time Parameter Notes:
Use ‘time’ or ‘slurm_time’ with integer minutes
Automatically converts to HH:MM:SS format (e.g., time=120 → 02:00:00)
Valid range: 1-10080 minutes (1 minute to 7 days)
- Example: Submit with account, partition, memory, and time limits
- uv run python -m phenotypic –mode full –pipeline pipeline.json –input ./images
–output ./results –slurm slurm_partition=compute –slurm slurm_account=lab_proj –slurm mem_gb=32 –slurm time=120 –slurm slurm_mail_type=END –slurm slurm_mail_user=user@university.edu –wait
- Example: Dry-run to preview SLURM submission plan
- uv run python -m phenotypic –mode full –pipeline pipeline.json –input ./images
–output ./results –slurm slurm_partition=compute –slurm slurm_account=lab_proj –dry-run