phenotypic.phenotypicCLI module#
PhenoTypic CLI (v2.0)#
A command-line interface for executing PhenoTypic ImagePipelines on images or directories of images with support for local parallel processing and autonomous SLURM cluster execution.
- Features:
Automatic timestamped output directories
Recursive directory support (1 level deep)
Dry-run mode for previewing processing plans
Sample processing mode for testing pipelines
Resume capability with state tracking
Local parallel execution (joblib)
Autonomous SLURM execution with bash scripts
HTML failure reports with tracebacks
Progress monitoring tools
- Usage:
python -m phenotypic PIPELINE_JSON INPUT_PATH [OPTIONS]
Examples
# Basic usage with auto-generated output directory uv run python -m phenotypic pipeline.json ./images
# Specify output directory uv run python -m phenotypic pipeline.json ./images -o ./results
# Dry-run to preview processing plan uv run python -m phenotypic pipeline.json ./images –dry-run
# Sample 5 images per dataset for testing uv run python -m phenotypic pipeline.json ./images –sample 5
# Resume interrupted processing uv run python -m phenotypic pipeline.json ./images -o ./results –resume
# Restart processing from beginning (clears previous state) uv run python -m phenotypic pipeline.json ./images -o ./results –restart
# SLURM execution (autonomous) uv run python -m phenotypic pipeline.json ./images –slurm slurm_partition=compute –slurm slurm_account=proj –slurm mem_gb=16
# SLURM with progress monitoring uv run python -m phenotypic pipeline.json ./images –slurm slurm_partition=compute –slurm slurm_account=proj –wait
# GridImage with custom dimensions uv run python -m phenotypic pipeline.json ./plates –image-type GridImage –nrows 16 –ncols 24
# Save PNG overlays alongside HDFs (opt-in; default: OFF) uv run python -m phenotypic pipeline.json ./images –save-overlays
# Rerun measurements on a previous forward run without re-detecting # (reads HDFs from <previous-output-dir>/results/*/hdf/, rewrites # parquet measurements + master CSV, skips detection, does NOT # regenerate overlays, does NOT touch processing state): uv run python -m phenotypic pipeline.json –measure -o <previous-output-dir>
- Outputs:
Forward runs write a single HDF5 per input image under <output>/results/<dataset>/hdf/<stem>.h5 (layers + metadata + grid state, reloadable via Image.load_hdf5 / GridImage.load_hdf5). Overlay PNGs under <output>/results/<dataset>/overlays/<stem>.png are opt-in and only written when –save-overlays is set.
- SLURM Execution (Autonomous HPC Cluster Processing):
Use –slurm to submit jobs to an HPC cluster via SLURM. The CLI will: 1. Generate SBATCH scripts for each dataset 2. Create array jobs for parallel image processing 3. Automatically handle dependencies and chunking 4. Support optional job monitoring with –wait
- Common Academic HPC SLURM Parameters:
slurm_partition Partition/queue name (e.g., compute, gpu, highmem) slurm_account Account for billing/fairshare (required on most clusters) slurm_qos Quality of Service tier (e.g., normal, high) time Wall time in minutes (auto-converts to HH:MM:SS) mem_gb Memory per node in GB (convenience param, adds “G” suffix) slurm_cpus_per_task CPUs per task (useful for joblib parallelism) slurm_constraint Node features/constraints (e.g., gpu_type, cpu_generation) slurm_mail_type Email notifications (e.g., END, FAIL, ALL) slurm_mail_user Email address for notifications
- Advanced SLURM Parameters:
slurm_nodes Number of nodes (default: 1) slurm_mem Memory with custom units (e.g., “32G”, “1024M”) slurm_mem_per_cpu Memory per CPU instead of per node slurm_gpus_per_node GPUs per node for GPU-accelerated operations
- Time Parameter Notes:
Use ‘time’ or ‘slurm_time’ with integer minutes
Automatically converts to HH:MM:SS format (e.g., time=120 → 02:00:00)
Valid range: 1-10080 minutes (1 minute to 7 days)
- Example: Submit with account, partition, memory, and time limits
- uv run python -m phenotypic pipeline.json ./images
–slurm slurm_partition=compute –slurm slurm_account=lab_proj –slurm mem_gb=32 –slurm time=120 –slurm slurm_mail_type=END –slurm slurm_mail_user=user@university.edu –wait
- Example: Dry-run to preview SLURM submission plan
- uv run python -m phenotypic pipeline.json ./images
–slurm slurm_partition=compute –slurm slurm_account=lab_proj –dry-run