phenotypic.sdk_ #

overlays_dir(dataset: str) → Path[source]

Return path to overlays/<dataset>/.

Parameters:: dataset (str) – Dataset name.
Returns:: Directory path for overlay PNGs of the given dataset.
Return type:: Path

property curation_labels_parquet: Path: Return path to deliverables/qc/curation_labels.parquet (durable labels store).

property custom_categories_json: Path: Return path to deliverables/qc/custom_categories.json (custom-category registry).

deliverables_base: Path

property error_analysis_csv: Path: Return path to error_analysis.csv in the deliverables base.

property error_analysis_html: Path: Return path to error_analysis.html in the deliverables base.

property error_analysis_parquet: Path: Return path to error_analysis.parquet in the deliverables base.

property errors_dir: Path: Return path to the errors/ directory under the deliverables base.

property has_results: bool: Return True when a results/ directory exists under the output root.

property master_csv: Path: Return path to master_measurements.csv in the deliverables base.

property master_parquet: Path: Return path to master_measurements.parquet in the deliverables base.

property mirror_csv: Path: Return path to measurements.csv (post-applied mirror).

property mirror_parquet: Path: Return path to measurements.parquet (post-applied mirror).

output_root: Path | None

property pipeline_config_path: Path: Return path to pipeline.json in the deliverables base.

property plots_dir: Path: Return the resolved plots/ directory inside the bundle.

property qc_dir: Path

Return the QC directory, resolving legacy <output>/qc/ layouts.

Prefers deliverables/qc/ when it exists. Falls back to the legacy root <output>/qc/ when only that exists (pre-relocation runs). Returns the canonical deliverables/qc/ path for fresh writes when neither is present.

property qc_duckdb: Path: Return path to qc/qc.duckdb (the QC analysis database).

property qc_review_state_path: Path: Return path to deliverables/qc/review_state.json (GUI-owned review progress).

property resolved_pipeline_config_path: Path

Return the best existing pipeline config path inside the bundle.

Mirrors resolve_pipeline_config_path()’s precedence but anchored on deliverables_base (so a standalone bundle resolves inside itself without double-joining deliverables/): the canonical typed config when present, else the legacy plain pipeline.json when present, else the canonical path (so writers naturally create typed config files).

property results_dir: Path | None: Return the results/ directory, or None for a standalone bundle.

property verified_parquet: Path: Return path to verified.parquet (GUI-written verified-good archive).

class phenotypic.sdk_.ChunkManifestKey[source]

Bases: object

Keys inside <output>/.phenotypic/progress/chunk_manifest.json.

CHUNKS: Final[str] = 'chunks'

DATASETS: Final[str] = 'datasets'

NAME: Final[str] = 'name'

ROWS: Final[str] = 'rows'

TOTAL_ROWS: Final[str] = 'total_rows'

class phenotypic.sdk_.ChunkStateKey[source]

Bases: object

Keys inside <output>/.phenotypic/progress/chunk_state.json.

CHUNKED_FILES: Final[str] = 'chunked_files'

NEXT_CHUNK_ID: Final[str] = 'next_chunk_id'

class phenotypic.sdk_.DashboardManifestKey[source]

Bases: object

Keys inside <output>/.phenotypic/progress/manifest.json.

The manifest is built by _cli._dashboard._manifest_builder.build_manifest() and consumed by both the dashboard JS and the GUI run-console’s runs registry (_runs_registry.py). Writers and readers must reference these constants rather than spelling the bare string.

ANALYSIS_DATA_VERSION: Final[str] = 'analysis_data_version'

COMPLETED: Final[str] = 'completed'

DATASETS: Final[str] = 'datasets'

EVENT_DIAGNOSTICS: Final[str] = 'event_diagnostics'

EXECUTION_MODE: Final[str] = 'execution_mode'

FAILED: Final[str] = 'failed'

FAILURE_CATEGORIES: Final[str] = 'failure_categories'

GUI_RECORD_GENERATION: Final[str] = 'gui_record_generation': Exact GUI generation that published this canonical local manifest. Omitted from non-GUI and scheduler manifests for compatibility.

INPUT_PATH: Final[str] = 'input_path'

IS_COMPLETE: Final[str] = 'is_complete'

LAST_UPDATED: Final[str] = 'last_updated'

PENDING: Final[str] = 'pending'

PROCESSING_GENERATION: Final[str] = 'processing_generation'

SLURM_INFO: Final[str] = 'slurm_info'

STARTED: Final[str] = 'started'

START_TIME: Final[str] = 'start_time'

SUCCESS_RATE: Final[str] = 'success_rate'

TOTAL_IMAGES: Final[str] = 'total_images'

VERSION: Final[str] = 'version'

class phenotypic.sdk_.DashboardManifestSlurmInfoKey[source]

Bases: object

Keys inside the slurm_info sub-dict of the dashboard manifest.

Distinct from JobMetadataKey, even when string values overlap — these describe the manifest contract, not the job-metadata sidecar.

ACTIVE_CHUNKS: Final[str] = 'active_chunks'

CHUNK_JOB_IDS: Final[str] = 'chunk_job_ids'

CHUNK_SCRIPTS: Final[str] = 'chunk_scripts'

COMPLETED_CHUNKS: Final[str] = 'completed_chunks'

PENDING_CHUNKS: Final[str] = 'pending_chunks'

TOTAL_CHUNKS: Final[str] = 'total_chunks'

class phenotypic.sdk_.EnvVar[source]

Bases: object

Environment variable names read or set by the CLI.

SLURM injects these into batch scripts; the CLI reads them to discover its execution context (job id, array task id, …) and to find node-local scratch storage.

SCRATCH: Final[str] = 'SCRATCH'

SLURM_ARRAY_JOB_ID: Final[str] = 'SLURM_ARRAY_JOB_ID'

SLURM_ARRAY_TASK_COUNT: Final[str] = 'SLURM_ARRAY_TASK_COUNT'

SLURM_ARRAY_TASK_ID: Final[str] = 'SLURM_ARRAY_TASK_ID'

SLURM_CPUS_PER_TASK: Final[str] = 'SLURM_CPUS_PER_TASK'

SLURM_JOB_ID: Final[str] = 'SLURM_JOB_ID'

SLURM_MEM_PER_NODE: Final[str] = 'SLURM_MEM_PER_NODE'

class phenotypic.sdk_.FootprintMixin[source]

Bases: object

Provides a mixin for creating morphological footprints for image processing.

The FootprintMixin class contains a static utility method to generate structuring elements (footprints) used in various image processing tasks. This functionality is particularly helpful in the context of analyzing microbial colonies on solid media agar plates. Morphological footprints are used to highlight specific features in images, such as colony edges, shapes, or connectivity, and can assist in segmentation, noise reduction, and feature extraction.

None

class phenotypic.sdk_.GridInferenceMixin[source]

Bases: object

Mixin providing grid inference capabilities from binary masks.

Provides static methods for inferring grid structure from colony patterns using peak detection on row/column projections. Used by detectors and refiners that work with gridded plate images (96-well, 384-well formats, pinned cultures).

All methods are static to support parallelization in pipeline operations.

This is an internal utility for advanced users creating custom grid-based operations. Most users should use RoundPeaksDetector for detection or GridAlignmentRefiner for post-detection refinement directly.

class phenotypic.sdk_.HDF(filepath, name: str, mode: Literal['single', 'set'])[source]

Bases: object

Represents an interface to manage HDF5 files with support for single or set image modes, and ensures safe and compatible file access with retry and error-handling mechanisms.

The class facilitates operations on HDF5 files commonly used for storing phenotypic data in both single image and image set modes. This class includes utilities to handle locking errors and ensure compatibility by initializing proper HDF5 modes while providing safe access methods for writing.

Parameters:

name (str)
mode (Literal['single', 'set'])

filepath

Path to the HDF5 file on the filesystem.

Type:: Path

name

Name associated with the HDF5 resource, often used as an identifier.

Type:: str

mode

Specifies the mode for the HDF5 file, either single image or image set.

Type:: Literal[‘single’, ‘set’]

root_posix

The root path for the HDF5 resource, determined by the mode.

Type:: str

home_posix

The specific root directory of the HDF5 resource in the file, derived based on its mode.

Type:: str

set_data_posix

The subgroup path for the data entity in image set mode, if applicable.

Type:: str, optional

SINGLE_IMAGE_ROOT_POSIX

Base path for single image mode.

Type:: str

IMAGE_SET_ROOT_POSIX

Base path for image set mode.

Type:: str

IMAGE_SET_DATA_POSIX

Subgroup marker for image set data.

Type:: str

EXT

Set of valid file extensions used to recognize HDF5 files.

Type:: set

IMAGE_MEASUREMENT_SUBGROUP_KEY

Key for accessing measurements in an image’s group.

Type:: str

IMAGE_STATUS_SUBGROUP_KEY

Key for accessing statuses in an image’s group.

Type:: str

static assert_swmr_on(g: Group) → None[source]

Assert that SWMR mode is enabled on the group’s file.

Parameters:: g (Group) – HDF5 group to check.
Raises:: RuntimeError – If SWMR mode is not enabled.
Return type:: None

static close_handle(handle: File | Group) → None[source]

Parameters:: handle (File | Group)
Return type:: None

static get_group(handle: File, posix) → Group[source]

Retrieves or creates a group in an HDF5 file.

This method checks the validity of the provided HDF5 file handle and tries to retrieve the specified group based on the given posix path. If the group does not exist and the file is not opened in read-only mode, the group gets created. If the file is in read-only mode and the group does not exist, an error is raised.

Parameters:

handle (h5py.File) – The HDF5 file handle to operate on.
posix (str) – The posix path of the group to retrieve or create in the HDF5 file.

Returns:

The corresponding h5py group within the HDF5 file.

Return type:

h5py.Group

Raises:

ValueError – If the HDF5 file handle is invalid or no longer valid.
ValueError – If the file handle mode cannot be determined.
KeyError – If the specified group does not exist in read-only mode.

static get_uncompressed_sizes_for_group(group: Group) → tuple[dict[str, int], int][source]

Recursively collect the uncompressed (logical) sizes of SWMR-compatible datasets.

This function walks the provided HDF5 group and inspects every dataset without reading any data. For each dataset that is compatible with SWMR writing rules (i.e., chunked layout and no variable-length data types), it computes the uncompressed size in bytes as: dtype.itemsize * number_of_elements.

Notes

This works regardless of whether datasets are stored compressed on disk; the reported size is the logical size when uncompressed in memory.
Variable-length strings (and datasets containing variable-length fields) are excluded because they are not SWMR-write friendly and their uncompressed size cannot be determined from metadata alone.
The operation is safe under SWMR: it only reads object metadata, creates no new refine, and does not modify the file.

Parameters:

group (Group) – The root h5py.Group to traverse.

Returns:

sizes: dict mapping absolute dataset paths (e.g., ‘/grp/ds’) to uncompressed size in bytes.
total_bytes: sum of all values in sizes.

Return type:

A tuple (sizes, total_bytes) where

static load_frame(group: Group, *, require_swmr: bool = False) → DataFrame[source]

Load a pandas DataFrame from HDF5 storage.

Parameters:

group (Group) – HDF5 group containing the DataFrame data.
require_swmr (bool) – If True, assert SWMR mode is enabled.

Returns:

Reconstructed pandas DataFrame.

Raises:

RuntimeError – If require_swmr=True and SWMR mode not enabled.
ValueError – If data validation fails.

Return type:

DataFrame

static load_series(group: Group, *, dataset: str = 'values', index_dataset: str = 'index', require_swmr: bool = False) → Series[source]

Load a pandas Series from HDF5 storage.

Reconstructs the Series with original name, index names, order, and missingness. Respects logical length from attributes.

Parameters:

group (Group) – HDF5 group containing the Series data.
dataset (str) – Name of the values dataset.
index_dataset (str) – Name of the index dataset.
require_swmr (bool) – If True, assert SWMR mode is enabled.

Returns:

Reconstructed pandas Series.

Raises:

RuntimeError – If require_swmr=True and SWMR mode not enabled.
ValueError – If data validation fails.

Return type:

Series

static preallocate_frame_layout(group: Group, dataframe: DataFrame, *, chunks: tuple[int, ...] = (25,), compression: str = 'gzip', preallocate: int = 100, string_fixed_length: int = 100, require_swmr: bool = False) → None[source]

Preallocate HDF5 layout for a pandas DataFrame without writing data.

Creates layout for shared index and column series using Series preallocation.

Parameters:

group (Group) – HDF5 group to write to.
dataframe (DataFrame) – pandas DataFrame to create layout for.
chunks (tuple[int, ...]) – Chunk shape for datasets.
compression (str) – Compression algorithm.
preallocate (int) – Initial allocation size.
string_fixed_length (int) – Character length for fixed-length strings.
require_swmr (bool) – If True, assert SWMR mode is enabled.

Raises:

RuntimeError – If require_swmr=True and SWMR mode not enabled. If group.file.swmr_mode is True and datasets don’t exist.
ValueError – If DataFrame validation fails.

Return type:

None

static preallocate_series_layout(group: Group, series: Series, *, dataset: str = 'values', index_dataset: str = 'index', chunks: tuple[int, ...] = (25,), compression: str = 'gzip', preallocate: int = 100, string_fixed_length: int = 100) → None[source]

Preallocate HDF5 layout for a pandas Series without writing data.

Creates resizable, chunked, compressed datasets with initial shape (preallocate,) and maxshape (None,). Initializes masks to zeros and sets len=0.

Parameters:

group (Group) – HDF5 group to write to.
series (Series) – pandas Series to create layout for (used for schema).
dataset (str) – Name for the values dataset.
index_dataset (str) – Name for the index dataset.
chunks (tuple[int, ...]) – Chunk shape for datasets.
compression (str) – Compression algorithm.
preallocate (int) – Initial allocation size.
string_fixed_length (int) – Character length for fixed-length strings.

Raises:

RuntimeError – If require_swmr=True and SWMR mode not enabled. If group.file.swmr_mode is True and datasets don’t exist.
ValueError – If series validation fails.

Return type:

None

static save_array2hdf5(group, array, name, **kwargs)[source]

Saves a given numpy array to an HDF5 group. If a dataset with the specified name already exists in the group, it checks if the shapes match. If the shapes match, it updates the existing dataset; otherwise, it removes the existing dataset and creates a new one with the specified name. If a dataset with the given name doesn’t exist, it creates a new dataset.

Parameters:

group – h5py.Group The HDF5 group in which the dataset will be saved.
array – numpy.ndarray The data array to be stored in the dataset.
name – str The name of the dataset within the group.
**kwargs – dict Additional keyword arguments to pass when creating a new dataset.

static save_frame_append(group: Group, dataframe: DataFrame, *, require_swmr: bool = True) → None[source]

Append a pandas DataFrame to existing HDF5 datasets.

Parameters:

group (Group) – HDF5 group containing existing datasets.
dataframe (DataFrame) – pandas DataFrame to append.
require_swmr (bool) – If True, assert SWMR mode is enabled.

Raises:

RuntimeError – If require_swmr=True and SWMR mode not enabled.
ValueError – If validation fails or schema mismatch.

Return type:

None

static save_frame_new(group: Group, dataframe: DataFrame, *, chunks: tuple[int, ...] = (25,), compression: str = 'gzip', preallocate: int = 100, string_fixed_length: int = 100, require_swmr: bool = False) → None[source]

Create datasets and write a pandas DataFrame to HDF5.

Parameters:

group (Group) – HDF5 group to write to.
dataframe (DataFrame) – pandas DataFrame to persist.
chunks (tuple[int, ...]) – Chunk shape for new datasets.
compression (str) – Compression algorithm for new datasets.
preallocate (int) – Initial allocation size for new datasets.
string_fixed_length (int) – Character length for fixed-length strings.
require_swmr (bool) – If True, assert SWMR mode is enabled.

Raises:

RuntimeError – If require_swmr=True and SWMR mode not enabled.
ValueError – If DataFrame validation fails.

Return type:

None

static save_frame_update(group: Group, dataframe: DataFrame, *, start: int = 0, require_swmr: bool = True) → None[source]

Update a pandas DataFrame in HDF5 at specified position.

Parameters:

group (Group) – HDF5 group containing existing datasets.
dataframe (DataFrame) – pandas DataFrame to write.
start (int) – Starting position for the update.
require_swmr (bool) – If True, assert SWMR mode is enabled.

Raises:

RuntimeError – If require_swmr=True and SWMR mode not enabled.
ValueError – If validation fails or schema mismatch.

Return type:

None

static save_series_append(group: Group, series: Series, *, dataset: str = 'values', index_dataset: str = 'index', require_swmr: bool = True) → None[source]

Append a pandas Series to existing HDF5 datasets.

Appends at the end using current logical length. Resizes datasets if needed and updates logical length.

Parameters:

group (Group) – HDF5 group containing existing datasets.
series (Series) – pandas Series to append.
dataset (str) – Name of the values dataset.
index_dataset (str) – Name of the index dataset.
require_swmr (bool) – If True, assert SWMR mode is enabled.

Raises:

RuntimeError – If require_swmr=True and SWMR mode not enabled.
ValueError – If validation fails or schema mismatch.

Return type:

None

static save_series_new(group: Group, series: Series, *, dataset: str = 'values', index_dataset: str = 'index', chunks: tuple[int, ...] = (25,), compression: str = 'gzip', preallocate: int = 100, string_fixed_length: int = 100, require_swmr: bool = False) → None[source]

Create datasets and write a pandas Series to HDF5.

Creates new datasets or reuses existing preallocated layout. Writes the first len(series) elements and sets logical length.

Parameters:

group (Group) – HDF5 group to write to.
series (Series) – pandas Series to persist.
dataset (str) – Name for the values dataset.
index_dataset (str) – Name for the index dataset.
chunks (tuple[int, ...]) – Chunk shape for new datasets.
compression (str) – Compression algorithm for new datasets.
preallocate (int) – Initial allocation size for new datasets.
string_fixed_length (int) – Character length for fixed-length strings.
require_swmr (bool) – If True, assert SWMR mode is enabled.

Raises:

RuntimeError – If require_swmr=True and SWMR mode not enabled.
ValueError – If series validation fails.

Return type:

None

static save_series_update(group: Group, series: Series, *, start: int = 0, dataset: str = 'values', index_dataset: str = 'index', require_swmr: bool = True) → None[source]

Update a pandas Series in HDF5 at specified position.

Overwrites [start:start+len(series)] and updates logical length to the largest contiguous written extent.

Parameters:

group (Group) – HDF5 group containing existing datasets.
series (Series) – pandas Series to write.
start (int) – Starting position for the update.
dataset (str) – Name of the values dataset.
index_dataset (str) – Name of the index dataset.
require_swmr (bool) – If True, assert SWMR mode is enabled.

Raises:

RuntimeError – If require_swmr=True and SWMR mode not enabled.
ValueError – If validation fails or schema mismatch.

Return type:

None

__init__(filepath, name: str, mode: Literal['single', 'set'])[source]

Initializes a class instance to manage HDF5 file structures for single or set image data based on the given filepath, name of the resource, and operational mode.

filepath

Path to the HDF5 file.

Type:: Path

name

Identifier for the resource within the HDF5 file.

Type:: str

mode

Operational mode determining the structure and organization within the HDF5 file. Must be either ‘single’ or ‘set’.

Type:: Literal[‘single’, ‘set’]

root_posix

Posix path representing the root directory within the HDF5 file based on the mode.

Type:: str

home_posix

Posix path representing the home directory for the resource within the HDF5 file based on the mode.

Type:: str

set_data_posix

Posix path for the data subdirectory within the resource home directory. Only initialized in ‘set’ mode.

Type:: Optional[str]

Parameters:

filepath – Path to the target HDF5 file. Must have an HDF5-compatible extension, or a ValueError is raised.
name (str) – Name of the resource to be managed in the file. Used to construct the home directory for the resource within the HDF5 file.
mode (Literal['single', 'set']) – Operational mode. Specifies whether the resource represents a ‘single’ or ‘set’ image data. If the mode is invalid, a ValueError is raised.

Raises:

ValueError – If the filepath does not have an HDF5-compatible extension.
ValueError – If the mode is neither ‘single’ nor ‘set’.

get_data_group(handle)[source]

get_home(handle)[source]

Retrieves a specific group from an HDF file corresponding to single image data.

This method is used to fetch a predefined group from an HDF container, where the group is identified by a constant key related to single image data. The function provides a static interface allowing invocation without requiring an instance of the class.

Parameters:

handle – The HDF file handle from which the group should be retrieved.

Returns:

The group corresponding to single image data, retrieved based on the defined SINGLE_IMAGE_ROOT_POSIX.

Raises:

Appropriate exceptions may be raised by the underlying HDF.get_group() method, –
based on the implementation and provided handle or key. –

get_image_group(handle, image_name)[source]

get_image_measurement_subgroup(handle, image_name)[source]

get_protected_metadata_subgroup(handle: File, image_name: str) → Group[source]

Parameters:

handle (File)
image_name (str)

Return type:

Group

get_public_metadata_subgroup(handle: File, image_name: str) → Group[source]

Parameters:

handle (File)
image_name (str)

Return type:

Group

get_root_group(handle) → Group[source]

Return type:: Group

get_status_subgroup(handle, image_name)[source]

reader() → File[source]

Return type:: File

safe_writer() → File[source]

Returns a writer object that provides safe and controlled write access to an HDF5 file at the specified filepath or creates it if it doesn’t exist. Ensures that the file uses the ‘latest’ version of the HDF5 library for compatibility and performance.

Handles HDF5 file locking conflicts by attempting to clear consistency flags and retrying file opening with exponential backoff.

Returns:: A file writer object with append mode and ‘latest’ library version enabled.
Return type:: h5py.File
Raises:: OSError – If file cannot be opened after all retry attempts.

strict_writer() → File[source]

Provides access to an HDF5 file in read/write mode using the h5py library. This property is used to obtain an h5py.File object configured with the latest library version.

Note

If using SWMR mode, don’t forget to enable SWMR mode:

>>> hdf = HDF(filepath)
>>> with hdf.writer as writer:
...     writer.swmr_mode = True
...     # rest of your code

Returns:: An HDF5 file object opened in ‘r+’ mode, enabling reading and writing.
Return type:: h5py.File
Raises:: OSError – If the file cannot be opened or accessed.

swmr_reader() → File[source]

Return type:: File

swmr_writer() → File[source]

Returns a writer object that provides safe SWMR-compatible write access to an HDF5 file. Creates the file if it doesn’t exist and enables SWMR mode properly.

This method ensures proper SWMR mode initialization by creating the file with the correct settings from the start, avoiding cache conflicts that occur when trying to enable SWMR mode after opening.

Returns:: A file writer object with SWMR mode enabled.
Return type:: h5py.File
Raises:: OSError – If file cannot be opened after all retry attempts.

EXT = {'.h5', '.hdf', '.hdf5', '.he5'}

IMAGE_MEASUREMENT_SUBGROUP_KEY = 'measurements'

IMAGE_SET_DATA_POSIX = 'data'

IMAGE_SET_ROOT_POSIX = '/phenotypic/image_sets/'

IMAGE_STATUS_SUBGROUP_KEY = 'status'

PROTECTED_METADATA_SUBGROUP_KEY = 'protected_metadata'

PUBLIC_METADATA_SUBGROUP_KEY = 'public_metadata'

SINGLE_IMAGE_ROOT_POSIX = '/phenotypic/images/'

class phenotypic.sdk_.HdfAttr[source]

Bases: object

Top-level attribute keys on per-image HDF5 files.

PHENOTYPIC_CLASS: Final[str] = 'phenotypic_class'

class phenotypic.sdk_.InputLayerMixin(*, input_layer: Literal['detect_mat', 'rgb'] = 'detect_mat')[source]

Bases: BaseModel

Adds an input_layer field selecting the operation’s source array.

Pointwise intensity curves are non-linear, so applying one to the three RGB channels and then collapsing to a detection matrix gives a different — often better — colony/background separation than collapsing first. This mixin exposes that choice without changing the output contract: the only layer an enhancer ever writes is still detect_mat.

When input_layer="rgb" the 3-D result is collapsed back to 2-D by projecting it through the image’s own detect_mode, so an upstream SetDetectMode(mode="MinRGB") is honoured.

The field is appended to the end of the subclass’s field order. When stacked with NormalizedOutputMixin, list this mixin first; the resulting order is […op params…, norm, input_layer].

Note

Reading rgb discards any enhancement a prior operation wrote to detect_mat — the same behaviour as SetDetectMode. This is documented, not enforced.

Parameters:: input_layer (Literal['detect_mat', 'rgb'])

classmethod __get_pydantic_json_schema__(core_schema: CoreSchema, handler: GetJsonSchemaHandler, /) → JsonSchemaValue

Hook into generating the model’s JSON schema.

Parameters:

core_schema (CoreSchema) – A pydantic-core CoreSchema. You can ignore this argument and call the handler with a new CoreSchema, wrap this CoreSchema ({‘type’: ‘nullable’, ‘schema’: current_schema}), or just call the handler with the original schema.
handler (GetJsonSchemaHandler) – Call into Pydantic’s internal JSON schema generation. This will raise a pydantic.errors.PydanticInvalidForJsonSchema if JSON schema generation fails. Since this gets called by BaseModel.model_json_schema you can override the schema_generator argument to that function to change JSON schema generation globally for a type.

Returns:

A JSON schema, as a Python object.

Return type:

JsonSchemaValue

classmethod __pydantic_init_subclass__(**kwargs: Any) → None[source]

Move input_layer to the end of the subclass’s field order.

Parameters:: kwargs (Any)
Return type:: None

classmethod __pydantic_on_complete__() → None

This is called once the class and its fields are fully initialized and ready to be used.

This typically happens when the class is created (just before [__pydantic_init_subclass__()][pydantic.main.BaseModel.__pydantic_init_subclass__] is called on the superclass), except when forward annotations are used that could not immediately be resolved. In that case, it will be called later, when the model is rebuilt automatically or explicitly using [model_rebuild()][pydantic.main.BaseModel.model_rebuild].

Return type:: None

classmethod construct(_fields_set: set[str] | None = None, **values: Any) → Self

Parameters:

_fields_set (set[str] | None)
values (Any)

Return type:

classmethod from_orm(obj: Any) → Self

Parameters:: obj (Any)
Return type:: Self

classmethod model_construct(_fields_set: set[str] | None = None, **values: Any) → Self

Creates a new instance of the Model class with validated data.

Creates a new model setting __dict__ and __pydantic_fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed.

!!! note: model_construct() generally respects the model_config.extra setting on the provided model. That is, if model_config.extra == ‘allow’, then all extra passed values are added to the model instance’s __dict__ and __pydantic_extra__ fields. If model_config.extra == ‘ignore’ (the default), then all extra passed values are ignored. Because no validation is performed with a call to model_construct(), having model_config.extra == ‘forbid’ does not result in an error if extra values are passed, but they will be ignored.

Parameters:

_fields_set (set[str] | None) – A set of field names that were originally explicitly set during instantiation. If provided, this is directly used for the [model_fields_set][pydantic.BaseModel.model_fields_set] attribute. Otherwise, the field names from the values argument will be used.
values (Any) – Trusted or pre-validated data dictionary.

Returns:

A new instance of the Model class with validated data.

Return type:

classmethod model_json_schema(by_alias: bool = True, ref_template: str = '#/$defs/{model}', schema_generator: type[~pydantic.json_schema.GenerateJsonSchema] = <class 'pydantic.json_schema.GenerateJsonSchema'>, mode: ~typing.Literal['validation', 'serialization'] = 'validation', *, union_format: ~typing.Literal['any_of', 'primitive_type_array'] = 'any_of') → dict[str, Any]

Generates a JSON schema for a model class.

Parameters:

by_alias (bool) – Whether to use attribute aliases or not.
ref_template (str) – The reference template.
union_format (Literal['any_of', 'primitive_type_array']) –
The format to use when combining schemas from unions together. Can be one of:
- ’any_of’: Use the [anyOf](https://json-schema.org/understanding-json-schema/reference/combining#anyOf)
keyword to combine schemas (the default). - ‘primitive_type_array’: Use the [type](https://json-schema.org/understanding-json-schema/reference/type) keyword as an array of strings, containing each type of the combination. If any of the schemas is not a primitive type (string, boolean, null, integer or number) or contains constraints/metadata, falls back to any_of.
schema_generator (type[GenerateJsonSchema]) – To override the logic used to generate the JSON schema, as a subclass of GenerateJsonSchema with your desired modifications
mode (Literal['validation', 'serialization']) – The mode in which to generate the schema.

Returns:

The JSON schema for the given model class.

Return type:

classmethod model_parametrized_name(params: tuple[type[Any], ...]) → str

Compute the class name for parametrizations of generic classes.

This method can be overridden to achieve a custom naming scheme for generic BaseModels.

Parameters:: params (tuple[type[Any], ...]) – Tuple of types of the class. Given a generic class Model with 2 type variables and a concrete model Model[str, int], the value (str, int) would be passed to params.
Returns:: String representing the new class where params are passed to cls as type variables.
Raises:: TypeError – Raised when trying to generate concrete names for non-generic models.
Return type:: str

classmethod model_rebuild(*, force: bool = False, raise_errors: bool = True, _parent_namespace_depth: int = 2, _types_namespace: MappingNamespace | None = None) → bool | None

Try to rebuild the pydantic-core schema for the model.

This may be necessary when one of the annotations is a ForwardRef which could not be resolved during the initial attempt to build the schema, and automatic rebuilding fails.

Parameters:

force (bool) – Whether to force the rebuilding of the model schema, defaults to False.
raise_errors (bool) – Whether to raise errors, defaults to True.
_parent_namespace_depth (int) – The depth level of the parent namespace, defaults to 2.
_types_namespace (MappingNamespace | None) – The types namespace, defaults to None.

Returns:

Returns None if the schema is already “complete” and rebuilding was not required. If rebuilding _was_ required, returns True if rebuilding was successful, otherwise False.

Return type:

bool | None

Validate a pydantic model instance.

Parameters:

obj (Any) – The object to validate.
strict (bool | None) – Whether to enforce types strictly.
extra (Literal['allow', 'ignore', 'forbid'] | None) – Whether to ignore, allow, or forbid extra data during model validation. See the [extra configuration value][pydantic.ConfigDict.extra] for details.
from_attributes (bool | None) – Whether to extract data from object attributes.
context (Any | None) – Additional context to pass to the validator.
by_alias (bool | None) – Whether to use the field’s alias when validating against the provided input data.
by_name (bool | None) – Whether to use the field’s name when validating against the provided input data.

Raises:

ValidationError – If the object could not be validated.

Returns:

The validated model instance.

Return type:

!!! abstract “Usage Documentation”: [JSON Parsing](../concepts/json.md#json-parsing)

Validate the given JSON data against the Pydantic model.

Parameters:

json_data (str | bytes | bytearray) – The JSON data to validate.
strict (bool | None) – Whether to enforce types strictly.
extra (Literal['allow', 'ignore', 'forbid'] | None) – Whether to ignore, allow, or forbid extra data during model validation. See the [extra configuration value][pydantic.ConfigDict.extra] for details.
context (Any | None) – Extra variables to pass to the validator.
by_alias (bool | None) – Whether to use the field’s alias when validating against the provided input data.
by_name (bool | None) – Whether to use the field’s name when validating against the provided input data.

Returns:

The validated Pydantic model.

Raises:

ValidationError – If json_data is not a JSON string or the object could not be validated.

Return type:

Validate the given object with string data against the Pydantic model.

Parameters:

obj (Any) – The object containing string data to validate.
strict (bool | None) – Whether to enforce types strictly.
extra (Literal['allow', 'ignore', 'forbid'] | None) – Whether to ignore, allow, or forbid extra data during model validation. See the [extra configuration value][pydantic.ConfigDict.extra] for details.
context (Any | None) – Extra variables to pass to the validator.
by_alias (bool | None) – Whether to use the field’s alias when validating against the provided input data.
by_name (bool | None) – Whether to use the field’s name when validating against the provided input data.

Returns:

The validated Pydantic model.

Return type:

classmethod parse_file(path: str | Path, *, content_type: str | None = None, encoding: str = 'utf8', proto: DeprecatedParseProtocol | None = None, allow_pickle: bool = False) → Self

Parameters:

path (str | Path)
content_type (str | None)
encoding (str)
proto (DeprecatedParseProtocol | None)
allow_pickle (bool)

Return type:

Self

classmethod parse_obj(obj: Any) → Self

Parameters:: obj (Any)
Return type:: Self

classmethod parse_raw(b: str | bytes, *, content_type: str | None = None, encoding: str = 'utf8', proto: DeprecatedParseProtocol | None = None, allow_pickle: bool = False) → Self

Parameters:

b (str | bytes)
content_type (str | None)
encoding (str)
proto (DeprecatedParseProtocol | None)
allow_pickle (bool)

Return type:

Self

classmethod schema(by_alias: bool = True, ref_template: str = '#/$defs/{model}') → Dict[str, Any]

Parameters:

by_alias (bool)
ref_template (str)

Return type:

classmethod schema_json(*, by_alias: bool = True, ref_template: str = '#/$defs/{model}', **dumps_kwargs: Any) → str

Parameters:

by_alias (bool)
ref_template (str)
dumps_kwargs (Any)

Return type:

classmethod update_forward_refs(**localns: Any) → None

Parameters:: localns (Any)
Return type:: None

classmethod validate(value: Any) → Self

Parameters:: value (Any)
Return type:: Self

__copy__() → Self

Returns a shallow copy of the model.

Return type:: Self

__deepcopy__(memo: dict[int, Any] | None = None) → Self

Returns a deep copy of the model.

Parameters:: memo (dict[int, Any] | None)
Return type:: Self

__init__(**data: Any) → None

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Parameters:: data (Any)
Return type:: None

__iter__() → Generator[tuple[str, Any], None, None]

So dict(model) works.

Return type:: Generator[tuple[str, Any], None, None]

__pretty__(fmt: Callable[[Any], Any], **kwargs: Any) → Generator[Any]

Used by devtools (https://python-devtools.helpmanual.io/) to pretty print objects.

Parameters:

fmt (Callable[[Any], Any])
kwargs (Any)

Return type:

Generator[Any]

__repr_name__() → str

Name of the instance’s class, used in __repr__.

Return type:: str

__repr_recursion__(object: Any) → str

Returns the string representation of a recursive object.

Parameters:: object (Any)
Return type:: str

__rich_repr__() → RichReprResult

Used by Rich (https://rich.readthedocs.io/en/stable/pretty.html) to pretty print objects.

Return type:: RichReprResult

Returns a copy of the model.

!!! warning “Deprecated”: This method is now deprecated; use model_copy instead.

If you need include or exclude, use:

`python {test="skip" lint="skip"} data = self.model_dump(include=include, exclude=exclude, round_trip=True) data = {**data, **(update or {})} copied = self.model_validate(data) `

Parameters:

include (AbstractSetIntStr | MappingIntStrAny | None) – Optional set or mapping specifying which fields to include in the copied model.
exclude (AbstractSetIntStr | MappingIntStrAny | None) – Optional set or mapping specifying which fields to exclude in the copied model.
update (Dict[str, Any] | None) – Optional dictionary of field-value pairs to override field values in the copied model.
deep (bool) – If True, the values of fields that are Pydantic models will be deep-copied.

Returns:

A copy of the model with included, excluded and updated fields as specified.

Return type:

Self

Parameters:

include (set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None)
exclude (set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None)
by_alias (bool)
exclude_unset (bool)
exclude_defaults (bool)
exclude_none (bool)

Return type:

Parameters:

include (set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None)
exclude (set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None)
by_alias (bool)
exclude_unset (bool)
exclude_defaults (bool)
exclude_none (bool)
encoder (Callable[[Any], Any] | None)
models_as_dict (bool)
dumps_kwargs (Any)

Return type:

model_copy(*, update: Mapping[str, Any] | None = None, deep: bool = False) → Self

!!! abstract “Usage Documentation”: [model_copy](../concepts/models.md#model-copy)

Returns a copy of the model.

!!! note: The underlying instance’s [__dict__][object.__dict__] attribute is copied. This might have unexpected side effects if you store anything in it, on top of the model fields (e.g. the value of [cached properties][functools.cached_property]).

Parameters:

update (Mapping[str, Any] | None) – Values to change/add in the new model. Note: the data is not validated before creating the new model. You should trust this data.
deep (bool) – Set to True to make a deep copy of the model.

Returns:

New model instance.

Return type:

!!! abstract “Usage Documentation”: [model_dump](../concepts/serialization.md#python-mode)

Generate a dictionary representation of the model, optionally specifying which fields to include or exclude.

Parameters:

mode (Literal['json', 'python'] | str) – The mode in which to_python should run. If mode is ‘json’, the output will only contain JSON serializable types. If mode is ‘python’, the output may contain non-JSON-serializable Python objects.
include (set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None) – A set of fields to include in the output.
exclude (set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None) – A set of fields to exclude from the output.
context (Any | None) – Additional context to pass to the serializer.
by_alias (bool | None) – Whether to use the field’s alias in the dictionary key if defined.
exclude_unset (bool) – Whether to exclude fields that have not been explicitly set.
exclude_defaults (bool) – Whether to exclude fields that are set to their default value.
exclude_none (bool) – Whether to exclude fields that have a value of None.
exclude_computed_fields (bool) – Whether to exclude computed fields. While this can be useful for round-tripping, it is usually recommended to use the dedicated round_trip parameter instead.
round_trip (bool) – If True, dumped values should be valid as input for non-idempotent types such as Json[T].
warnings (bool | Literal['none', 'warn', 'error']) – How to handle serialization errors. False/”none” ignores them, True/”warn” logs errors, “error” raises a [PydanticSerializationError][pydantic_core.PydanticSerializationError].
fallback (Callable[[Any], Any] | None) – A function to call when an unknown value is encountered. If not provided, a [PydanticSerializationError][pydantic_core.PydanticSerializationError] error is raised.
serialize_as_any (bool) – Whether to serialize fields with duck-typing serialization behavior.

Returns:

A dictionary representation of the model.

Return type:

!!! abstract “Usage Documentation”: [model_dump_json](../concepts/serialization.md#json-mode)

Generates a JSON representation of the model using Pydantic’s to_json method.

Parameters:

indent (int | None) – Indentation to use in the JSON output. If None is passed, the output will be compact.
ensure_ascii (bool) – If True, the output is guaranteed to have all incoming non-ASCII characters escaped. If False (the default), these characters will be output as-is.
include (set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None) – Field(s) to include in the JSON output.
exclude (set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None) – Field(s) to exclude from the JSON output.
context (Any | None) – Additional context to pass to the serializer.
by_alias (bool | None) – Whether to serialize using field aliases.
exclude_unset (bool) – Whether to exclude fields that have not been explicitly set.
exclude_defaults (bool) – Whether to exclude fields that are set to their default value.
exclude_none (bool) – Whether to exclude fields that have a value of None.
exclude_computed_fields (bool) – Whether to exclude computed fields. While this can be useful for round-tripping, it is usually recommended to use the dedicated round_trip parameter instead.
round_trip (bool) – If True, dumped values should be valid as input for non-idempotent types such as Json[T].
warnings (bool | Literal['none', 'warn', 'error']) – How to handle serialization errors. False/”none” ignores them, True/”warn” logs errors, “error” raises a [PydanticSerializationError][pydantic_core.PydanticSerializationError].
fallback (Callable[[Any], Any] | None) – A function to call when an unknown value is encountered. If not provided, a [PydanticSerializationError][pydantic_core.PydanticSerializationError] error is raised.
serialize_as_any (bool) – Whether to serialize fields with duck-typing serialization behavior.

Returns:

A JSON string representation of the model.

Return type:

model_post_init(context: Any, /) → None

Override this method to perform additional initialization after __init__ and model_construct. This is useful if you want to do some validation that requires the entire model to be initialized.

Parameters:: context (Any)
Return type:: None

input_layer: InputLayer

model_computed_fields = {}

model_config: ClassVar[ConfigDict] = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

property model_extra: dict[str, Any] | None

Get extra fields set during validation.

Returns:: A dictionary of extra fields, or None if config.extra is not set to “allow”.

model_fields = {'input_layer': FieldInfo(annotation=Literal['detect_mat', 'rgb'], required=False, default='detect_mat')}

property model_fields_set: set[str]

Returns the set of fields that have been explicitly set on this model instance.

Returns:

A set of strings representing the fields that have been set,: i.e. that were not filled from defaults.

class phenotypic.sdk_.JobMetadataKey[source]

Bases: object

Keys inside <output>/.phenotypic/progress/job_metadata.json.

Writers (CLI execution strategies) and readers (recompile worker, sentinel, checkpoint handler, GUI runs registry) must reference these constants — never the bare string. Renaming a key here should fail fast at every site.

CHUNK_JOB_IDS: Final[str] = 'chunk_job_ids'

CHUNK_SCRIPTS: Final[str] = 'chunk_scripts'

DATASETS: Final[str] = 'datasets'

EXECUTION_MODE: Final[str] = 'execution_mode'

GUI_RECORD_GENERATION: Final[str] = 'gui_record_generation': GUI owner-record generation that initiated this scheduler launch. Together with slurm_generation this is the durable restart-safe binding between GUI identity and the CLI lifecycle epoch.

IMAGE_TASK_MAPPING: Final[str] = 'image_task_mapping'

IMAGE_TYPE: Final[str] = 'image_type'

INCLUDE_DATASET_COLUMN: Final[str] = 'include_dataset_column'

INPUT_PATH: Final[str] = 'input_path'

METADATA_CSV: Final[str] = 'metadata_csv'

NCOLS: Final[str] = 'ncols'

NO_QC: Final[str] = 'no_qc': Whether the recompile finalizer task should skip QC compute. Set on the SLURM recompile finalizer task dict alongside METADATA_CSV; read by _cli_recompile_worker._run_post_master_steps.

NROWS: Final[str] = 'nrows'

ORCHESTRATION_EPOCH: Final[str] = 'orchestration_epoch'

PIPELINE_PATH: Final[str] = 'pipeline_path'

PROCESSING_GENERATION: Final[str] = 'processing_generation'

SLURM_JOB_IDS: Final[str] = 'slurm_job_ids'

START_TIME: Final[str] = 'start_time'

class phenotypic.sdk_.LazyWidgetMixin[source]

Bases: object

Mixin providing a lazy ipywidget interface.

This mixin allows ImageOperation classes to automatically generate a Jupyter widget interface for parameter tuning and visualization.

The six lazily-populated UI handles this mixin uses (_ui, _param_widgets, _view_dropdown, _update_button, _output_widget, _image_ref) are declared as PrivateAttr on ImageOperation rather than here. A plain (non-BaseModel) mixin’s PrivateAttr declarations are not collected by pydantic, so this mixin stays a stateless methods-only class and the private attrs live on the pydantic model that mixes it in. This keeps the mixin safe to combine with any BaseModel in any MRO position.

__getstate__()[source]

Prepare the object for pickling by disposing of any widgets.

This ensures that UI components (which may contain unpickleable objects like input functions or thread locks) are cleaned up before serialization.

Note

This method modifies the object state by calling dispose_widgets(). Any active widgets will be detached from the object.

widget(image: Image | None = None, show: bool = False) → Widget[source]

Return (and optionally display) the root widget.

Parameters:

image (Image | None) – Optional image to visualize. If provided, visualization controls will be added to the widget.
show (bool) – Whether to display the widget immediately. Defaults to False.

Returns:

The root widget.

Return type:

ipywidgets.Widget

Raises:

ImportError – If ipywidgets or IPython are not installed.

class phenotypic.sdk_.ModulePath[source]

Bases: object

Importable module paths used in dynamic importlib.import_module dispatch.

Spelled out here so a renamed sub-package fails at type-check time (consumers reference ModulePath.POST — a typo there is caught by mypy) rather than silently at runtime.

ANALYSIS: Final[str] = 'phenotypic.analysis'

POST: Final[str] = 'phenotypic.post'

class phenotypic.sdk_.NormControlMixin[source]

Bases: object

Mixin for operations that need to disable normalization of inner operations.

Provides a method to create copies of ImageEnhancer or ImagePipeline instances with output normalization disabled. This is useful for composite operations where an inner enhancer operates on non-normalized data (e.g., variance-stabilized values from the Generalized Anscombe Transform, typically in the range ~1-32), where clipping or rescaling to [0, 1] would destroy the inverse transform.

The mixin uses duck typing to check for a norm attribute on operations. If an operation has one, the _disable_normalization method will create a shallow copy with norm=None. This preserves the original operation unchanged while allowing the copy to operate without output normalization. Operations that carry no norm field (e.g. BlurGauss) are returned unchanged.

Note

Renamed from ClipControlMixin in 0.18.0, when clip: bool became NormOut. The old name is gone.

Example

Creating a normalization-disabled copy of an enhancer:

>>> from phenotypic.abc_ import ImageEnhancer
>>> from phenotypic.sdk_ import NormalizedOutputMixin, NormControlMixin
>>>
>>> class Denoise(NormalizedOutputMixin, ImageEnhancer):
...     '''Denoise a colony plate.
...
...     Args:
...         sigma: Smoothing width in pixels.
...         norm: Output normalization policy.
...     '''
...
...     sigma: float = 1.0
...
...     def _operate(self, image):
...         return image
>>>
>>> enh = Denoise(sigma=5.0, norm="clip")
>>> copied = NormControlMixin._disable_normalization(enh)
>>> # Original unchanged, copy has norm=None
>>> enh.norm, copied.norm
('clip', None)

Creating a normalization-disabled copy of a pipeline:

>>> from phenotypic import ImagePipeline
>>> from phenotypic.enhance import BlurGauss
>>>
>>> pipeline = ImagePipeline(pipe_cfgs=[
...     BlurGauss(sigma=1.0),
...     Denoise(sigma=5.0, norm="clip")
... ])
>>> copied_pipe = NormControlMixin._disable_normalization(pipeline)
>>> # Only Denoise has a norm attribute, so only it is affected
>>> # _ops is a dict with operation names as keys
>>> [getattr(op, "norm", "absent") for op in copied_pipe._ops.values()]
['absent', None]

class phenotypic.sdk_.NormalizedOutputMixin(*, norm: Literal['clip', 'rescale'] | None = 'clip')[source]

Bases: BaseModel

Adds a norm field controlling how an operation’s output is range-guarded.

detect_mat is contractually [0, 1]. norm selects how an operation upholds that contract:

"clip" (default) saturates out-of-range values. It is the identity for in-range pixels, so absolute intensity is preserved and detect_mat stays comparable across a batch of plates.
"rescale" linearly remaps the full observed range onto [0, 1]. Ordering survives, absolute scale does not: a single specular highlight sets the max.
None passes values through untouched. Required inside a Generalized Anscombe Transform region (where the signal is deliberately not in [0, 1]) and by CompositeEnhance on non-normalized maps.

The field is appended to the end of the subclass’s field order rather than frontloaded, so an operation’s own parameters keep their natural position in model_json_schema() and to_json().

Note

Replaces the clip: bool field removed in 0.18.0. A bool cannot express "rescale", and the attribute name clip is claimed by NormControlMixin, which duck-types on it.

Parameters:: norm (Literal['clip', 'rescale'] | None)

classmethod __get_pydantic_json_schema__(core_schema: CoreSchema, handler: GetJsonSchemaHandler, /) → JsonSchemaValue

Hook into generating the model’s JSON schema.

Parameters:

core_schema (CoreSchema) – A pydantic-core CoreSchema. You can ignore this argument and call the handler with a new CoreSchema, wrap this CoreSchema ({‘type’: ‘nullable’, ‘schema’: current_schema}), or just call the handler with the original schema.
handler (GetJsonSchemaHandler) – Call into Pydantic’s internal JSON schema generation. This will raise a pydantic.errors.PydanticInvalidForJsonSchema if JSON schema generation fails. Since this gets called by BaseModel.model_json_schema you can override the schema_generator argument to that function to change JSON schema generation globally for a type.

Returns:

A JSON schema, as a Python object.

Return type:

JsonSchemaValue

classmethod __pydantic_init_subclass__(**kwargs: Any) → None[source]

Move norm to the end of the subclass’s field order.

Parameters:: kwargs (Any)
Return type:: None

classmethod __pydantic_on_complete__() → None

This is called once the class and its fields are fully initialized and ready to be used.

This typically happens when the class is created (just before [__pydantic_init_subclass__()][pydantic.main.BaseModel.__pydantic_init_subclass__] is called on the superclass), except when forward annotations are used that could not immediately be resolved. In that case, it will be called later, when the model is rebuilt automatically or explicitly using [model_rebuild()][pydantic.main.BaseModel.model_rebuild].

Return type:: None

classmethod construct(_fields_set: set[str] | None = None, **values: Any) → Self

Parameters:

_fields_set (set[str] | None)
values (Any)

Return type:

classmethod from_orm(obj: Any) → Self

Parameters:: obj (Any)
Return type:: Self

classmethod model_construct(_fields_set: set[str] | None = None, **values: Any) → Self

Creates a new instance of the Model class with validated data.

Creates a new model setting __dict__ and __pydantic_fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed.

!!! note: model_construct() generally respects the model_config.extra setting on the provided model. That is, if model_config.extra == ‘allow’, then all extra passed values are added to the model instance’s __dict__ and __pydantic_extra__ fields. If model_config.extra == ‘ignore’ (the default), then all extra passed values are ignored. Because no validation is performed with a call to model_construct(), having model_config.extra == ‘forbid’ does not result in an error if extra values are passed, but they will be ignored.

Parameters:

_fields_set (set[str] | None) – A set of field names that were originally explicitly set during instantiation. If provided, this is directly used for the [model_fields_set][pydantic.BaseModel.model_fields_set] attribute. Otherwise, the field names from the values argument will be used.
values (Any) – Trusted or pre-validated data dictionary.

Returns:

A new instance of the Model class with validated data.

Return type:

classmethod model_json_schema(by_alias: bool = True, ref_template: str = '#/$defs/{model}', schema_generator: type[~pydantic.json_schema.GenerateJsonSchema] = <class 'pydantic.json_schema.GenerateJsonSchema'>, mode: ~typing.Literal['validation', 'serialization'] = 'validation', *, union_format: ~typing.Literal['any_of', 'primitive_type_array'] = 'any_of') → dict[str, Any]

Generates a JSON schema for a model class.

Parameters:

by_alias (bool) – Whether to use attribute aliases or not.
ref_template (str) – The reference template.
union_format (Literal['any_of', 'primitive_type_array']) –
The format to use when combining schemas from unions together. Can be one of:
- ’any_of’: Use the [anyOf](https://json-schema.org/understanding-json-schema/reference/combining#anyOf)
keyword to combine schemas (the default). - ‘primitive_type_array’: Use the [type](https://json-schema.org/understanding-json-schema/reference/type) keyword as an array of strings, containing each type of the combination. If any of the schemas is not a primitive type (string, boolean, null, integer or number) or contains constraints/metadata, falls back to any_of.
schema_generator (type[GenerateJsonSchema]) – To override the logic used to generate the JSON schema, as a subclass of GenerateJsonSchema with your desired modifications
mode (Literal['validation', 'serialization']) – The mode in which to generate the schema.

Returns:

The JSON schema for the given model class.

Return type:

classmethod model_parametrized_name(params: tuple[type[Any], ...]) → str

Compute the class name for parametrizations of generic classes.

This method can be overridden to achieve a custom naming scheme for generic BaseModels.

Parameters:: params (tuple[type[Any], ...]) – Tuple of types of the class. Given a generic class Model with 2 type variables and a concrete model Model[str, int], the value (str, int) would be passed to params.
Returns:: String representing the new class where params are passed to cls as type variables.
Raises:: TypeError – Raised when trying to generate concrete names for non-generic models.
Return type:: str

classmethod model_rebuild(*, force: bool = False, raise_errors: bool = True, _parent_namespace_depth: int = 2, _types_namespace: MappingNamespace | None = None) → bool | None

Try to rebuild the pydantic-core schema for the model.

This may be necessary when one of the annotations is a ForwardRef which could not be resolved during the initial attempt to build the schema, and automatic rebuilding fails.

Parameters:

force (bool) – Whether to force the rebuilding of the model schema, defaults to False.
raise_errors (bool) – Whether to raise errors, defaults to True.
_parent_namespace_depth (int) – The depth level of the parent namespace, defaults to 2.
_types_namespace (MappingNamespace | None) – The types namespace, defaults to None.

Returns:

Returns None if the schema is already “complete” and rebuilding was not required. If rebuilding _was_ required, returns True if rebuilding was successful, otherwise False.

Return type:

bool | None

Validate a pydantic model instance.

Parameters:

obj (Any) – The object to validate.
strict (bool | None) – Whether to enforce types strictly.
extra (Literal['allow', 'ignore', 'forbid'] | None) – Whether to ignore, allow, or forbid extra data during model validation. See the [extra configuration value][pydantic.ConfigDict.extra] for details.
from_attributes (bool | None) – Whether to extract data from object attributes.
context (Any | None) – Additional context to pass to the validator.
by_alias (bool | None) – Whether to use the field’s alias when validating against the provided input data.
by_name (bool | None) – Whether to use the field’s name when validating against the provided input data.

Raises:

ValidationError – If the object could not be validated.

Returns:

The validated model instance.

Return type:

!!! abstract “Usage Documentation”: [JSON Parsing](../concepts/json.md#json-parsing)

Validate the given JSON data against the Pydantic model.

Parameters:

json_data (str | bytes | bytearray) – The JSON data to validate.
strict (bool | None) – Whether to enforce types strictly.
extra (Literal['allow', 'ignore', 'forbid'] | None) – Whether to ignore, allow, or forbid extra data during model validation. See the [extra configuration value][pydantic.ConfigDict.extra] for details.
context (Any | None) – Extra variables to pass to the validator.
by_alias (bool | None) – Whether to use the field’s alias when validating against the provided input data.
by_name (bool | None) – Whether to use the field’s name when validating against the provided input data.

Returns:

The validated Pydantic model.

Raises:

ValidationError – If json_data is not a JSON string or the object could not be validated.

Return type:

Validate the given object with string data against the Pydantic model.

Parameters:

obj (Any) – The object containing string data to validate.
strict (bool | None) – Whether to enforce types strictly.
extra (Literal['allow', 'ignore', 'forbid'] | None) – Whether to ignore, allow, or forbid extra data during model validation. See the [extra configuration value][pydantic.ConfigDict.extra] for details.
context (Any | None) – Extra variables to pass to the validator.
by_alias (bool | None) – Whether to use the field’s alias when validating against the provided input data.
by_name (bool | None) – Whether to use the field’s name when validating against the provided input data.

Returns:

The validated Pydantic model.

Return type:

classmethod parse_file(path: str | Path, *, content_type: str | None = None, encoding: str = 'utf8', proto: DeprecatedParseProtocol | None = None, allow_pickle: bool = False) → Self

Parameters:

path (str | Path)
content_type (str | None)
encoding (str)
proto (DeprecatedParseProtocol | None)
allow_pickle (bool)

Return type:

Self

classmethod parse_obj(obj: Any) → Self

Parameters:: obj (Any)
Return type:: Self

classmethod parse_raw(b: str | bytes, *, content_type: str | None = None, encoding: str = 'utf8', proto: DeprecatedParseProtocol | None = None, allow_pickle: bool = False) → Self

Parameters:

b (str | bytes)
content_type (str | None)
encoding (str)
proto (DeprecatedParseProtocol | None)
allow_pickle (bool)

Return type:

Self

classmethod schema(by_alias: bool = True, ref_template: str = '#/$defs/{model}') → Dict[str, Any]

Parameters:

by_alias (bool)
ref_template (str)

Return type:

classmethod schema_json(*, by_alias: bool = True, ref_template: str = '#/$defs/{model}', **dumps_kwargs: Any) → str

Parameters:

by_alias (bool)
ref_template (str)
dumps_kwargs (Any)

Return type:

classmethod update_forward_refs(**localns: Any) → None

Parameters:: localns (Any)
Return type:: None

classmethod validate(value: Any) → Self

Parameters:: value (Any)
Return type:: Self

__copy__() → Self

Returns a shallow copy of the model.

Return type:: Self

__deepcopy__(memo: dict[int, Any] | None = None) → Self

Returns a deep copy of the model.

Parameters:: memo (dict[int, Any] | None)
Return type:: Self

__init__(**data: Any) → None

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Parameters:: data (Any)
Return type:: None

__iter__() → Generator[tuple[str, Any], None, None]

So dict(model) works.

Return type:: Generator[tuple[str, Any], None, None]

__pretty__(fmt: Callable[[Any], Any], **kwargs: Any) → Generator[Any]

Used by devtools (https://python-devtools.helpmanual.io/) to pretty print objects.

Parameters:

fmt (Callable[[Any], Any])
kwargs (Any)

Return type:

Generator[Any]

__repr_name__() → str

Name of the instance’s class, used in __repr__.

Return type:: str

__repr_recursion__(object: Any) → str

Returns the string representation of a recursive object.

Parameters:: object (Any)
Return type:: str

__rich_repr__() → RichReprResult

Used by Rich (https://rich.readthedocs.io/en/stable/pretty.html) to pretty print objects.

Return type:: RichReprResult

Returns a copy of the model.

!!! warning “Deprecated”: This method is now deprecated; use model_copy instead.

If you need include or exclude, use:

`python {test="skip" lint="skip"} data = self.model_dump(include=include, exclude=exclude, round_trip=True) data = {**data, **(update or {})} copied = self.model_validate(data) `

Parameters:

include (AbstractSetIntStr | MappingIntStrAny | None) – Optional set or mapping specifying which fields to include in the copied model.
exclude (AbstractSetIntStr | MappingIntStrAny | None) – Optional set or mapping specifying which fields to exclude in the copied model.
update (Dict[str, Any] | None) – Optional dictionary of field-value pairs to override field values in the copied model.
deep (bool) – If True, the values of fields that are Pydantic models will be deep-copied.

Returns:

A copy of the model with included, excluded and updated fields as specified.

Return type:

Self

Parameters:

include (set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None)
exclude (set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None)
by_alias (bool)
exclude_unset (bool)
exclude_defaults (bool)
exclude_none (bool)

Return type:

Parameters:

include (set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None)
exclude (set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None)
by_alias (bool)
exclude_unset (bool)
exclude_defaults (bool)
exclude_none (bool)
encoder (Callable[[Any], Any] | None)
models_as_dict (bool)
dumps_kwargs (Any)

Return type:

model_copy(*, update: Mapping[str, Any] | None = None, deep: bool = False) → Self

!!! abstract “Usage Documentation”: [model_copy](../concepts/models.md#model-copy)

Returns a copy of the model.

!!! note: The underlying instance’s [__dict__][object.__dict__] attribute is copied. This might have unexpected side effects if you store anything in it, on top of the model fields (e.g. the value of [cached properties][functools.cached_property]).

Parameters:

update (Mapping[str, Any] | None) – Values to change/add in the new model. Note: the data is not validated before creating the new model. You should trust this data.
deep (bool) – Set to True to make a deep copy of the model.

Returns:

New model instance.

Return type:

!!! abstract “Usage Documentation”: [model_dump](../concepts/serialization.md#python-mode)

Generate a dictionary representation of the model, optionally specifying which fields to include or exclude.

Parameters:

mode (Literal['json', 'python'] | str) – The mode in which to_python should run. If mode is ‘json’, the output will only contain JSON serializable types. If mode is ‘python’, the output may contain non-JSON-serializable Python objects.
include (set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None) – A set of fields to include in the output.
exclude (set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None) – A set of fields to exclude from the output.
context (Any | None) – Additional context to pass to the serializer.
by_alias (bool | None) – Whether to use the field’s alias in the dictionary key if defined.
exclude_unset (bool) – Whether to exclude fields that have not been explicitly set.
exclude_defaults (bool) – Whether to exclude fields that are set to their default value.
exclude_none (bool) – Whether to exclude fields that have a value of None.
exclude_computed_fields (bool) – Whether to exclude computed fields. While this can be useful for round-tripping, it is usually recommended to use the dedicated round_trip parameter instead.
round_trip (bool) – If True, dumped values should be valid as input for non-idempotent types such as Json[T].
warnings (bool | Literal['none', 'warn', 'error']) – How to handle serialization errors. False/”none” ignores them, True/”warn” logs errors, “error” raises a [PydanticSerializationError][pydantic_core.PydanticSerializationError].
fallback (Callable[[Any], Any] | None) – A function to call when an unknown value is encountered. If not provided, a [PydanticSerializationError][pydantic_core.PydanticSerializationError] error is raised.
serialize_as_any (bool) – Whether to serialize fields with duck-typing serialization behavior.

Returns:

A dictionary representation of the model.

Return type:

!!! abstract “Usage Documentation”: [model_dump_json](../concepts/serialization.md#json-mode)

Generates a JSON representation of the model using Pydantic’s to_json method.

Parameters:

indent (int | None) – Indentation to use in the JSON output. If None is passed, the output will be compact.
ensure_ascii (bool) – If True, the output is guaranteed to have all incoming non-ASCII characters escaped. If False (the default), these characters will be output as-is.
include (set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None) – Field(s) to include in the JSON output.
exclude (set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None) – Field(s) to exclude from the JSON output.
context (Any | None) – Additional context to pass to the serializer.
by_alias (bool | None) – Whether to serialize using field aliases.
exclude_unset (bool) – Whether to exclude fields that have not been explicitly set.
exclude_defaults (bool) – Whether to exclude fields that are set to their default value.
exclude_none (bool) – Whether to exclude fields that have a value of None.
exclude_computed_fields (bool) – Whether to exclude computed fields. While this can be useful for round-tripping, it is usually recommended to use the dedicated round_trip parameter instead.
round_trip (bool) – If True, dumped values should be valid as input for non-idempotent types such as Json[T].
warnings (bool | Literal['none', 'warn', 'error']) – How to handle serialization errors. False/”none” ignores them, True/”warn” logs errors, “error” raises a [PydanticSerializationError][pydantic_core.PydanticSerializationError].
fallback (Callable[[Any], Any] | None) – A function to call when an unknown value is encountered. If not provided, a [PydanticSerializationError][pydantic_core.PydanticSerializationError] error is raised.
serialize_as_any (bool) – Whether to serialize fields with duck-typing serialization behavior.

Returns:

A JSON string representation of the model.

Return type:

model_post_init(context: Any, /) → None

Override this method to perform additional initialization after __init__ and model_construct. This is useful if you want to do some validation that requires the entire model to be initialized.

Parameters:: context (Any)
Return type:: None

model_computed_fields = {}

model_config: ClassVar[ConfigDict] = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

property model_extra: dict[str, Any] | None

Get extra fields set during validation.

Returns:: A dictionary of extra fields, or None if config.extra is not set to “allow”.

model_fields = {'norm': FieldInfo(annotation=Union[Literal['clip', 'rescale'], NoneType], required=False, default='clip')}

property model_fields_set: set[str]

Returns the set of fields that have been explicitly set on this model instance.

Returns:

A set of strings representing the fields that have been set,: i.e. that were not filled from defaults.

norm: NormOut

class phenotypic.sdk_.ProcessingStateKey[source]

Bases: object

Keys inside <output>/.phenotypic/processing_state.json.

Distinct from JobMetadataKey even where string values overlap (e.g. EXECUTION_MODE, INPUT_PATH) — these describe the processing_state.json contract, not the SLURM job metadata sidecar. Some values intentionally match across the two contracts so that a single field (like execution_mode) can be migrated atomically; the test_processing_state_keys_match_job_metadata_keys regression test asserts the overlap.

COMPLETED: Final[str] = 'completed'

CONFIG: Final[str] = 'config'

DATASETS: Final[str] = 'datasets'

ERRORS: Final[str] = 'errors'

EXECUTION_MODE: Final[str] = 'execution_mode'

FAILED: Final[str] = 'failed'

INITIAL_IMAGES: Final[str] = 'initial_images'

INPUT_PATH: Final[str] = 'input_path'

LAST_UPDATED: Final[str] = 'last_updated'

OUTPUT_DIR: Final[str] = 'output_dir'

PIPELINE_PATH: Final[str] = 'pipeline_path'

STARTED: Final[str] = 'started'

TIMESTAMP: Final[str] = 'timestamp'

VERSION: Final[str] = 'version'

phenotypic.sdk_.analysis_full_parquet_path(progress_dir_: Path) → Path[source]

Return <progress>/analysis_full.parquet (incremental combined frame).

Takes a progress_dir (not the run output root) since this file lives inside progress/ — it’s an internal mid-run artifact, not a user- facing output.

Parameters:: progress_dir_ (Path)
Return type:: Path

phenotypic.sdk_.analysis_html_path(output_dir: Path) → Path[source]

Return <output>/deliverables/analysis.html.

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.analysis_manifest_path(deliverables_base: Path) → Path[source]

Return the analysis manifest inside a deliverables directory.

Parameters:: deliverables_base (Path)
Return type:: Path

phenotypic.sdk_.analysis_scatter_json_path(progress_dir_: Path) → Path[source]

Return <progress>/analysis_scatter.json.

Parameters:: progress_dir_ (Path)
Return type:: Path

phenotypic.sdk_.atomic_write_bytes(path: str | Path, data: bytes, *, pre_replace: Callable[[], None] | None = None) → None[source]

Atomically write data to path (temp sibling + os.replace).

The bytes counterpart of atomic_write_text() for binary payloads (e.g. a serialized parquet buffer). Same crash-safety guarantees: an all-or-nothing replace and no partial/leftover temp file on failure.

Parameters:

path (str | Path) – The destination file path.
data (bytes) – The full binary payload to write.
pre_replace (Callable[[], None] | None) – Optional callback invoked immediately before publication.

Raises:

OSError – If the write or rename fails (the temp file is removed first).

Return type:

None

phenotypic.sdk_.atomic_write_json(path: str | Path, payload: Mapping[str, Any] | list[Any], *, indent: int = 2, sort_keys: bool = True, ensure_ascii: bool = False, pre_replace: Callable[[], None] | None = None) → None[source]

Atomically write a JSON payload with deterministic formatting.

Parameters:

path (str | Path) – Destination JSON path.
payload (Mapping[str, Any] | list[Any]) – JSON-serializable mapping or list.
indent (int) – Indentation passed to json.dumps().
sort_keys (bool) – Whether mapping keys are sorted for deterministic output.
ensure_ascii (bool) – Whether non-ASCII characters are escaped.
pre_replace (Callable[[], None] | None) – Optional callback invoked immediately before publication.

Return type:

None

phenotypic.sdk_.atomic_write_parquet(path: str | Path, frame: Any, **kwargs: Any) → None[source]

Atomically write a pandas-like frame with shared parquet defaults.

Parameters:

path (str | Path) – Destination parquet path.
frame (Any) – Object exposing to_parquet(path, **kwargs).
**kwargs (Any) – Per-call parquet writer overrides.

Return type:

None

phenotypic.sdk_.atomic_write_text(path: str | Path, text: str, *, encoding: str = 'utf-8', pre_replace: Callable[[], None] | None = None) → None[source]

Atomically write text to path (temp sibling + os.replace).

A drop-in replacement for Path(path).write_text(text) that never leaves a half-written file: a concurrent reader sees either the old contents or the complete new ones, and an exception mid-write leaves any pre-existing file intact with no .tmp debris.

Parameters:

path (str | Path) – The destination file path.
text (str) – The full text payload to write.
encoding (str) – The text encoding (default "utf-8").
pre_replace (Callable[[], None] | None) – Optional callback invoked immediately before publication.

Raises:

OSError – If the write or rename fails (the temp file is removed first).

Return type:

None

phenotypic.sdk_.atomic_write_with_writer(path: str | Path, writer: Callable[[str], None], *, pre_replace: Callable[[], None] | None = None) → None[source]

Atomically write path using a callback that receives a temp path.

Parameters:

path (str | Path) – Final destination path.
writer (Callable[[str], None]) – Callable that writes complete output to a temporary path string.
pre_replace (Callable[[], None] | None) – Optional callback invoked after the temporary file is synced, immediately before it replaces path. An exception aborts publication and removes the temporary file.

Raises:

OSError – Propagated from the writer or rename after temp cleanup.

Return type:

None

phenotypic.sdk_.best_params_path(output_dir: Path) → Path[source]

Return <output>/deliverables/best_params.json (winner params sidecar).

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.best_pipeline_path(output_dir: Path) → Path[source]

Return the canonical typed tuned-winner pipeline path.

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.bytes_fingerprint(data: bytes) → str[source]

Return a versioned SHA-256 fingerprint for exact bytes.

Parameters:: data (bytes) – Bytes to fingerprint.
Returns:: A "sha256:<hex>" content fingerprint.
Return type:: str

phenotypic.sdk_.canonical_metadata_order() → dict[str, int][source]

Global rank for every known metadata header (cluster then definition order).

Cluster-order major, enum definition-order minor. A header absent from this map is an unknown/uncategorized user tag; callers rank those last. The map is derived entirely from the import-time schema enums, so it is cached. The returned dict is read-only by contract; callers must not mutate it (mirrors header_to_module()).

Return type:: dict[str, int]

phenotypic.sdk_.checkpoint_lock_filename(checkpoint_type: Literal['manifest', 'finalize']) → str[source]

Filename of the SLURM-sentinel exclusive lock for a checkpoint task.

Parameters:: checkpoint_type (Literal['manifest', 'finalize']) – "manifest" or "finalize". Validated by the phenotypic.sdk_.typing_.CheckpointType Literal alias at type-check time.
Returns:: Filename relative to <progress>/ (hidden file, leading .).
Return type:: str

phenotypic.sdk_.checkpoint_lock_path(progress_dir_: Path, checkpoint_type: Literal['manifest', 'finalize']) → Path[source]

Return <progress>/.{checkpoint_type}_lock.

Parameters:

progress_dir_ (Path)
checkpoint_type (Literal['manifest', 'finalize'])

Return type:

phenotypic.sdk_.chunk_lock_path(progress_dir_: Path) → Path[source]

Return <progress>/.chunk_lock.

Parameters:: progress_dir_ (Path)
Return type:: Path

phenotypic.sdk_.chunk_manifest_path(output_dir: Path) → Path[source]

Return <output>/.phenotypic/progress/chunk_manifest.json.

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.chunk_parquet_filename(chunk_id: int) → str[source]

Filename of a dashboard chunk Parquet (zero-padded chunk id).

Parameters:: chunk_id (int) – Zero-based chunk index, formatted {chunk_id:03d}.
Returns:: Filename relative to <progress>/chunks/.
Return type:: str

phenotypic.sdk_.chunk_parquet_path(progress_dir_: Path, chunk_id: int) → Path[source]

Return <progress>/chunks/chunk_<id:03d>.parquet.

Parameters:

progress_dir_ (Path)
chunk_id (int)

Return type:

phenotypic.sdk_.chunk_state_path(output_dir: Path) → Path[source]

Return <output>/.phenotypic/progress/chunk_state.json.

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.chunks_dir(progress_dir_: Path) → Path[source]

Return <progress>/chunks/ for the dashboard chunk parquets.

Parameters:: progress_dir_ (Path)
Return type:: Path

phenotypic.sdk_.clear_machine_state(output_dir: Path) → bool[source]

Remove all of a run’s machine-state for a clean --restart.

Deletes the .phenotypic/ cache (progress/, processing_state.json, processing_events.log, logs, and generated SLURM scripts) and any pre-migration root-level machine-state, while leaving user-facing output artifacts (deliverables/, results/, qc/, …) untouched. This is the difference between --restart (re-run the orchestration against clean state, keep outputs) and --overwrite (delete the whole output dir). Clearing the event log here is what stops a restart from appending to — and rebuilding its manifest/failure records from — the prior run’s events.

Returns:: True if any machine-state was removed, else False.
Parameters:: output_dir (Path)
Return type:: bool

phenotypic.sdk_.curation_labels_parquet_path(output_dir: Path) → Path[source]

Return <output>/deliverables/qc/curation_labels.parquet (durable labels store).

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.custom_categories_json_path(output_dir: Path) → Path[source]

Return <output>/deliverables/qc/custom_categories.json (custom-category registry).

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.dashboard_html_path(output_dir: Path) → Path[source]

Return <output>/deliverables/dashboard.html.

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.dataset_hdf_dir(output_dir: Path, dataset: str) → Path[source]

Return <output>/results/<dataset>/hdf/.

Parameters:

output_dir (Path)
dataset (str)

Return type:

phenotypic.sdk_.dataset_measurements_dir(output_dir: Path, dataset: str) → Path[source]

Return <output>/results/<dataset>/measurements/.

Parameters:

output_dir (Path)
dataset (str)

Return type:

phenotypic.sdk_.dataset_overlays_dir(output_dir: Path, dataset: str) → Path[source]

Return <output>/deliverables/overlays/<dataset>/.

Parameters:

output_dir (Path)
dataset (str)

Return type:

phenotypic.sdk_.dataset_results_dir(output_dir: Path, dataset: str) → Path[source]

Return <output>/results/<dataset>/.

Parameters:

output_dir (Path)
dataset (str)

Return type:

phenotypic.sdk_.default_output_dir_name(now: datetime | None = None) → str[source]

Default name for an auto-generated output directory.

Legacy helper for timestamped output-directory names.

Parameters:: now (datetime | None) – Override clock for tests; defaults to datetime.now().
Returns:: "phenotypic_results_YYYYMMDD_HHMMSS".
Return type:: str

phenotypic.sdk_.deliverables_dir(output_dir: Path) → Path[source]

Return <output>/deliverables/ — the user-facing-output folder.

Pure path expression; callers are responsible for mkdir when they intend to write into it. Writers that go through phenotypic.sdk_.atomic_write_with_writer() get the mkdir for free (it creates target.parent); direct write_text/write_bytes writers must mkdir explicitly.

Every artifact helper that previously rooted at <output>/ (master / measurements / per-feature split / analysis / dashboard / report / pipeline.json / README) now composes from here, so a future relocation is a one-line change.

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.ensure_metadata_prefix(name: str) → str[source]

Prefix a bare metadata label with its schema category, else generic.

Strain -> MetadataGenetic_Strain (the owning enum’s category); an unknown Foo -> Metadata_Foo (kept, uncategorized). Names that already carry a metadata prefix – any Metadata<Topic>_ category prefix or the generic Metadata_ – pass through unchanged.

Parameters:: name (str)
Return type:: str

phenotypic.sdk_.ensure_typed_json_suffix(path: str | Path, suffix: str) → Path[source]

Return path with the canonical typed JSON suffix appended.

Bare stems receive the full typed suffix. Legacy .json paths receive only the typed tail, preserving the user-provided stem and case.

Parameters:

path (str | Path)
suffix (str)

Return type:

phenotypic.sdk_.error_analysis_csv_path(output_dir: Path) → Path[source]

Return <output>/deliverables/error_analysis.csv.

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.error_analysis_html_path(output_dir: Path) → Path[source]

Return <output>/deliverables/error_analysis.html.

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.error_analysis_parquet_path(output_dir: Path) → Path[source]

Return <output>/deliverables/error_analysis.parquet.

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.error_category_parquet_path(output_dir: Path, category: str) → Path[source]

Return <output>/deliverables/errors/<category>.parquet.

Parameters:

output_dir (Path) – Run output directory.
category (str) – A bare, already-sanitized category token (e.g. "background_noise"). The caller is responsible for sanitization.

Return type:

phenotypic.sdk_.errors_dir(output_dir: Path) → Path[source]

Return <output>/deliverables/errors/ (per-category error parquets).

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.event_log_path(output_dir: Path) → Path[source]

Return <output>/.phenotypic/processing_events.log.

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.failures_jsonl_path(output_dir: Path) → Path[source]

Return <output>/.phenotypic/progress/failures.jsonl.

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.file_fingerprint(path: Path) → str[source]

Return a versioned SHA-256 fingerprint for one file’s contents.

Parameters:: path (Path) – Existing regular file to fingerprint.
Returns:: A "sha256:<hex>" content fingerprint.
Return type:: str

phenotypic.sdk_.generalization_path(output_dir: Path) → Path[source]

Return <output>/deliverables/generalization.json — the held-out report.

The winner’s generalization verdict (calibration vs held-out score, the gap, and the pass/fail margin), a user-facing deliverable. The held-out pass that writes it is Phase 4.5 part 2; this helper resolves the canonical location.

Parameters:: output_dir (Path) – The run output directory.
Returns:: <output_dir>/deliverables/generalization.json.
Return type:: Path

phenotypic.sdk_.generation_staging_path(target: Path, generation: str) → Path[source]

Return a sibling staging path for an explicit publication generation.

Raises:

ValueError – If generation is not a safe path component.

Parameters:

target (Path)
generation (str)

Return type:

phenotypic.sdk_.gui_launch_owner_path(output_dir: Path) → Path[source]

Return the canonical GUI launch-generation owner record path.

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.has_config_suffix(path: str | Path, suffixes: Iterable[str]) → bool[source]

Return whether path ends with any configured suffix.

Matching is case-insensitive so callers can discover user-provided files from case-preserving filesystems without rewriting their names.

Parameters:

path (str | Path)
suffixes (Iterable[str])

Return type:

bool

phenotypic.sdk_.is_binary_mask(arr: numpy.ndarray)[source]

Parameters:: arr (numpy.ndarray)

phenotypic.sdk_.is_metadata_header(col: str) → bool[source]

True if col is a metadata-family column.

Matches any per-topic Metadata<Topic>_ prefix (e.g. MetadataGenetic_) and the generic Metadata_ fallback used for uncategorized user metadata, per the namespace design (spec §10.3): the whole Metadata family is recognized, measurement/identity columns (Shape_, Object_, …) are not.

Parameters:: col (str)
Return type:: bool

phenotypic.sdk_.job_metadata_path(output_dir: Path) → Path[source]

Return <output>/.phenotypic/progress/job_metadata.json.

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.load_image_from_hdf(hdf_path: Path, *, fallback: ImageTypeName = 'Image') → _Image | _GridImage[source]

Open an HDF5, read its phenotypic_class attr, dispatch to the right Image class.

Replaces the 3 ad-hoc h5py.File(...) → fh.attrs.get('phenotypic_class', 'Image') → GridImage if cls_attr == 'GridImage' else Image patterns in _cli_recompile_worker, _cli_execution_strategies, and phenotypicCLI.

Parameters:

hdf_path (Path) – Path to a per-image HDF5 file.
fallback (ImageTypeName) – Image class name to use when the HDF lacks the phenotypic_class attribute (legacy files). Type-checked (statically) against ImageTypeName — there is no runtime validation; the only effect of an unrecognized string is that the dispatch falls through to Image.

Returns:

An Image or GridImage instance loaded from the HDF.

Return type:

_Image | _GridImage

phenotypic.sdk_.load_master_measurements(output_dir: Path) → 'pl.DataFrame' | None[source]

Read <output>/master_measurements.csv into a polars DataFrame.

Parameters:: output_dir (Path) – Run output directory.
Returns:: DataFrame, or None when the file is missing.
Return type:: Optional[‘pl.DataFrame’]

phenotypic.sdk_.logs_dir(output_dir: Path) → Path[source]

Return <output>/.phenotypic/logs/.

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.manifest_json_path(output_dir: Path) → Path[source]

Return <output>/.phenotypic/progress/manifest.json.

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.master_measurements_csv_path(output_dir: Path) → Path[source]

Return <output>/deliverables/master_measurements.csv.

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.master_measurements_parquet_path(output_dir: Path) → Path[source]

Return <output>/deliverables/master_measurements.parquet.

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.matches_any_suffix(path: str | Path, suffixes: Iterable[str]) → bool[source]

Return whether path ends with any suffix in suffixes.

Parameters:

path (str | Path)
suffixes (Iterable[str])

Return type:

bool

phenotypic.sdk_.measurements_by_feature_dir(output_dir: Path) → Path[source]

Return <output>/deliverables/measurements_by_feature/.

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.measurements_csv_path(output_dir: Path) → Path[source]

Return <output>/deliverables/measurements.csv (post-applied mirror).

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.measurements_parquet_path(output_dir: Path) → Path[source]

Return <output>/deliverables/measurements.parquet (post-applied mirror).

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.metadata_category_for_label(label: str) → str | None[source]

Category that owns a bare label ('Strain' -> 'MetadataGenetic'), or None.

Parameters:: label (str)
Return type:: str | None

phenotypic.sdk_.metadata_category_prefixes() → tuple[str, ...][source]

All metadata category prefixes (e.g. 'MetadataGenetic_') in cluster order.

Ordered by the bio-semantic cluster order (_METADATA_CLUSTER_ORDER), then deduplicated, so callers building bucket-priority lists get a stable, canonical ordering. REMBI is a separate axis (see by_module / header_to_module).

Return type:: tuple[str, …]

phenotypic.sdk_.metadata_csv_deliverable_path(output_dir: Path) → Path[source]

Return <output>/deliverables/metadata.csv (co-located --metadata copy).

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.metadata_only_mask(df: pandas.DataFrame) → pandas.Series[source]

Mask of --metadata phantom rows; all-False when unknowable.

A phantom row is one the CLI’s --metadata left join carried through from the metadata CSV even though no measured object matched its key — every measurement/info column on it is null. Those rows are marked with the METADATA_ONLY (QC_MetadataOnly) boolean column.

The flag is CLI-only, so public analysis/post entry points that a user calls on a hand-built or measure() frame see no flag at all. This helper degrades to an all-False mask in that case, which reproduces exactly the pre-left-join behavior for every caller.

The dtype check is deliberately strict: only a real boolean column is trusted. An object/string column is rejected rather than coerced, because pd.Series(["False", "True"]).astype(bool) is [True, True] — the string "False" is truthy — which would silently mark every row a phantom. Rejecting costs nothing (it falls back to today’s behavior); a lenient coercion would corrupt every result. Both real CLI round-trips preserve the dtype: parquet stores a native bool, and polars’ write_csv emits true/false, which pd.read_csv parses to bool.

Parameters:: df (pandas.DataFrame) – Any measurement-shaped DataFrame.
Returns:: True where the row is a metadata-only phantom, False everywhere else (and everywhere when the flag column is absent or not a boolean column).
Return type:: Boolean Series aligned to df.index

Examples

>>> import pandas as pd
>>> from phenotypic.sdk_ import metadata_only_mask
>>> # A frame from a notebook ``image.measure()`` carries no flag.
>>> metadata_only_mask(pd.DataFrame({"Shape_Area": [10.0, 12.0]})).tolist()
[False, False]
>>> # A CLI mirror does: the undetected strain is flagged.
>>> mirror = pd.DataFrame({"QC_MetadataOnly": [False, True]})
>>> metadata_only_mask(mirror).tolist()
[False, True]

phenotypic.sdk_.migrate_legacy_machine_state(output_dir: Path) → bool[source]

Move a pre-migration run’s machine-state into .phenotypic/.

If legacy machine-state (progress/, processing_state.json, processing_events.log) is present at the output root, move each artifact into the .phenotypic/ cache so the run proceeds coherently against a single location. A no-op when no legacy state is present or everything is already migrated.

Robust to interruption and concurrency (the SLURM array case): each artifact is moved only when its source still exists and its destination does not, so a migration interrupted mid-move completes on the next call rather than leaving split state; and a lost move race (a concurrent worker moved the artifact first) is ignored rather than crashing. Keying per-artifact instead of on cache.exists() is what makes both safe.

Returns:: True if this call moved anything, else False.
Parameters:: output_dir (Path)
Return type:: bool

phenotypic.sdk_.migrate_legacy_qc(output_dir: Path) → bool[source]

Move a pre-relocation run’s <output>/qc/ into deliverables/qc/.

Hard cutover (MOVE, no duplication), mirroring migrate_legacy_machine_state(). A no-op when there is no legacy qc/ or when the canonical deliverables/qc/ already exists (the move is whole-directory; we never merge a half-written canonical with legacy).

Returns:: True if this call moved the directory, else False.
Parameters:: output_dir (Path)
Return type:: bool

phenotypic.sdk_.migration_backup_dir(config_path: Path) → Path[source]

Return the dedicated sibling backup directory for a configuration.

Parameters:: config_path (Path)
Return type:: Path

phenotypic.sdk_.migration_backup_path(config_path: Path, *, timestamp: str, source_fingerprint: str) → Path[source]

Return a timestamped, fingerprinted backup path.

Parameters:

config_path (Path)
timestamp (str)
source_fingerprint (str)

Return type:

phenotypic.sdk_.migration_lock_path(config_path: Path) → Path[source]

Return the shared pipeline publication lock used by migrations.

This compatibility alias preserves the original SDK name while ensuring migrations coordinate with every ordinary canonical pipeline writer.

Parameters:: config_path (Path)
Return type:: Path

phenotypic.sdk_.migration_receipt_path(config_path: Path, *, resulting_fingerprint: str) → Path[source]

Return the durable receipt path for one migrated generation.

Parameters:

config_path (Path)
resulting_fingerprint (str)

Return type:

phenotypic.sdk_.named_analysis_csv_path(deliverables_base: Path, analysis_id: str) → Path[source]

Return the named CSV artifact path for analysis_id.

Parameters:

deliverables_base (Path)
analysis_id (str)

Return type:

phenotypic.sdk_.named_analysis_parquet_path(deliverables_base: Path, analysis_id: str) → Path[source]

Return the named Parquet artifact path for analysis_id.

Parameters:

deliverables_base (Path)
analysis_id (str)

Return type:

phenotypic.sdk_.named_analysis_paths(deliverables_base: Path, analysis_id: str) → AnalysisArtifactPaths[source]

Return all persisted paths associated with one analysis ID.

Parameters:

deliverables_base (Path)
analysis_id (str)

Return type:

AnalysisArtifactPaths

phenotypic.sdk_.order_measurement_columns(columns: Sequence[str]) → list[str][source]

Canonical measurement-frame column order.

[front metadata] -> [measurements] -> [MetadataImage_*] -> [info block].

Front (user/experimental) metadata is cluster/definition ordered via canonical_metadata_order(); unknown/uncategorized Metadata_* tags fall to the end of the front block alphabetically. The framework MetadataImage_* block is per-image provenance and trails the measurements. The per-object info block (Object_Label + Bbox_* / Grid_*) is detected by name and moves last. Measurements keep their incoming relative order.

Pure over column-name strings, so both the pandas (df[...]) and polars (df.select(...)) paths reuse it.

Parameters:: columns (Sequence[str])
Return type:: list[str]

phenotypic.sdk_.overlay_manifest_path(output_dir: Path) → Path[source]

Return <output>/deliverables/overlays/overlay_manifest.json.

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.overlays_dir(output_dir: Path) → Path[source]

Return <output>/deliverables/overlays/ — the overlay package root.

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.param_importance_path(output_dir: Path) → Path[source]

Return <output>/deliverables/param_importance.json (the report).

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.pareto_best_pipeline_path(output_dir: Path, objective: str) → Path[source]

Return deliverables/pareto/best_<objective>.json (a per-axis winner).

The pipeline maximizing the single objective axis on the Pareto front. objective is the objective name as it appears in objectives_json (a scorer-defined label, e.g. "Dice" or a composite child handle "s0").

Parameters:

output_dir (Path) – The run directory.
objective (str) – The objective-axis name (the best_<objective>.json stem).

Returns:

The per-objective best-pipeline path under pareto_dir().

Return type:

phenotypic.sdk_.pareto_dir(output_dir: Path) → Path[source]

Return <output>/deliverables/pareto/ — the multi-objective sub-folder.

Holds a multi-objective tune run’s Pareto front + per-objective best pipelines. A single-objective run never creates it (the back-compat lock).

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.pareto_front_parquet_path(output_dir: Path) → Path[source]

Return <output>/deliverables/pareto/pareto_front.parquet (the front).

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.pareto_importance_path(output_dir: Path, objective: str) → Path[source]

Return deliverables/pareto/param_importance_<objective>.json.

The per-objective RF-permutation importance report (the multi-objective sibling of param_importance_path()). objective is the objective name as it appears in objectives_json (a scorer-defined label, e.g. "Dice" or a composite child handle "s0").

Parameters:

output_dir (Path) – The run directory.
objective (str) – The objective-axis name (the filename’s <objective> slot).

Returns:

The per-objective importance-report path under pareto_dir().

Return type:

phenotypic.sdk_.paths_fingerprint(paths: Iterable[Path], *, root: Path | None = None) → str[source]

Fingerprint an ordered set of named filesystem entries without writes.

Files include their exact contents, directories include their normalized names, and missing entries are represented explicitly. Sorting by name makes the result independent of caller enumeration order.

Parameters:

paths (Iterable[Path]) – Files to include.
root (Path | None) – Optional anchor used to normalize names.

Returns:

A deterministic "sha256:<hex>" fingerprint.

Return type:

phenotypic.sdk_.phenotypic_cache_dir(output_dir: Path) → Path[source]

Return <output>/.phenotypic/ — the hidden machine-state root.

Pure path expression; callers mkdir when they intend to write.

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.phenotypic_cache_pipeline_json_path(output_dir: Path) → Path[source]

Return <output>/.phenotypic/pipeline.json.pht-pipe — the process-only run’s reproducibility copy. Distinct from pipeline_json_path(), which roots under deliverables/ (process-only writes no deliverables).

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.pipeline_json_path(output_dir: Path) → Path[source]

Return the canonical typed pipeline config path under deliverables/.

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.pipeline_publication_lock(config_path: Path, *, timeout: float = 30.0) → Iterator[None][source]

Serialize a complete read/check/write pipeline transaction.

Callers must hold this lock across both their generation or staleness check and the final atomic replacement. Atomic rename prevents partial files; this lock prevents a valid concurrent generation from being silently overwritten after a check-then-replace race.

Parameters:

config_path (Path) – Canonical pipeline configuration being published.
timeout (float) – Maximum seconds to wait for a competing publisher.

Yields:

None while the shared publication lock is held.

Return type:

Iterator[None]

phenotypic.sdk_.pipeline_publication_lock_path(config_path: Path) → Path[source]

Return the shared interprocess lock path for pipeline publication.

Every production writer of a canonical output pipeline must acquire this lock before checking a source generation or replacing the file. The lock is intentionally about publication, not one particular migration, so CLI, QC, Analysis, and compatibility writers serialize against each other.

Legacy pipeline.json and canonical pipeline.json.pht-pipe paths intentionally map to the same output-level identity. This prevents a V1 reader/writer and a V2 writer from bypassing one another merely because they selected different compatibility filenames.

Parameters:: config_path (Path)
Return type:: Path

phenotypic.sdk_.plots_dir(output_dir: Path) → Path[source]

Return <output>/deliverables/plots/.

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.processing_report_html_path(output_dir: Path) → Path[source]

Return <output>/deliverables/processing_report.html.

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.processing_state_path(output_dir: Path) → Path[source]

Return <output>/.phenotypic/processing_state.json.

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.progress_dir(output_dir: Path) → Path[source]

Return <output>/.phenotypic/progress/.

Pure path expression; callers are responsible for mkdir when they intend to write into it.

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.qc_dir(output_dir: Path) → Path[source]

Return <output>/deliverables/qc/ — durable QC + curation state.

Relocated under deliverables/ so a deliverables bundle is self-contained and portable. Use resolve_qc_dir() for reads that must honour the legacy root <output>/qc/ layout of pre-relocation runs.

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.qc_duckdb_path(output_dir: Path) → Path[source]

Return <output>/deliverables/qc/qc.duckdb.

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.qc_review_state_path(output_dir: Path) → Path[source]

Return <output>/deliverables/qc/review_state.json (GUI-owned review progress).

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.read_run_manifest(output_dir: Path) → dict | None[source]

Read the run manifest if present, resolving legacy layouts.

Reads <output>/.phenotypic/progress/manifest.json, falling back to the pre-migration <output>/progress/manifest.json for legacy runs (via resolve_manifest_json_path()). Replaces 4 inline json.loads(manifest_path.read_text()) blocks.

Parameters:: output_dir (Path) – Run output directory containing progress/.
Returns:: Parsed manifest dict, or None when the file is missing or unparseable (callers can decide whether absence is fatal).
Return type:: dict | None

phenotypic.sdk_.readme_md_path(output_dir: Path) → Path[source]

Return <output>/deliverables/README.md.

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.recompile_dir(progress_dir_: Path) → Path[source]

Return <progress>/recompile/.

Parameters:: progress_dir_ (Path)
Return type:: Path

phenotypic.sdk_.recompile_status_dir(progress_dir_: Path) → Path[source]

Return <progress>/recompile/status/.

Parameters:: progress_dir_ (Path)
Return type:: Path

phenotypic.sdk_.rembi_manifest_path(output_dir: Path) → Path[source]

Return <output>/deliverables/rembi.yaml — the REMBI run manifest.

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.resolve_best_pipeline_path(output_dir: Path) → Path[source]

Return the best existing tuned-winner pipeline path for output_dir.

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.resolve_event_log_path(output_dir: Path) → Path[source]

Return the event log sibling of the resolved progress dir.

The event log lives beside progress/ (D14): in .phenotypic/ for a migrated/new run, at the output root for a not-yet-migrated legacy read. Read-only helper for resume/discovery; never mutates the run dir.

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.resolve_execution_mode(job_meta: dict | None) → Literal['local', 'slurm'][source]

Extract ExecutionMode from job metadata, defaulting to "local".

Replaces a 5-site copy-paste of the job_meta.get("execution_mode", "local") if job_meta else "local" pattern.

Silent coercion: any value that isn’t exactly "slurm" collapses to "local" — including None (no metadata file), {} (no key), garbage strings (e.g. "validate", ""), and the literal None value. The function never raises. Callers who need to detect an unknown mode and warn / refuse should inspect job_meta directly before calling this helper.

Parameters:: job_meta (dict | None) – Parsed .phenotypic/progress/job_metadata.json content, or None when the file is absent.
Returns:: "local" or "slurm".
Return type:: Literal[‘local’, ‘slurm’]

phenotypic.sdk_.resolve_manifest_json_path(output_dir: Path) → Path[source]

Return <progress>/manifest.json resolving the progress dir for legacy runs.

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.resolve_pipeline_config_path(output_dir: Path) → Path[source]

Return the best existing pipeline config path for output_dir.

Resolution prefers the canonical typed path, falls back to legacy pipeline.json when present, and returns the canonical path when neither exists so writers naturally create typed config files.

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.resolve_processing_state_path(output_dir: Path) → Path[source]

Return the processing-state file that exists, preferring .phenotypic/.

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.resolve_progress_dir(output_dir: Path) → Path[source]

Return the progress dir that exists, preferring .phenotypic/.

Read-only helper for resume/discovery so a pre-migration run (progress at the output root) is still found. Falls back to the new location when neither exists (the default for fresh writes).

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.resolve_qc_dir(output_dir: Path) → Path[source]

Return the qc dir that exists, preferring deliverables/qc/.

Read-only resolver: deliverables/qc if present, else legacy root qc if present, else the canonical deliverables/qc (for fresh writes).

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.resolve_split_assignment_path(output_dir: Path) → Path[source]

Return the split assignment that exists, preferring .pht-tune-cache/.

Read-only resolver mirroring resolve_progress_dir(). The held-out split is checked in the hidden tune cache FIRST, THEN at the legacy output root — a missing split silently RE-DERIVES a fresh held-out partition on resume (a reproducibility / held-out-leak bug), so resume MUST find a legacy-root split.json. Falls back to the new location when neither exists (the default for a fresh derive-and-write).

Parameters:: output_dir (Path) – The run output directory.
Returns:: The split-assignment path that exists, else the new cache location.
Return type:: Path

phenotypic.sdk_.resolve_study_db_path(output_dir: Path) → Path[source]

Return the study DB that exists, preferring .pht-tune-cache/.

Read-only resolver: a relocated run keeps study.db under the hidden tune cache; a legacy run kept it at the output root. Falls back to the new location when neither exists (so a cold sampler restart from a missing study.db is harmless — no migration is performed).

Parameters:: output_dir (Path) – The run output directory.
Returns:: The study DB path that exists, else the new cache location.
Return type:: Path

phenotypic.sdk_.resolve_tuning_spec_path(output_dir: Path) → Path[source]

Return the best existing tuning spec path for output_dir.

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.results_dir(output_dir: Path) → Path[source]

Return <output>/results/.

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.run_completion_marker_path(output_dir: Path) → Path[source]

Return the canonical generation-bearing completion marker path.

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.sentinel_resubmitted_path(progress_dir_: Path) → Path[source]

Return <progress>/sentinel_resubmitted marker file path.

Parameters:: progress_dir_ (Path)
Return type:: Path

phenotypic.sdk_.shard_parquet_filename(shard_id: int) → str[source]

Filename of a per-shard Parquet inside the recompile worker.

Parameters:: shard_id (int) – Zero-based shard index.
Returns:: Filename relative to the recompile shard directory.
Return type:: str

phenotypic.sdk_.slurm_scripts_dir(output_dir: Path) → Path[source]

Return <output>/.phenotypic/slurm_scripts/.

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.source_cache_key(source: Path, fingerprint: str) → str[source]

Return an opaque cache key bound to canonical source and content.

Parameters:

source (Path)
fingerprint (str)

Return type:

phenotypic.sdk_.task_status_filename(task_index: int) → str[source]

Filename of a per-task SLURM-recompile status JSON.

Parameters:: task_index (int) – Zero-based recompile task index.
Returns:: Filename relative to <progress>/recompile/status/.
Return type:: str

phenotypic.sdk_.task_status_path(output_dir: Path, task_index: int) → Path[source]

Return <progress>/recompile/status/task_<idx>.json.

Parameters:

output_dir (Path)
task_index (int)

Return type:

phenotypic.sdk_.timed_execution(func)[source]: Decorator to measure and print the execution time of a function.

phenotypic.sdk_.trials_parquet_path(output_dir: Path) → Path[source]

Return <output>/trials.parquet (the trial journal; output-dir root).

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.tune_cache_dir(output_dir: Path) → Path[source]

Return <output>/.pht-tune-cache/ — the tune run’s machine-state root.

The tune-side sibling of phenotypic_cache_dir(). Pure path expression; callers mkdir when they intend to write.

Parameters:: output_dir (Path) – The run output directory.
Returns:: <output_dir>/.pht-tune-cache/.
Return type:: Path

phenotypic.sdk_.tune_cache_run_marker_path(output_dir: Path) → Path[source]

Return <output>/.pht-tune-cache/run.json — the tune-run marker.

Written at run START (before any deliverable lands) so a live or finished tune output is GUI-discoverable. See RUN_MARKER_JSON.

Parameters:: output_dir (Path) – The run output directory.
Returns:: <output_dir>/.pht-tune-cache/run.json.
Return type:: Path

phenotypic.sdk_.tune_cache_split_assignment_path(output_dir: Path) → Path[source]

Return <output>/.pht-tune-cache/splits/split.json — the held-out split.

The persisted calibration / held-out partition (plate names + split kind + dataset identity + seed entropy). Read-if-exists-else-derive on resume, so a re-run reuses the original partition regardless of the new master seed. A legacy run wrote it under <output>/splits/; use resolve_split_assignment_path() to read either location.

Parameters:: output_dir (Path) – The run output directory.
Returns:: <output_dir>/.pht-tune-cache/splits/split.json.
Return type:: Path

phenotypic.sdk_.tune_cache_splits_dir(output_dir: Path) → Path[source]

Return <output>/.pht-tune-cache/splits/ — the held-out split folder.

Machine state that must survive a fresh-master rewrite and gate resume, so it lives in the hidden tune cache, not under deliverables_dir(). Pure path expression; callers mkdir when they intend to write.

Parameters:: output_dir (Path) – The run output directory.
Returns:: <output_dir>/.pht-tune-cache/splits/.
Return type:: Path

phenotypic.sdk_.tune_cache_study_db_path(output_dir: Path) → Path[source]

Return <output>/.pht-tune-cache/study.db (the Optuna study DB).

The canonical SQLite-WAL storage for the Optuna-backed OptunaStudyStore when the tune extra is installed, relocated into the hidden tune cache. A legacy run wrote it at the output root; use resolve_study_db_path() to read either location.

Parameters:: output_dir (Path) – The run output directory.
Returns:: <output_dir>/.pht-tune-cache/study.db.
Return type:: Path

phenotypic.sdk_.tuning_spec_path(output_dir: Path) → Path[source]

Return the canonical typed tuning spec path under deliverables/.

Parameters:: output_dir (Path)
Return type:: Path

phenotypic.sdk_.validate_analysis_id(analysis_id: str) → str[source]

Validate an analysis ID used as a deliverables artifact stem.

Parameters:

analysis_id (str) – Candidate stable analysis identity.

Returns:

The unchanged validated ID.

Raises:

TypeError – If analysis_id is not a string.
ValueError – If the ID is unsafe or collides with a canonical table.

Return type: