clearex.io.experiment

Navigate experiment ingestion and 6D Zarr analysis-store utilities.

Functions

`create_dask_client`(*[, scheduler_address, ...])	Create Dask distributed client (local default, cluster optional).
`default_analysis_store_path`(experiment)	Return canonical OME-Zarr analysis-store path for an experiment.
`experiment_data_search_directories`(experiment, *)	Return ordered directories to inspect for acquisition data.
`find_experiment_data_candidates`(experiment, *)	Find candidate acquisition data files/stores for an experiment.
`has_canonical_data_component`(zarr_path)	Return whether a store contains canonical 6D runtime-cache source data.
`has_complete_canonical_data_store`(zarr_path, *)	Return whether canonical ingestion is complete for a Zarr/N5 store.
`infer_zyx_shape`(experiment, image_info)	Infer `(z, y, x)` shape for 6D analysis store allocation.
`initialize_analysis_store`(experiment, ...[, ...])	Initialize canonical runtime-cache arrays within an OME-Zarr store.
`is_navigate_experiment_file`(path)	Return whether a path is a Navigate experiment descriptor file.
`load_navigate_experiment`(path)	Load and normalize Navigate `experiment.yml` metadata.
`load_navigate_experiment_source_image_info`(*, ...)	Load source metadata for one Navigate experiment input path.
`load_store_spatial_calibration`(zarr_path)	Load store-level spatial calibration from namespaced store metadata.
`materialize_experiment_data_store`(*, ...[, ...])	Materialize experiment source data into canonical Zarr `data` pyramid.
`resolve_data_store_path`(experiment, source_path)	Resolve destination Zarr store path for materialized source data.
`resolve_experiment_data_path`(experiment, *)	Resolve primary acquisition data path from experiment metadata.
`save_store_spatial_calibration`(zarr_path, ...)	Persist store-level spatial calibration into namespaced store metadata.
`summarize_navigate_bdv_n5_image_info`(*, ...)	Summarize a Navigate BDV N5 collection as canonical `ImageInfo`.
`write_zyx_block`(zarr_path, block, *, ...[, ...])	Write one non-overlapping `(z, y, x)` block into 6D analysis store.

Classes

`MaterializedDataStore`(source_path, ...)	Metadata for a source-to-Zarr materialization run.
`NavigateChannel`(name, laser, laser_index, ...)	Selected channel metadata from a Navigate experiment.
`NavigateExperiment`(path, raw, ...[, ...])	Parsed Navigate experiment metadata required by ClearEx.

Exceptions

ExperimentDataResolutionError(*, experiment, ...)

Raised when acquisition data cannot be located for an experiment.

clearex.io.experiment.has_canonical_data_component(zarr_path)

Return whether a store contains canonical 6D runtime-cache source data.

Parameters:: zarr_path (str or pathlib.Path) – Candidate Zarr/N5 store path.
Returns:: True when the store exposes a namespaced runtime-cache data array in canonical (t, p, c, z, y, x) form. If axis metadata is present, it must also normalize to that same canonical order.
Return type:: bool

Notes

This helper is intentionally conservative and returns False for any: unreadable, missing, or malformed runtime-cache data component.

clearex.io.experiment.load_store_spatial_calibration(zarr_path)

Load store-level spatial calibration from namespaced store metadata.

Parameters:: zarr_path (str or pathlib.Path) – Analysis-store path.
Returns:: Parsed store calibration. Missing attrs resolve to identity.
Return type:: SpatialCalibrationConfig
Raises:: ValueError – If stored spatial calibration metadata is malformed.

clearex.io.experiment.save_store_spatial_calibration(zarr_path, calibration)

Persist store-level spatial calibration into namespaced store metadata.

Parameters:

zarr_path (str or pathlib.Path) – Analysis-store path.
calibration (SpatialCalibrationConfig) – Calibration payload to persist.

Returns:

Normalized calibration written to the store.

Return type:

SpatialCalibrationConfig

clearex.io.experiment.has_complete_canonical_data_store(zarr_path, *, expected_chunks_tpczyx=None, expected_pyramid_factors=None)

Return whether canonical ingestion is complete for a Zarr/N5 store.

Parameters:

zarr_path (str or pathlib.Path) – Candidate Zarr/N5 store path.
expected_chunks_tpczyx (tuple[int, int, int, int, int, int], optional) – Expected base-level chunking in canonical order.
expected_pyramid_factors (tuple[tuple[int, ...], ...], optional) – Expected per-axis pyramid factors in canonical order.

Returns:

True when base data is canonical, expected pyramid structure exists, and ingestion progress metadata indicates a completed run.

Return type:

bool

Raises:

None – Unreadable or malformed stores return False.

Notes

Stores created before ingestion progress tracking existed are accepted when structural validation succeeds and no progress record is present.

clearex.io.experiment.summarize_navigate_bdv_n5_image_info(*, experiment, source_path)

Summarize a Navigate BDV N5 collection as canonical ImageInfo.

Parameters:

experiment (NavigateExperiment) –
source_path (Path) –

Return type:

Optional[ImageInfo]

clearex.io.experiment.load_navigate_experiment_source_image_info(*, experiment, source_path, opener=None, prefer_dask=True, chunks=None)

Load source metadata for one Navigate experiment input path.

Parameters:

experiment (NavigateExperiment) – Parsed experiment metadata.
source_path (pathlib.Path) – Resolved acquisition source path.
opener (object, optional) – Existing ImageOpener-compatible instance for non-N5 sources.
prefer_dask (bool, default=True) – Forwarded to ImageOpener.open when non-N5 metadata is read.
chunks (int or tuple[int, ...], optional) – Forwarded to ImageOpener.open when non-N5 metadata is read.

Returns:

Metadata summary for the resolved acquisition source.

Return type:

ImageInfo

Raises:

ValueError – If Navigate file_type: N5 does not resolve to a BDV-style N5 source layout.

class clearex.io.experiment.MaterializedDataStore(source_path, store_path, source_component, source_image_info, data_image_info, chunks_tpczyx)

Bases: object

Metadata for a source-to-Zarr materialization run.

Variables:

source_path (pathlib.Path) – Resolved acquisition source path used for reading.
store_path (pathlib.Path) – Resolved target Zarr store path containing canonical data array.
source_component (str, optional) – Component path selected within the source store, when applicable.
source_image_info (ImageInfo) – Source image metadata used for logging/provenance.
data_image_info (ImageInfo) – Canonical materialized data array metadata.
chunks_tpczyx (tuple[int, int, int, int, int, int]) – Effective write chunks in canonical axis order.

Parameters:

source_path (Path) –
store_path (Path) –
source_component (Optional[str]) –
source_image_info (ImageInfo) –
data_image_info (ImageInfo) –
chunks_tpczyx (tuple[int, int, int, int, int, int]) –

source_path: Path

store_path: Path

source_component: Optional[str]

source_image_info: ImageInfo

data_image_info: ImageInfo

chunks_tpczyx: tuple[int, int, int, int, int, int]

clearex.io.experiment.resolve_data_store_path(experiment, source_path)

Resolve destination Zarr store path for materialized source data.

Parameters:

experiment (NavigateExperiment) – Parsed experiment metadata.
source_path (str or pathlib.Path) – Resolved or candidate source path.

Returns:

Destination OME-Zarr v3 store path. Existing OME-Zarr sources are reused in-place only when they are already canonical ClearEx stores; all other sources are materialized as data_store.ome.zarr next to experiment.yml.

Return type:

pathlib.Path

Raises:

None – This helper does not raise custom exceptions.

clearex.io.experiment.materialize_experiment_data_store(*, experiment, source_path, chunks=(1, 1, 1, 256, 256, 256), pyramid_factors=((1,), (1,), (1,), (1, 2, 4, 8), (1, 2, 4, 8), (1, 2, 4, 8)), client=None, force_rebuild=False, progress_callback=None)

Materialize experiment source data into canonical Zarr data pyramid.

Parameters:

experiment (NavigateExperiment) – Parsed experiment metadata.
source_path (str or pathlib.Path) – Acquisition source path resolved from experiment metadata.
chunks (tuple[int, int, int, int, int, int], default=(1,1,1,256,256,256)) – Target chunking in (t, p, c, z, y, x) order.
pyramid_factors (tuple[tuple[int, ...], ...], optional) – Resolution pyramid factors in (t, p, c, z, y, x) order.
client (dask.distributed.Client, optional) – Active Dask distributed client used to execute graph writes.
force_rebuild (bool, default=False) – Whether to rebuild the canonical store even when a complete canonical store already exists.
progress_callback (callable, optional) – Callback invoked as progress_callback(percent, message) with stage-level progress updates from 0 to 100.

Returns:

Materialization summary including source/store metadata for base level.

Return type:

MaterializedDataStore

Raises:

FileNotFoundError – If source_path does not exist.
ValueError – If source format or dimensionality cannot be normalized.

Notes

The workflow writes canonical runtime-cache base data and then writes downsampled levels under the canonical source pyramid according to pyramid_factors. Existing stores are only reused in place when the input is already a canonical ClearEx OME-Zarr store; generic source .ome.zarr inputs are treated as source formats and materialized into a fresh data_store.ome.zarr beside experiment.yml.

class clearex.io.experiment.NavigateChannel(name, laser, laser_index, exposure_ms, is_selected)

Bases: object

Selected channel metadata from a Navigate experiment.

Variables:

name (str) – Channel key in the experiment state (e.g., "channel_1").
laser (str, optional) – Laser label assigned for the channel.
laser_index (int, optional) – Laser index used by acquisition.
exposure_ms (float, optional) – Camera exposure time in milliseconds.
is_selected (bool) – Whether the channel was selected for acquisition.

Parameters:

name (str) –
laser (Optional[str]) –
laser_index (Optional[int]) –
exposure_ms (Optional[float]) –
is_selected (bool) –

name: str

laser: Optional[str]

laser_index: Optional[int]

exposure_ms: Optional[float]

is_selected: bool

class clearex.io.experiment.NavigateExperiment(path, raw, save_directory, file_type, microscope_name, image_mode, timepoints, number_z_steps, y_pixels, x_pixels, multiposition_count, selected_channels, xy_pixel_size_um=None, z_step_um=None, navigate_oblique_geometry=None)

Bases: object

Parsed Navigate experiment metadata required by ClearEx.

Variables:

path (pathlib.Path) – Absolute path to the source experiment.yml.
raw (dict[str, Any]) – Full parsed experiment mapping.
save_directory (pathlib.Path) – Acquisition output directory.
file_type (str) – Declared output file type from acquisition settings.
microscope_name (str, optional) – Active microscope profile name used during acquisition.
image_mode (str, optional) – Acquisition mode (e.g., "z-stack").
timepoints (int) – Number of timepoints captured.
number_z_steps (int) – Number of z slices per stack.
y_pixels (int) – Image height in pixels.
x_pixels (int) – Image width in pixels.
multiposition_count (int) – Number of multiposition entries recorded.
selected_channels (list[NavigateChannel]) – Channels marked as selected in acquisition state.
xy_pixel_size_um (float, optional) – Estimated sample-space XY pixel size in microns.
z_step_um (float, optional) – Z-step size in microns.
navigate_oblique_geometry (dict[str, Any], optional) – Navigate stage-scan geometry payload for deskew-aware transforms.

Parameters:

path (Path) –
raw (Dict[str, Any]) –
save_directory (Path) –
file_type (str) –
microscope_name (Optional[str]) –
image_mode (Optional[str]) –
timepoints (int) –
number_z_steps (int) –
y_pixels (int) –
x_pixels (int) –
multiposition_count (int) –
selected_channels (list[clearex.io.experiment.NavigateChannel]) –
xy_pixel_size_um (Optional[float]) –
z_step_um (Optional[float]) –
navigate_oblique_geometry (Optional[Dict[str, Any]]) –

path: Path

raw: Dict[str, Any]

save_directory: Path

file_type: str

microscope_name: Optional[str]

image_mode: Optional[str]

timepoints: int

number_z_steps: int

y_pixels: int

x_pixels: int

multiposition_count: int

selected_channels: list[clearex.io.experiment.NavigateChannel]

xy_pixel_size_um: Optional[float] = None

z_step_um: Optional[float] = None

navigate_oblique_geometry: Optional[Dict[str, Any]] = None

property channel_count: int

Return number of selected channels.

Returns:: Count of selected channels, defaulting to 1.
Return type:: int

to_metadata_dict()

Convert parsed experiment into JSON-friendly metadata.

Returns:: Compact metadata mapping suitable for Zarr attrs/provenance.
Return type:: dict[str, Any]

exception clearex.io.experiment.ExperimentDataResolutionError(*, experiment, searched_directories)

Bases: FileNotFoundError

Raised when acquisition data cannot be located for an experiment.

Parameters:

experiment (NavigateExperiment) – Parsed experiment metadata for the failed lookup.
searched_directories (tuple[pathlib.Path, ...]) – Directories inspected for candidate acquisition data.

Return type:

None

clearex.io.experiment.experiment_data_search_directories(experiment, *, search_directory=None)

Return ordered directories to inspect for acquisition data.

Parameters:

experiment (NavigateExperiment) – Parsed experiment metadata.
search_directory (str or pathlib.Path, optional) – Explicit user-selected override directory. This is searched first.

Returns:

Ordered search roots with duplicates removed.

Return type:

list[pathlib.Path]

clearex.io.experiment.is_navigate_experiment_file(path)

Return whether a path is a Navigate experiment descriptor file.

Parameters:: path (str or pathlib.Path) – Path to evaluate.
Returns:: True when basename is experiment.yml or experiment.yaml.
Return type:: bool

clearex.io.experiment.load_navigate_experiment(path)

Load and normalize Navigate experiment.yml metadata.

Parameters:

path (str or pathlib.Path) – Path to experiment.yml.

Returns:

Parsed experiment metadata.

Return type:

NavigateExperiment

Raises:

FileNotFoundError – If path does not exist.
ValueError – If file content is invalid or missing required structure.

clearex.io.experiment.find_experiment_data_candidates(experiment, *, search_directory=None)

Find candidate acquisition data files/stores for an experiment.

Parameters:

experiment (NavigateExperiment) – Parsed experiment metadata.
search_directory (str or pathlib.Path, optional) – Explicit user-selected override directory searched ahead of metadata defaults.

Returns:

Candidate paths sorted by deterministic preference.

Return type:

list[pathlib.Path]

clearex.io.experiment.resolve_experiment_data_path(experiment, *, search_directory=None)

Resolve primary acquisition data path from experiment metadata.

Parameters:

experiment (NavigateExperiment) – Parsed experiment metadata.
search_directory (str or pathlib.Path, optional) – Explicit user-selected override directory searched ahead of metadata defaults.

Returns:

Selected source data path.

Return type:

pathlib.Path

Raises:

FileNotFoundError – If no compatible source data can be found.

clearex.io.experiment.default_analysis_store_path(experiment)

Return canonical OME-Zarr analysis-store path for an experiment.

Parameters:: experiment (NavigateExperiment) – Parsed experiment metadata.
Returns:: Path to canonical analysis store (analysis.ome.zarr).
Return type:: pathlib.Path

clearex.io.experiment.infer_zyx_shape(experiment, image_info)

Infer (z, y, x) shape for 6D analysis store allocation.

Parameters:

experiment (NavigateExperiment) – Parsed experiment metadata.
image_info (ImageInfo, optional) – Metadata from source image reader.

Returns:

Inferred spatial dimensions (z, y, x).

Return type:

tuple[int, int, int]

clearex.io.experiment.initialize_analysis_store(experiment, zarr_path, *, image_info=None, shape_tpczyx=None, overwrite=False, chunks=(1, 1, 1, 256, 256, 256), pyramid_factors=((1,), (1,), (1,), (1, 2, 4, 8), (1, 2, 4, 8), (1, 2, 4, 8)), dtype=None)

Initialize canonical runtime-cache arrays within an OME-Zarr store.

Parameters:

experiment (NavigateExperiment) – Parsed experiment metadata.
zarr_path (str or pathlib.Path) – Output Zarr store path.
image_info (ImageInfo, optional) – Source image metadata used for dtype/shape inference.
shape_tpczyx (tuple[int, int, int, int, int, int], optional) – Explicit canonical shape override in (t, p, c, z, y, x) order.
overwrite (bool, default=False) – Whether to overwrite existing data array when present.
chunks (tuple[int, int, int, int, int, int], default=(1,1,1,256,256,256)) – Target 6D chunking in (t, p, c, z, y, x) order.
pyramid_factors (tuple[tuple[int, ...], ...], optional) – Downsampling factors in (t, p, c, z, y, x) order. Each axis must provide at least one positive factor and start with 1.
dtype (str, optional) – Explicit output dtype. Defaults to source dtype or uint16.

Returns:

Resolved OME-Zarr store path.

Return type:

pathlib.Path

Raises:

ValueError – If chunks or pyramid_factors are not valid for six axes.

clearex.io.experiment.create_dask_client(*, scheduler_address=None, n_workers=None, threads_per_worker=1, processes=False, memory_limit='auto', local_directory=None, dashboard_address=':0', gpu_enabled=False, gpu_device_ids=None)

Create Dask distributed client (local default, cluster optional).

Parameters:

scheduler_address (str, optional) – Existing Dask scheduler address for multi-node operation.
n_workers (int, optional) – Number of local workers when using local mode.
threads_per_worker (int, default=1) – Threads per worker for local mode.
processes (bool, default=False) – Whether local workers should be process-based. I/O-dominant workloads typically perform better with thread-based workers.
memory_limit (str or float, default="auto") – Memory limit per worker for local mode.
local_directory (str or pathlib.Path, optional) – Worker local directory for spills/temp files.
dashboard_address (str, optional, default=":0") – Dashboard bind address for local mode. ":0" requests an available ephemeral port to avoid collisions when multiple local clusters run.
gpu_enabled (bool, default=False) – Whether to launch a GPU-pinned local cluster with one worker process per assigned CUDA device.
gpu_device_ids (sequence of int or str, optional) – Explicit CUDA device identifiers (for example [0, 1, 2, 3]). When omitted and gpu_enabled is true, devices are auto-detected from nvidia-smi.

Returns:

Connected Dask client.

Return type:

dask.distributed.Client

Raises:

ImportError – If dask.distributed is not available.

clearex.io.experiment.write_zyx_block(zarr_path, block, *, t_index, p_index, c_index, compute=True)

Write one non-overlapping (z, y, x) block into 6D analysis store.

Parameters:

zarr_path (str or pathlib.Path) – Analysis Zarr store path.
block (numpy.ndarray or dask.array.Array) – (z, y, x) volume to write.
t_index (int) – Time index.
p_index (int) – Position index.
c_index (int) – Channel index.
compute (bool, default=True) – If block is Dask, whether to execute the write immediately.

Returns:

None for NumPy writes. For Dask writes, returns a delayed object when compute=False.

Return type:

Any

Raises:

ValueError – If block is not 3D.
TypeError – If block type is unsupported.