clearex.io.provenance

Utilities for FAIR-aligned provenance and latest-only outputs in Zarr stores.

Functions

attach_audit_log_manifest_to_latest_run(...)

Attach an audit-log manifest to the latest provenance run.

is_zarr_store_path(path)

Return whether a path appears to reference a Zarr/N5 store.

load_latest_analysis_gui_state(zarr_path)

Load latest dataset-local GUI analysis state.

load_latest_completed_workflow_state(zarr_path)

Return the latest completed workflow payload from provenance.

persist_latest_analysis_gui_state(zarr_path, ...)

Persist latest dataset-local GUI analysis state.

persist_run_provenance(zarr_path, workflow, ...)

Append a provenance run record to a Zarr/N5 store.

register_latest_output_reference(zarr_path, ...)

Register metadata for latest output location of an analysis.

store_latest_analysis_output(zarr_path, ...)

Store only the latest version of a large analysis output array.

summarize_analysis_history(zarr_path, ...[, ...])

Summarize successful-run history for an analysis operation.

verify_provenance_chain(zarr_path)

Verify run-record hash chain stored in a Zarr/N5 store.
clearex.io.provenance.is_zarr_store_path(path)

Return whether a path appears to reference a Zarr/N5 store.

Parameters:

path (str or pathlib.Path) – Path to evaluate.

Returns:

True if the path suffix is .zarr or .n5.

Return type:

bool

clearex.io.provenance.summarize_analysis_history(zarr_path, analysis_name, *, parameters=None)

Summarize successful-run history for an analysis operation.

Parameters:

  • zarr_path (str or pathlib.Path) – Path to a Zarr/N5 store.

  • analysis_name (str) – Analysis operation name.

  • parameters (mapping[str, Any], optional) – Candidate parameter mapping used to detect exact prior matches.

Returns:

History summary with keys: has_successful_run, latest_success_run_id, latest_success_ended_utc, matches_parameters, matching_run_id, matching_ended_utc.

Return type:

dict[str, Any]

Raises:

ValueError – If zarr_path is not a Zarr/N5 path.

clearex.io.provenance.load_latest_completed_workflow_state(zarr_path)

Return the latest completed workflow payload from provenance.

Parameters:

zarr_path (str or pathlib.Path) – Path to a Zarr/N5 store.

Returns:

Mapping with run_id, ended_utc, and workflow payload when a completed run record exists, otherwise None.

Return type:

dict[str, Any], optional

Raises:

ValueError – If zarr_path is not a Zarr/N5 path.

clearex.io.provenance.persist_latest_analysis_gui_state(zarr_path, workflow, *, source='analysis_dialog')

Persist latest dataset-local GUI analysis state.

Parameters:

  • zarr_path (str or pathlib.Path) – Path to a Zarr/N5 store.

  • workflow (mapping[str, Any]) – GUI workflow-state payload to persist.

  • source (str, default="analysis_dialog") – Source identifier used for diagnostics.

Returns:

GUI-state metadata is written in-place to the Zarr store.

Return type:

None

Raises:

ValueError – If zarr_path is not a Zarr/N5 path.

clearex.io.provenance.load_latest_analysis_gui_state(zarr_path)

Load latest dataset-local GUI analysis state.

Parameters:

zarr_path (str or pathlib.Path) – Path to a Zarr/N5 store.

Returns:

Mapping with updated_utc, source, and workflow payload when saved GUI state exists, otherwise None.

Return type:

dict[str, Any], optional

Raises:

ValueError – If zarr_path is not a Zarr/N5 path.

clearex.io.provenance.register_latest_output_reference(zarr_path, analysis_name, component, *, run_id=None, output_hash=None, metadata=None)

Register metadata for latest output location of an analysis.

Parameters:

  • zarr_path (str or pathlib.Path) – Path to Zarr/N5 store.

  • analysis_name (str) – Analysis name.

  • component (str) – Zarr component path that points to latest output.

  • run_id (str, optional) – Provenance run identifier that produced this output.

  • output_hash (str, optional) – Optional hash for output integrity tracking.

  • metadata (mapping, optional) – Additional JSON-serializable metadata.

Returns:

Metadata is written in-place to the Zarr store.

Return type:

None

Raises:

ValueError – If zarr_path is not a Zarr/N5 path.

clearex.io.provenance.store_latest_analysis_output(zarr_path, analysis_name, output_array, *, run_id=None, chunks=None, metadata=None)

Store only the latest version of a large analysis output array.

Parameters:

  • zarr_path (str or pathlib.Path) – Path to Zarr/N5 store.

  • analysis_name (str) – Analysis name used for output namespace.

  • output_array (numpy.ndarray or dask.array.Array) – Array to write as latest output.

  • run_id (str, optional) – Provenance run identifier that produced this output.

  • chunks (int or tuple of int, optional) – Chunk specification. Applied for NumPy writes and optional for Dask writes through rechunking.

  • metadata (mapping, optional) – Additional metadata to attach to output reference record.

Returns:

Zarr component path where latest output was written.

Return type:

str

Raises:

  • ValueError – If zarr_path is not a Zarr/N5 path.

  • TypeError – If output_array is neither NumPy nor Dask array.

Notes

Output path is always results/<analysis>/latest and is overwritten, ensuring only one large output version is retained.

clearex.io.provenance.persist_run_provenance(zarr_path, workflow, image_info, *, steps=None, outputs=None, status='completed', started_at_utc=None, ended_at_utc=None, repo_root=None, run_id=None, audit_log=None)

Append a provenance run record to a Zarr/N5 store.

Parameters:

  • zarr_path (str or pathlib.Path) – Path to Zarr/N5 store.

  • workflow (WorkflowConfig) – Workflow configuration used for the run.

  • image_info (ImageInfo, optional) – Input image metadata loaded for the run.

  • steps (sequence of mapping, optional) – Explicit ordered analysis steps. Default steps are derived from workflow flags when omitted.

  • outputs (mapping, optional) – Output references. Defaults to latest-only output components for selected analyses.

  • status (str, default="completed") – Run terminal status.

  • started_at_utc (datetime, optional) – Run start timestamp in UTC.

  • ended_at_utc (datetime, optional) – Run end timestamp in UTC.

  • repo_root (str or pathlib.Path, optional) – Repository root for Git/environment metadata resolution.

  • run_id (str, optional) – Run identifier to use. A new UUID hex identifier is generated when omitted.

  • audit_log (mapping, optional) – Structured audit-log manifest to embed in the run record.

Returns:

Newly created run identifier.

Return type:

str

Raises:

ValueError – If zarr_path is not a Zarr/N5 path.

clearex.io.provenance.attach_audit_log_manifest_to_latest_run(zarr_path, run_id, audit_log)

Attach an audit-log manifest to the latest provenance run.

Parameters:

  • zarr_path (str or pathlib.Path) – Path to Zarr/N5 store.

  • run_id (str) – Latest provenance run identifier to update.

  • audit_log (mapping) – Structured audit-log manifest.

Returns:

The latest run record and provenance hash are updated in-place.

Return type:

None

Raises:

ValueError – If zarr_path is not a Zarr/N5 path, the run is missing, or the requested run is no longer the latest run.

clearex.io.provenance.verify_provenance_chain(zarr_path)

Verify run-record hash chain stored in a Zarr/N5 store.

Parameters:

zarr_path (str or pathlib.Path) – Path to Zarr/N5 store.

Returns:

(is_valid, issues) where issues is empty when valid.

Return type:

tuple[bool, list[str]]

Raises:

ValueError – If zarr_path is not a Zarr/N5 path.