Skip to content

v3

zarr_metadata.v3

Zarr v3 metadata types, and the front door to the v3 definitions.

This module holds the JSON of any metadata field (a bare name or a ZarrV3NamedConfigJSON envelope) and the node name and path types. Each document type is in the submodule for its node: array, group and consolidated. The extensions are in codec, data_type, chunk_grid and chunk_key_encoding, and the v3 kinds and scopes are in definition.

This module also exports every v3 definition (BYTES_CODEC, FLOAT64_DATA_TYPE, REGULAR_CHUNK_GRID, ...), the scopes CORE_CONTEXT, CORE_AND_EXTENSIONS_CONTEXT and ZARR_PYTHON_CONTEXT, and the scope class ZarrV3Context, so that one import builds or updates a scope. Each is the same object as in its own submodule, where a definition's configuration types and constants are.

zarr_metadata.v3.ZarrV3MetadataFieldJSON module-attribute 

ZarrV3MetadataFieldJSON: TypeAlias = (
    str | ZarrV3NamedConfigJSON
)

The JSON shape of any v3 metadata extension-point entry: either a bare short-hand name string or a {name, configuration, must_understand} envelope.

Used for data_type, chunk_grid, chunk_key_encoding, individual codec entries, and storage_transformers in v3 array metadata, and for the inner codecs / index_codecs lists of the sharding_indexed codec.

zarr_metadata.v3.ZarrV3NamedConfigJSON

Bases: TypedDict

Externally-tagged union member for a metadata field.

The optional configuration mapping holds arbitrary JSON-encodable values. must_understand is implicitly true when absent.

name and configuration are ReadOnly (PEP 705) so that concrete entity types — BloscCodecObject, RegularChunkGridObject, and the rest — are assignable to this type, and therefore to ZarrV3MetadataFieldJSON. Without ReadOnly both items are invariant, so a concrete name: Literal["blosc"] does not satisfy name: str, and a required configuration does not satisfy a NotRequired one. That made the package's own codec types unusable in the very fields they describe (codecs, data_type, chunk_grid, ...), and made TypeIs-based codec classification impossible to declare, since TypeIs requires the narrowed type to be assignable to the input type.

must_understand stays writable: nothing needs to narrow it, and keeping it mutable lets writers set it on an already-constructed field.

The type is closed (PEP 728): the spec's named-configuration envelope has exactly these three members, and closing it is also what makes this type — and every concrete entity type embedding it, e.g. the sharding_indexed configuration's inner codecs list — assignable to Mapping[str, JSONValue] (i.e. usable as a JSONValue).

Source code in src/zarr_metadata/_common.py 
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
class ZarrV3NamedConfigJSON(TypedDict, closed=True):
    """
    Externally-tagged union member for a metadata field.

    The optional `configuration` mapping holds arbitrary JSON-encodable
    values. `must_understand` is implicitly true when absent.

    `name` and `configuration` are `ReadOnly` (PEP 705) so that concrete
    entity types — `BloscCodecObject`, `RegularChunkGridObject`, and the
    rest — are assignable to this type, and therefore to
    `ZarrV3MetadataFieldJSON`. Without `ReadOnly` both items are invariant,
    so a concrete `name: Literal["blosc"]` does not satisfy `name: str`, and
    a required `configuration` does not satisfy a `NotRequired` one. That
    made the package's own codec types unusable in the very fields they
    describe (`codecs`, `data_type`, `chunk_grid`, ...), and made
    `TypeIs`-based codec classification impossible to declare, since `TypeIs`
    requires the narrowed type to be assignable to the input type.

    `must_understand` stays writable: nothing needs to narrow it, and
    keeping it mutable lets writers set it on an already-constructed field.

    The type is `closed` (PEP 728): the spec's named-configuration envelope
    has exactly these three members, and closing it is also what makes this
    type — and every concrete entity type embedding it, e.g. the
    `sharding_indexed` configuration's inner `codecs` list — assignable to
    `Mapping[str, JSONValue]` (i.e. usable as a `JSONValue`).
    """

    name: ReadOnly[str]
    """The extension's name."""
    configuration: NotRequired[ReadOnly[Mapping[str, JSONValue]]]
    """The extension's parameters. May be absent."""
    must_understand: NotRequired[bool]
    """Whether a reader that does not recognize this extension must refuse the document.

    Absent means `true`. The spec does not permit `false` for a data type, chunk
    grid or chunk key encoding
    (https://github.com/zarr-developers/zarr-specs/blob/fc7dd9c9beb5a50b87f9b08b00bf50fc0048482f/docs/v3/core/index.rst#L1571-L1581).
    """

configuration instance-attribute 

configuration: NotRequired[
    ReadOnly[Mapping[str, JSONValue]]
]

The extension's parameters. May be absent.

must_understand instance-attribute 

must_understand: NotRequired[bool]

Whether a reader that does not recognize this extension must refuse the document.

Absent means true. The spec does not permit false for a data type, chunk grid or chunk key encoding (https://github.com/zarr-developers/zarr-specs/blob/fc7dd9c9beb5a50b87f9b08b00bf50fc0048482f/docs/v3/core/index.rst#L1571-L1581).

name instance-attribute 

name: ReadOnly[str]

The extension's name.

zarr_metadata.v3.ZarrV3ExtensionField module-attribute 

ZarrV3ExtensionField: TypeAlias = JSONValue

The JSON value of an unknown top-level v3 metadata field.

An object carrying the literal member must_understand: false may be ignored. Every other JSON shape implicitly requires understanding; recognition itself belongs to the reader rather than this structural type.

zarr_metadata.v3.NodeName module-attribute 

NodeName = NewType('NodeName', str)

The name of a node in a Zarr v3 hierarchy.

The root's name is "". Any other name is not empty, contains no /, is not made only of periods (such as . or ..), does not start with the reserved __, and is not zarr.json. Names are case-sensitive: foo and FOO are two different names (https://github.com/zarr-developers/zarr-specs/blob/fc7dd9c9beb5a50b87f9b08b00bf50fc0048482f/docs/v3/core/index.rst#L818-L837).

zarr_metadata.v3.NodePath module-attribute 

NodePath = NewType('NodePath', str)

The path of a node in a Zarr v3 hierarchy.

The root's path is /. Any other node's path is its parent's path, followed by / (unless the parent is the root), followed by its name. So a path starts with /, does not end with /, and has a node name between each pair of / (https://github.com/zarr-developers/zarr-specs/blob/fc7dd9c9beb5a50b87f9b08b00bf50fc0048482f/docs/v3/core/index.rst#L211-L229).

zarr_metadata.v3.array

Zarr v3 array metadata types.

ZARR_V3_ARRAY_METADATA_STORE_KEY module-attribute 

ZARR_V3_ARRAY_METADATA_STORE_KEY: Final[
    ZarrV3ArrayMetadataStoreKey
] = "zarr.json"

The store key a v3 array's metadata document is persisted under.

v3 uses one key for both node types; the document's node_type field distinguishes an array from a group.

ZarrV3ArrayMetadataStoreKey module-attribute 

ZarrV3ArrayMetadataStoreKey = Literal['zarr.json']

Literal type of the store key holding a v3 array's metadata document.

__all__ module-attribute 

__all__ = [
    "ZARR_V3_ARRAY_METADATA_STORE_KEY",
    "ZarrV3ArrayMetadataJSON",
    "ZarrV3ArrayMetadataStoreKey",
]

ZarrV3ArrayMetadataJSON

Bases: TypedDict

Zarr v3 array metadata document (the zarr.json content for an array).

Extra keys may contain arbitrary JSON values. Each extension point is annotated with the field alias of its kind. For example, data_type is a ZarrV3DataTypeField, and each entry of codecs is a ZarrV3CodecField. The alias describes the JSON of the metadata field, and it tells a scope which kind to read the field as.

See https://zarr-specs.readthedocs.io/en/latest/v3/core/index.html#array-metadata

Source code in src/zarr_metadata/v3/array.py 
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
class ZarrV3ArrayMetadataJSON(TypedDict, extra_items=ZarrV3ExtensionField):
    """
    Zarr v3 array metadata document (the `zarr.json` content for an array).

    Extra keys may contain arbitrary JSON values. Each extension point is
    annotated with the field alias of its kind. For example, `data_type` is
    a `ZarrV3DataTypeField`, and each entry of `codecs` is a
    `ZarrV3CodecField`. The alias describes the JSON of the metadata field,
    and it tells a scope which kind to read the field as.

    See https://zarr-specs.readthedocs.io/en/latest/v3/core/index.html#array-metadata
    """

    zarr_format: Literal[3]
    """The Zarr format version, `3`."""
    node_type: Literal["array"]
    """The kind of node the document describes, `"array"`."""
    data_type: ZarrV3DataTypeField
    """The data type of the array's elements, e.g. `"float64"`."""
    shape: tuple[Annotated[int, Ge(0)], ...]
    """The length of each dimension of the array."""
    chunk_grid: ZarrV3ChunkGridField
    """How the array is divided into chunks, e.g. a `"regular"` grid of one chunk shape."""
    chunk_key_encoding: ZarrV3ChunkKeyEncodingField
    """How a chunk's grid coordinates become its key in the store."""
    fill_value: JSONValue
    """The value of an element in a chunk that was never written.

    Which JSON values it may take depends on `data_type`.
    """
    codecs: tuple[ZarrV3CodecField, ...]
    """The codecs that encode each chunk, in encoding order.

    Zero or more array -> array codecs, then exactly one array -> bytes codec,
    then zero or more bytes -> bytes codecs, so the list is never empty
    (https://github.com/zarr-developers/zarr-specs/blob/fc7dd9c9beb5a50b87f9b08b00bf50fc0048482f/docs/v3/core/index.rst#L974-L983).
    """
    attributes: NotRequired[Mapping[str, JSONValue]]
    """User attributes: arbitrary JSON values by string key."""
    storage_transformers: NotRequired[tuple[ZarrV3StorageTransformerField, ...]]
    """The storage transformers between the array and its store.

    Absent means none, as does an empty list.
    """
    dimension_names: NotRequired[tuple[str | None, ...]]
    """A name for each dimension, `null` for an unnamed one.

    As long as `shape`; absent means every dimension is unnamed.
    """

attributes instance-attribute 

attributes: NotRequired[Mapping[str, JSONValue]]

User attributes: arbitrary JSON values by string key.

chunk_grid instance-attribute 

chunk_grid: ZarrV3ChunkGridField

How the array is divided into chunks, e.g. a "regular" grid of one chunk shape.

chunk_key_encoding instance-attribute 

chunk_key_encoding: ZarrV3ChunkKeyEncodingField

How a chunk's grid coordinates become its key in the store.

codecs instance-attribute 

codecs: tuple[ZarrV3CodecField, ...]

The codecs that encode each chunk, in encoding order.

Zero or more array -> array codecs, then exactly one array -> bytes codec, then zero or more bytes -> bytes codecs, so the list is never empty (https://github.com/zarr-developers/zarr-specs/blob/fc7dd9c9beb5a50b87f9b08b00bf50fc0048482f/docs/v3/core/index.rst#L974-L983).

data_type instance-attribute 

data_type: ZarrV3DataTypeField

The data type of the array's elements, e.g. "float64".

dimension_names instance-attribute 

dimension_names: NotRequired[tuple[str | None, ...]]

A name for each dimension, null for an unnamed one.

As long as shape; absent means every dimension is unnamed.

fill_value instance-attribute 

fill_value: JSONValue

The value of an element in a chunk that was never written.

Which JSON values it may take depends on data_type.

node_type instance-attribute 

node_type: Literal['array']

The kind of node the document describes, "array".

shape instance-attribute 

shape: tuple[Annotated[int, Ge(0)], ...]

The length of each dimension of the array.

storage_transformers instance-attribute 

storage_transformers: NotRequired[
    tuple[ZarrV3StorageTransformerField, ...]
]

The storage transformers between the array and its store.

Absent means none, as does an empty list.

zarr_format instance-attribute 

zarr_format: Literal[3]

The Zarr format version, 3.

zarr_metadata.v3.group

Zarr v3 group metadata types.

See https://zarr-specs.readthedocs.io/en/latest/v3/core/index.html#group-metadata

ZARR_V3_GROUP_METADATA_STORE_KEY module-attribute 

ZARR_V3_GROUP_METADATA_STORE_KEY: Final[
    ZarrV3GroupMetadataStoreKey
] = "zarr.json"

The store key a v3 group's metadata document is persisted under.

v3 uses one key for both node types; the document's node_type field distinguishes a group from an array.

ZarrV3GroupMetadataStoreKey module-attribute 

ZarrV3GroupMetadataStoreKey = Literal['zarr.json']

Literal type of the store key holding a v3 group's metadata document.

__all__ module-attribute 

__all__ = [
    "ZARR_V3_GROUP_METADATA_STORE_KEY",
    "ZarrV3GroupMetadataJSON",
    "ZarrV3GroupMetadataStoreKey",
]

ZarrV3GroupMetadataJSON

Bases: TypedDict

Zarr v3 group metadata document (the zarr.json content for a group).

Extra keys may contain arbitrary JSON values.

See https://zarr-specs.readthedocs.io/en/latest/v3/core/index.html#group-metadata

Source code in src/zarr_metadata/v3/group.py 
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
class ZarrV3GroupMetadataJSON(TypedDict, extra_items=ZarrV3ExtensionField):
    """
    Zarr v3 group metadata document (the `zarr.json` content for a group).

    Extra keys may contain arbitrary JSON values.

    See https://zarr-specs.readthedocs.io/en/latest/v3/core/index.html#group-metadata
    """

    zarr_format: Literal[3]
    """The Zarr format version, `3`."""
    node_type: Literal["group"]
    """The kind of node the document describes, `"group"`."""
    attributes: NotRequired[Mapping[str, JSONValue]]
    """User attributes: arbitrary JSON values by string key."""

attributes instance-attribute 

attributes: NotRequired[Mapping[str, JSONValue]]

User attributes: arbitrary JSON values by string key.

node_type instance-attribute 

node_type: Literal['group']

The kind of node the document describes, "group".

zarr_format instance-attribute 

zarr_format: Literal[3]

The Zarr format version, 3.

zarr_metadata.v3.consolidated

Zarr v3 consolidated metadata types.

There is no Zarr v3 specification for consolidated metadata. This module models the inline-on-group convention used by the reference Python implementation (and zarrs), where consolidated metadata is embedded as an extension field on a group's zarr.json.

This is a known non-core interoperability extension. The core spec fixes only its {kind, must_understand, metadata} envelope, which ZarrV3ConsolidatedMetadataEnvelopeJSON types with any JSON as the entries. CONSOLIDATED_METADATA_EXTRA_FIELD reads a group's member as that envelope, and CORE_CONTEXT holds it. ZarrV3ConsolidatedMetadataJSON types the entries as the reference implementation writes them.

CONSOLIDATED_METADATA_EXTRA_FIELD module-attribute 

CONSOLIDATED_METADATA_EXTRA_FIELD: Final[
    GroupExtraFieldDefinition[Any]
] = GroupExtraFieldDefinition(
    name=ZARR_V3_CONSOLIDATED_METADATA_KEY,
    configuration=ZarrV3ConsolidatedMetadataEnvelopeJSON,
    finite=False,
)

A group's consolidated_metadata member, read as the envelope the core spec fixes. The documents it lists are not read: they may hold the non-finite numbers that node attributes may, and ZarrV3ConsolidatedMetadata reads them.

ZARR_V3_CONSOLIDATED_METADATA_KEY module-attribute 

ZARR_V3_CONSOLIDATED_METADATA_KEY: Final = (
    "consolidated_metadata"
)

The key under which consolidated metadata is embedded in a v3 group document.

Unlike the v2 .zmetadata file, this is not a store key: consolidated metadata is carried as an additional field inside the group's own zarr.json. The core spec names the field and its envelope ("For historical reasons, group metadata documents may contain an additional field named consolidated_metadata"); the entry format, like the v2 counterpart, is a reference-implementation convention. https://github.com/zarr-developers/zarr-specs/blob/fc7dd9c9beb5a50b87f9b08b00bf50fc0048482f/docs/v3/core/index.rst#L802-L816

__all__ module-attribute 

__all__ = [
    "CONSOLIDATED_METADATA_EXTRA_FIELD",
    "ZARR_V3_CONSOLIDATED_METADATA_KEY",
    "ZarrV3ConsolidatedMetadataEnvelopeJSON",
    "ZarrV3ConsolidatedMetadataJSON",
]

ZarrV3ConsolidatedMetadataEnvelopeJSON

Bases: TypedDict

The consolidated_metadata member of a v3 group document, as the core spec fixes it.

The spec allows this one additional field on a group, for historical reasons, and fixes its envelope: kind is "inline", must_understand is false, and metadata is an object. The spec leaves the entries of metadata to the reference implementation, so here they are any JSON, as user data: the attributes of a listed document may hold NaN, Infinity and -Infinity. ZarrV3ConsolidatedMetadataJSON types the entries as the reference implementation writes them: an array or group document at each path.

See https://github.com/zarr-developers/zarr-specs/blob/fc7dd9c9beb5a50b87f9b08b00bf50fc0048482f/docs/v3/core/index.rst#L802-L816

Source code in src/zarr_metadata/v3/consolidated.py 
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
class ZarrV3ConsolidatedMetadataEnvelopeJSON(TypedDict, closed=True):
    """
    The `consolidated_metadata` member of a v3 group document, as the core spec fixes it.

    The spec allows this one additional field on a group, for historical
    reasons, and fixes its envelope: `kind` is `"inline"`,
    `must_understand` is false, and `metadata` is an object. The spec
    leaves the entries of `metadata` to the reference implementation, so
    here they are any JSON, as user data: the attributes of a listed
    document may hold `NaN`, `Infinity` and `-Infinity`.
    `ZarrV3ConsolidatedMetadataJSON` types the
    entries as the reference implementation writes them: an array or group
    document at each path.

    See https://github.com/zarr-developers/zarr-specs/blob/fc7dd9c9beb5a50b87f9b08b00bf50fc0048482f/docs/v3/core/index.rst#L802-L816
    """

    kind: Literal["inline"]
    """How the entries are carried, `"inline"`: in the group's own document."""
    must_understand: Literal[False]
    """Always `false`: a reader that does not recognize consolidated metadata may ignore it."""
    metadata: Mapping[str, UserJSONValue]
    """The entries, by each node's path from the group, as any JSON."""

kind instance-attribute 

kind: Literal['inline']

How the entries are carried, "inline": in the group's own document.

metadata instance-attribute 

metadata: Mapping[str, UserJSONValue]

The entries, by each node's path from the group, as any JSON.

must_understand instance-attribute 

must_understand: Literal[False]

Always false: a reader that does not recognize consolidated metadata may ignore it.

ZarrV3ConsolidatedMetadataJSON

Bases: TypedDict

Inline consolidated metadata embedded in a v3 group.

The metadata map contains only v3 array and group entries. V2 entries are excluded from this interoperability convention by design. The v3 core specification acknowledges consolidated_metadata as a historical additional field and fixes this envelope, but leaves the entries to the reference implementation: https://github.com/zarr-developers/zarr-specs/blob/fc7dd9c9beb5a50b87f9b08b00bf50fc0048482f/docs/v3/core/index.rst#L802-L816

Source code in src/zarr_metadata/v3/consolidated.py 
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
class ZarrV3ConsolidatedMetadataJSON(TypedDict, closed=True):
    """
    Inline consolidated metadata embedded in a v3 group.

    The `metadata` map contains only v3 array and group entries. V2 entries
    are excluded from this interoperability convention by design. The v3 core
    specification acknowledges `consolidated_metadata` as a historical
    additional field and fixes this envelope, but leaves the entries to the
    reference implementation:
      https://github.com/zarr-developers/zarr-specs/blob/fc7dd9c9beb5a50b87f9b08b00bf50fc0048482f/docs/v3/core/index.rst#L802-L816
    """

    kind: Literal["inline"]
    """How the entries are carried, `"inline"`: in the group's own document."""
    must_understand: Literal[False]
    """Always `false`: a reader that does not recognize consolidated metadata may ignore it."""
    metadata: Mapping[str, ZarrV3ArrayMetadataJSON | ZarrV3GroupMetadataJSON]
    """The metadata document of each node below the group, by its path from the group.

    A path joins its segments with `"/"`, e.g. `"child/grandchild"`.
    """

kind instance-attribute 

kind: Literal['inline']

How the entries are carried, "inline": in the group's own document.

metadata instance-attribute 

metadata: Mapping[
    str, ZarrV3ArrayMetadataJSON | ZarrV3GroupMetadataJSON
]

The metadata document of each node below the group, by its path from the group.

A path joins its segments with "/", e.g. "child/grandchild".

must_understand instance-attribute 

must_understand: Literal[False]

Always false: a reader that does not recognize consolidated metadata may ignore it.