• September 10, 2026
• 8 min read

zarr 3.4.0

zarr 3.4.0 is out! This post covers the highlights of the latest version of zarr. We will also introduce some new subpackages we are developing as part of an effort to make zarr more modular. See the full release notes for the per-PR breakdown, or keep reading to catch the digested version.

Stackification

This first item is not a new zarr feature. Rather, it's a Zarr-Python feature: Zarr-Python (the project) is becoming a stack of related Python packages, foremost among them zarr, the Python library. In the last few months Zarr-Python gained the following stand-alone Python packages:

• zarr-metadata, a library narrowly scoped to modelling Zarr metadata documents (e.g., zarr.json), with precise type definitions and validation routines. Designed for anyone who wants type safety for Zarr V2 or V3 metadata without re-implementing the Zarr specs. zarr-metadata is very lightweight -- it does not depend on zarr -- so it should be a very cheap dependency for projects to add, either as a runtime or test dependency. We hope to replace most of the metadata logic in zarr with the contents of zarr-metadata.
• zarr-indexing, a library concerned with data structures and algorithms for lazy, chunked array indexing routines based on Google's trailblazing TensorStore library. We hope to replace the array indexing logic in zarr with routines defined in zarr-indexing, and we also want to use zarr-indexing to support new lazy indexing functionality for arrays in zarr.
• zarr-http-server, a library that serves a zarr Store, Array or Group over HTTP via an ASGI app. This allows any array-like data to be dynamically exposed as Zarr. For example, you can take TIFF stacks, access them with tifffile, and use tifffile's Zarr layer with zarr-http-server to expose the TIFF data as Zarr over HTTP for consumption by clients that don't know how to read TIFF. There are no plans for zarr to depend on this package.

These three packages are just a start. We have in-flight PRs for defining packages for Zarr chunk key encodings (PR 4298), stores (PR 4318), and codecs (PR 4319).

All of these packages are pre-1.0 and changing rapidly; zarr does not yet depend on them. When the relevant APIs stabilize, we plan to gradually replace functionality defined in zarr today for the equivalent functionality in a subpackage. For example, the store PR includes a legacy module that contains a verbatim copy of zarr's current Store API, so that when zarr depends on zarr-storage, we can swap out functionality defined in zarr for identical functionality in zarr-storage, and users won't notice any difference beyond an additional installed package.

The strategy here is to identify the logically separate parts of a Zarr implementation, and build a separate package around each part. This is not an original idea: the Rust zarrs project is composed of many separate packages and this architecture has worked well for that project.

Why are we doing this? Factoring zarr into separate packages may seem like a lot of churn with little upside beyond more maintenance. It's true that there will be new kinds of maintenance, and publishing gets a bit more complicated. But we see the following advantages to organizing Zarr-Python into subpackages:

• Modularity: Installing zarr brings in numcodecs and other dependencies required for the full Zarr format. If a Python user only needs to parse Zarr metadata documents, e.g. because they are using zarrista or tensorstore to do IO, then the full set of zarr dependencies is wasteful. We want to make it as easy as possible for people to use Zarr correctly. That means it should be easy to install a package that defines the syntax and semantics of the Zarr metadata documents, without committing consumers of that package to a full runtime.
• Agility: We believe that separate packages will enable more effective development of the Zarr stack. For example, we want to develop a new version of our storage APIs that formally separate sync and async stores. We could carry out this development inside zarr, but zarr releases would mix storage API developments with ordinary zarr development, and so changes to the store API itself would not be transparent to consumers, e.g. implementers of third-party storage backends. If the storage API is defined entirely in a separate package, then we can version that package exactly when the storage API changes, which makes the design history much more visible to consumers.
• Focus: Functionality in zarr has a tendency to be as good as it needs to be for zarr's needs. This is reasonable, but it can complicate the decision about when to make internal APIs public. For example, the core metadata APIs in zarr are not clearly marked public, which is unfortunate because the metadata is a really important part of Zarr! We hope that devoting an entire package to, e.g., Zarr metadata, makes it easier to define a public API surface that zarr and any other consumer can depend on. We want these subpackages to treat zarr as the main, but not sole, consumer, which means we can implement useful features even if zarr doesn't need them today.
• Packaging: Defining core APIs in separate packages solves a packaging problem for zarr. If zarr defines core APIs like storage and codec interfaces internally, third-party packages must depend on zarr to access those APIs. This in turn means zarr cannot offer those third-party packages as optional dependencies without incurring a circular dependency. Introducing a common dependency for zarr and third-party-zarr-package removes the circular dependency issue entirely.

For the full list of subpackages, see the "Related Projects" page in our docs.

Our Roadmap

With the Zarr V2 -> V3 format transition largely settled, it's worth reflecting on how we want zarr to evolve. The previous section outlined one facet of that evolution -- defining zarr as the top of a stack of interlocking Zarr tools. There are other big changes ahead, foremost among them our goal of unlocking a vast performance improvement by wrapping the zarrs Rust crate via PyO3 bindings defined in zarrista. You can read more about this direction and other future plans in our new roadmap document, which is published as part of the 3.4.0 release.

Rectilinear Chunk Grid Improvements

The experimental rectilinear (irregular) chunk grid support (opt in with zarr.config.set({"array.rectilinear_chunks": True})) got several API improvements and bug fixes, mostly in #4218 and #4272:

• Chunk grid flavor (regular vs rectilinear) now consistently follows input syntax: a nested per-dimension list always requests a rectilinear grid, even when the edges happen to be uniform. This fixes a regression in 3.3.0, where uniform-plus-short-tail lists requested a regular grid, which changed resize semantics.
• zarr.from_array(..., chunks="keep") reproduces a rectilinear source grid exactly (in O(ndim)); sharding under a rectilinear shard grid is preserved instead of dropped.
• Array.chunks is defined for any sharded array; Array.info and Array.nchunks_initialized work for rectilinear shard grids instead of raising (info shows the shard shape as <variable>).
• Zero-length axes: chunks=-1 and chunks=False resolve to chunk size 1 (#4307, #4328) instead of ZeroDivisionError / invalid metadata.

Useful Codec Errors

zarr does not support every possible codec out of the box. Instead, we allow external packages to register their codec implementations with zarr at runtime, which broadens the palette of codecs we can support. When zarr reads an array with a codec it doesn't recognize, there's a good chance that the fix is to install the package that ships that codec implementation. But how should a user know which package they need?

Previously, zarr handled an unknown codec with an opaque error, conveying only that zarr doesn't recognize the codec. Now, thanks to PRs #4277 and #4351, we maintain a table that relates the names of well-known codecs zarr doesn't ship with to packages that provide implementations. We use this table to provide a useful error message when an unknown codec is encountered. For example, opening an array whose metadata names the imagecodecs_jpeg codec, without any package that provides it installed:

import json

import zarr
from zarr.buffer.cpu import buffer_prototype

metadata = {
    "zarr_format": 3,
    "node_type": "array",
    "shape": [4, 4],
    "data_type": "uint8",
    "chunk_grid": {"name": "regular", "configuration": {"chunk_shape": [4, 4]}},
    "chunk_key_encoding": {"name": "default"},
    "fill_value": 0,
    "codecs": [{"name": "bytes"}, {"name": "imagecodecs_jpeg"}],
    "attributes": {},
}
store = {"zarr.json": buffer_prototype.buffer.from_bytes(json.dumps(metadata).encode())}

try:
    zarr.open_array(store, mode="r")
except zarr.errors.UnknownCodecError as e:
    print(e)

An implementation for codec 'imagecodecs_jpeg' is not available. Register one explicitly using the codec registry (see https://zarr.readthedocs.io/en/stable/user-guide/extending/#custom-codecs), or install a Python package that registers a codec implementation with zarr. Known packages supporting this codec: imagecodecs-zarr, virtual-tiff.

The table lives in zarr.registry; if you maintain a package that provides a codec, please add it there.

New Dependency: msgspec

Zarr metadata is composed of JSON documents that must comply with various specification documents, e.g. the Zarr V2 and V3 specs. zarr should model these JSON documents in a way that's correct, so that we generate interoperable data, and useful for developers, so they can also generate interoperable data. To do that we need to define two things: precise types, and routines for coercing or narrowing input values with unknown shapes into our precise types.

Writing types for JSON data structures has gotten a lot easier in Python with the development of TypedDict utility types, but writing a validation function that checks an unknown input against an arbitrary TypedDict is arbitrary work. This isn't a hard programming task; rather, it's one that's very easy to mess up, e.g. by omitting an important check, or failing to model a field as optional.

Runtime validation libraries like pydantic, beartype, and msgspec use type annotations to generate runtime type checking routines. Using one of these libraries could spare us a lot of hand-written type checking code. We have long discussed adding a runtime type checker to zarr, and as of 3.4.0 we are now using msgspec in this capacity (#4063, resolving #3285). We chose msgspec because of its performance and minimal dependency count, and we expect it to deliver big value whenever we introduce new codecs, chunk grids, or data types to zarr, as we can just define the type for the metadata and get a validation routine for free.

Tell us what you think

As always, we would love any and all feedback about the work highlighted in this release! We would especially appreciate feedback from Python users who work with Zarr data and don't use the zarr package, since we are hoping to target their needs with our new family of subpackages.

Contributors

Thanks to everyone who contributed to this release (* marks a first-time contributor to Zarr-Python):

• @arcusbuilds *
• @cucuwang *
• @d-v-b
• @dylanpulver *
• @glaziermag *
• @jhamman
• @JOhnsonKC201 *
• @paraseba
• @selmanozleyen
• @sheikhayaan *
• @Tomatokeftes *
• @vup903 *

Our previous blog post omitted the contributor list, so we've updated that post to include one.

AI Usage Disclaimer

The outline for this post was generated by Claude Fable after prompting with the set of unreleased changes since 3.3.0. That outline was then heavily expanded by @d-v-b, introducing a large number of typographical errors. Those errors were removed, code snippets added, and hyperlinks resolved, by Claude.