Zarr

Zarr is a format for the storage of chunked, compressed, N-dimensional arrays inspired by HDF5, h5py and bcolz.

The project is fiscally sponsored by NumFOCUS, a US 501(c)(3) public charity, and development is supported by the MRC Centre for Genomics and Global Health and the Chan Zuckerberg Initiative.

These documents describe the Zarr Python implementation. More information about the Zarr format can be found on the main website.

Highlights

• Create N-dimensional arrays with any NumPy dtype.

• Chunk arrays along any dimension.

• Compress and/or filter chunks using any NumCodecs codec.

• Store arrays in memory, on disk, inside a Zip file, on S3, …

• Read an array concurrently from multiple threads or processes.

• Write to an array concurrently from multiple threads or processes.

• Organize arrays into hierarchies via groups.

Contributing

Feedback and bug reports are very welcome, please get in touch via the GitHub issue tracker. See Contributing to Zarr for further information about contributing to Zarr.

Projects using Zarr

If you are using Zarr, we would love to hear about it.

Acknowledgments

The following people have contributed to the development of Zarr by contributing code, documentation, code reviews, comments and/or ideas:

Alistair Miles Altay Sansal Anderson Banihirwe Andrew Fulton Andrew Thomas Anthony Scopatz Attila Bergou BGCMHou Ben Jeffery Ben Williams Boaz Mohar Charles Noyes Chris Barnes David Baddeley Davis Bennett Dimitri Papadopoulos Orfanos Eduardo Gonzalez Elliott Sales de Andrade Eric Prestat Eric Younkin Francesc Alted Greggory Lee Gregory R. Lee Ian Hunt-Isaak James Bourbeau Jan Funke Jerome Kelleher Joe Hamman Joe Jevnik John Kirkham Josh Moore Juan Nunez-Iglesias Justin Swaney Mads R. B. Kristensen Mamy Ratsimbazafy Martin Durant Matthew Rocklin Matthias Bussonnier Mattia Almansi Noah D Brenowitz Oren Watson Pavithra Eswaramoorthy Poruri Sai Rahul Prakhar Goel Raphael Dussin Ray Bell Richard Scott Richard Shaw Ryan Abernathey Ryan Williams Saransh Chopra Sebastian Grill Shikhar Goenka Shivank Chaudhary Stephan Hoyer Stephan Saalfeld Tarik Onalan Tim Crone Tobias Kölling Tom Augspurger Tom White Tommy Tran Trevor Manz Vincent Schut Vyas Ramasubramani Zain Patel @gsakkis hailiangzhang pmav99 sbalmer

Contents

• Installation
• Tutorial
  • Creating an array
  • Reading and writing data
  • Persistent arrays
  • Resizing and appending
  • Compressors
  • Filters
  • Groups
  • Array and group diagnostics
  • User attributes
  • Advanced indexing
  • Storage alternatives
  • Copying/migrating data
  • String arrays
  • Object arrays
  • Chunk optimizations
  • Parallel computing and synchronization
  • Pickle support
  • Datetimes and timedeltas
  • Usage tips
• API reference
  • Array creation (zarr.creation)
  • The Array class (zarr.core)
  • Groups (zarr.hierarchy)
  • Storage (zarr.storage)
  • N5 (zarr.n5)
  • Convenience functions (zarr.convenience)
  • Compressors and filters (zarr.codecs)
  • The Attributes class (zarr.attrs)
  • Synchronization (zarr.sync)
• Specifications
  • Zarr storage specification version 3 (under development)
  • Zarr storage specification version 2
  • Zarr storage specification version 1
• Contributing to Zarr
  • Asking for help
  • Bug reports
  • Enhancement proposals
  • Contributing code and/or documentation
  • Development best practices, policies and procedures
• Release notes
  • 2.13.5
  • 2.13.4
  • 2.13.3
  • 2.13.2
  • 2.13.1
  • 2.13.0
  • 2.12.0
  • 2.11.3
  • 2.11.2
  • 2.11.1
  • 2.11.0
  • 2.10.3
  • 2.10.2
  • 2.10.1
  • 2.10.0
  • 2.9.5
  • 2.9.4
  • 2.9.3
  • 2.9.2
  • 2.9.1
  • 2.9.0
  • 2.8.3
  • 2.8.2
  • 2.8.1
  • 2.8.0
  • 2.7.1
  • 2.7.0
  • 2.6.1
  • 2.6.0
  • 2.5.0
  • 2.4.0
  • 2.3.2
  • 2.3.1
  • 2.3.0
  • 2.2.0
  • 2.1.4
  • 2.1.3
  • 2.1.2
  • 2.1.1
  • 2.1.0
  • 2.0.1
  • 2.0.0
  • 1.1.0
  • 1.0.0
  • 0.4.0
  • 0.3.0
• License
• View homepage

Indices and tables

• Index

• Module Index

• Search Page