Zarr¶

Array creation (zarr.creation)¶

zarr.creation.create(shape, chunks=None, dtype=None, compressor='default', fill_value=0, order='C', store=None, synchronizer=None, overwrite=False, path=None, chunk_store=None, filters=None, cache_metadata=True, **kwargs)¶

Create an array.

Parameters:

shape : int or tuple of ints Array shape. chunks : int or tuple of ints, optional Chunk shape. If not provided, will be guessed from shape and dtype. dtype : string or dtype, optional NumPy dtype. compressor : Codec, optional Primary compressor. fill_value : object Default value to use for uninitialized portions of the array. order : {‘C’, ‘F’}, optional Memory layout to be used within each chunk. store : MutableMapping or string Store or path to directory in file system. synchronizer : object, optional Array synchronizer. overwrite : bool, optional If True, delete all pre-existing data in store at path before creating the array. path : string, optional Path under which array is stored. chunk_store : MutableMapping, optional Separate storage for chunks. If not provided, store will be used for storage of both chunks and metadata. filters : sequence of Codecs, optional Sequence of filters to use to encode chunk data prior to compression. cache_metadata : bool, optional If True, array configuration metadata will be cached for the lifetime of the object. If False, array metadata will be reloaded prior to all data access and modification operations (may incur overhead depending on storage and data access pattern).

Returns:

z : zarr.core.Array

Examples

Create an array with default settings:

>>> import zarr
>>> z = zarr.create((10000, 10000), chunks=(1000, 1000))
>>> z
Array((10000, 10000), float64, chunks=(1000, 1000), order=C)
  nbytes: 762.9M; nbytes_stored: 323; ratio: 2476780.2; initialized: 0/100
  compressor: Blosc(cname='lz4', clevel=5, shuffle=1)
  store: dict

zarr.creation.empty(shape, **kwargs)¶

Create an empty array.

For parameter definitions see zarr.creation.create().

Notes

The contents of an empty Zarr array are not defined. On attempting to retrieve data from an empty Zarr array, any values may be returned, and these are not guaranteed to be stable from one access to the next.

zarr.creation.zeros(shape, **kwargs)¶

Create an array, with zero being used as the default value for uninitialized portions of the array.

For parameter definitions see zarr.creation.create().

Examples

>>> import zarr
>>> z = zarr.zeros((10000, 10000), chunks=(1000, 1000))
>>> z
Array((10000, 10000), float64, chunks=(1000, 1000), order=C)
  nbytes: 762.9M; nbytes_stored: 323; ratio: 2476780.2; initialized: 0/100
  compressor: Blosc(cname='lz4', clevel=5, shuffle=1)
  store: dict
>>> z[:2, :2]
array([[ 0.,  0.],
       [ 0.,  0.]])

zarr.creation.ones(shape, **kwargs)¶

Create an array, with one being used as the default value for uninitialized portions of the array.

For parameter definitions see zarr.creation.create().

Examples

>>> import zarr
>>> z = zarr.ones((10000, 10000), chunks=(1000, 1000))
>>> z
Array((10000, 10000), float64, chunks=(1000, 1000), order=C)
  nbytes: 762.9M; nbytes_stored: 323; ratio: 2476780.2; initialized: 0/100
  compressor: Blosc(cname='lz4', clevel=5, shuffle=1)
  store: dict
>>> z[:2, :2]
array([[ 1.,  1.],
       [ 1.,  1.]])

zarr.creation.full(shape, fill_value, **kwargs)¶

Create an array, with fill_value being used as the default value for uninitialized portions of the array.

For parameter definitions see zarr.creation.create().

Examples

>>> import zarr
>>> z = zarr.full((10000, 10000), chunks=(1000, 1000), fill_value=42)
>>> z
Array((10000, 10000), float64, chunks=(1000, 1000), order=C)
  nbytes: 762.9M; nbytes_stored: 324; ratio: 2469135.8; initialized: 0/100
  compressor: Blosc(cname='lz4', clevel=5, shuffle=1)
  store: dict
>>> z[:2, :2]
array([[ 42.,  42.],
       [ 42.,  42.]])

zarr.creation.array(data, **kwargs)¶

Create an array filled with data.

The data argument should be a NumPy array or array-like object. For other parameter definitions see zarr.creation.create().

Examples

>>> import numpy as np
>>> import zarr
>>> a = np.arange(100000000).reshape(10000, 10000)
>>> z = zarr.array(a, chunks=(1000, 1000))
>>> z
Array((10000, 10000), int64, chunks=(1000, 1000), order=C)
  nbytes: 762.9M; nbytes_stored: 15.2M; ratio: 50.2; initialized: 100/100
  compressor: Blosc(cname='lz4', clevel=5, shuffle=1)
  store: dict

zarr.creation.open_array(store=None, mode='a', shape=None, chunks=None, dtype=None, compressor='default', fill_value=0, order='C', synchronizer=None, filters=None, cache_metadata=True, path=None, **kwargs)¶

Open array using mode-like semantics.

Parameters:

store : MutableMapping or string Store or path to directory in file system. mode : {‘r’, ‘r+’, ‘a’, ‘w’, ‘w-‘} Persistence mode: ‘r’ means read only (must exist); ‘r+’ means read/write (must exist); ‘a’ means read/write (create if doesn’t exist); ‘w’ means create (overwrite if exists); ‘w-‘ means create (fail if exists). shape : int or tuple of ints Array shape. chunks : int or tuple of ints, optional Chunk shape. If not provided, will be guessed from shape and dtype. dtype : string or dtype, optional NumPy dtype. compressor : Codec, optional Primary compressor. fill_value : object Default value to use for uninitialized portions of the array. order : {‘C’, ‘F’}, optional Memory layout to be used within each chunk. synchronizer : object, optional Array synchronizer. filters : sequence, optional Sequence of filters to use to encode chunk data prior to compression. cache_metadata : bool, optional If True, array configuration metadata will be cached for the lifetime of the object. If False, array metadata will be reloaded prior to all data access and modification operations (may incur overhead depending on storage and data access pattern). path : string, optional Array path.

Returns:

z : zarr.core.Array

Notes

There is no need to close an array. Data are automatically flushed to the file system.

Examples

>>> import numpy as np
>>> import zarr
>>> z1 = zarr.open_array('example.zarr', mode='w', shape=(10000, 10000),
...                      chunks=(1000, 1000), fill_value=0)
>>> z1[:] = np.arange(100000000).reshape(10000, 10000)
>>> z1
Array((10000, 10000), float64, chunks=(1000, 1000), order=C)
  nbytes: 762.9M; nbytes_stored: 23.0M; ratio: 33.2; initialized: 100/100
  compressor: Blosc(cname='lz4', clevel=5, shuffle=1)
  store: DirectoryStore
>>> z2 = zarr.open_array('example.zarr', mode='r')
>>> z2
Array((10000, 10000), float64, chunks=(1000, 1000), order=C)
  nbytes: 762.9M; nbytes_stored: 23.0M; ratio: 33.2; initialized: 100/100
  compressor: Blosc(cname='lz4', clevel=5, shuffle=1)
  store: DirectoryStore
>>> np.all(z1[:] == z2[:])
True

zarr.creation.empty_like(a, **kwargs)¶

Create an empty array like a.

zarr.creation.zeros_like(a, **kwargs)¶

Create an array of zeros like a.

zarr.creation.ones_like(a, **kwargs)¶

Create an array of ones like a.

zarr.creation.full_like(a, **kwargs)¶

Create a filled array like a.

zarr.creation.open_like(a, path, **kwargs)¶

Open a persistent array like a.

The Array class (zarr.core)¶

class zarr.core.Array(store, path=None, read_only=False, chunk_store=None, synchronizer=None, cache_metadata=True)¶

Instantiate an array from an initialized store.

Parameters:

store : MutableMapping Array store, already initialized. path : string, optional Storage path. read_only : bool, optional True if array should be protected against modification. chunk_store : MutableMapping, optional Separate storage for chunks. If not provided, store will be used for storage of both chunks and metadata. synchronizer : object, optional Array synchronizer. cache_metadata : bool, optional If True, array configuration metadata will be cached for the lifetime of the object. If False, array metadata will be reloaded prior to all data access and modification operations (may incur overhead depending on storage and data access pattern).

Attributes

store	A MutableMapping providing the underlying storage for the array.
path	Storage path.
name	Array name following h5py convention.
read_only	A boolean, True if modification operations are not permitted.
chunk_store	A MutableMapping providing the underlying storage for array chunks.
shape	A tuple of integers describing the length of each dimension of the array.
chunks	A tuple of integers describing the length of each dimension of a chunk of the array.
dtype	The NumPy data type.
fill_value	A value used for uninitialized portions of the array.
order	A string indicating the order in which bytes are arranged within chunks of the array.
synchronizer	Object used to synchronize write access to the array.
filters	One or more codecs used to transform data prior to compression.
attrs	A MutableMapping containing user-defined attributes.
size	The total number of elements in the array.
itemsize	The size in bytes of each item in the array.
nbytes	The total number of bytes that would be required to store the array without compression.
nbytes_stored	The total number of stored bytes of data for the array.
cdata_shape	A tuple of integers describing the number of chunks along each dimension of the array.
nchunks	Total number of chunks.
nchunks_initialized	The number of chunks that have been initialized with some data.
is_view	A boolean, True if this array is a view on another array.

compression
compression_opts

Methods

__getitem__(item)	Retrieve data for some portion of the array.
__setitem__(item, value)	Modify data for some portion of the array.
resize(*args)	Change the shape of the array by growing or shrinking one or more dimensions.
append(data[, axis])	Append data to axis.
view([shape, chunks, dtype, fill_value, ...])	Return an array sharing the same data.

__getitem__(item)¶

Retrieve data for some portion of the array. Most NumPy-style slicing operations are supported.

Returns:	out : ndarray A NumPy array containing the data for the requested region.

Examples

Setup a 1-dimensional array:

>>> import zarr
>>> import numpy as np
>>> z = zarr.array(np.arange(100000000), chunks=1000000, dtype='i4')
>>> z
Array((100000000,), int32, chunks=(1000000,), order=C)
  nbytes: 381.5M; nbytes_stored: 6.4M; ratio: 59.9; initialized: 100/100
  compressor: Blosc(cname='lz4', clevel=5, shuffle=1)
  store: dict

Take some slices:

>>> z[5]
5
>>> z[:5]
array([0, 1, 2, 3, 4], dtype=int32)
>>> z[-5:]
array([99999995, 99999996, 99999997, 99999998, 99999999], dtype=int32)
>>> z[5:10]
array([5, 6, 7, 8, 9], dtype=int32)
>>> z[:]
array([       0,        1,        2, ..., 99999997, 99999998, 99999999], dtype=int32)

Setup a 2-dimensional array:

>>> import zarr
>>> import numpy as np
>>> z = zarr.array(np.arange(100000000).reshape(10000, 10000),
...                chunks=(1000, 1000), dtype='i4')
>>> z
Array((10000, 10000), int32, chunks=(1000, 1000), order=C)
  nbytes: 381.5M; nbytes_stored: 9.2M; ratio: 41.6; initialized: 100/100
  compressor: Blosc(cname='lz4', clevel=5, shuffle=1)
  store: dict

Take some slices:

>>> z[2, 2]
20002
>>> z[:2, :2]
array([[    0,     1],
       [10000, 10001]], dtype=int32)
>>> z[:2]
array([[    0,     1,     2, ...,  9997,  9998,  9999],
       [10000, 10001, 10002, ..., 19997, 19998, 19999]], dtype=int32)
>>> z[:, :2]
array([[       0,        1],
       [   10000,    10001],
       [   20000,    20001],
       ...,
       [99970000, 99970001],
       [99980000, 99980001],
       [99990000, 99990001]], dtype=int32)
>>> z[:]
array([[       0,        1,        2, ...,     9997,     9998,     9999],
       [   10000,    10001,    10002, ...,    19997,    19998,    19999],
       [   20000,    20001,    20002, ...,    29997,    29998,    29999],
       ...,
       [99970000, 99970001, 99970002, ..., 99979997, 99979998, 99979999],
       [99980000, 99980001, 99980002, ..., 99989997, 99989998, 99989999],
       [99990000, 99990001, 99990002, ..., 99999997, 99999998, 99999999]], dtype=int32)

__setitem__(item, value)¶

Modify data for some portion of the array.

Examples

Setup a 1-dimensional array:

>>> import zarr
>>> z = zarr.zeros(100000000, chunks=1000000, dtype='i4')
>>> z
Array((100000000,), int32, chunks=(1000000,), order=C)
  nbytes: 381.5M; nbytes_stored: 301; ratio: 1328903.7; initialized: 0/100
  compressor: Blosc(cname='lz4', clevel=5, shuffle=1)
  store: dict

Set all array elements to the same scalar value:

>>> z[:] = 42
>>> z[:]
array([42, 42, 42, ..., 42, 42, 42], dtype=int32)

Set a portion of the array:

>>> z[:100] = np.arange(100)
>>> z[-100:] = np.arange(100)[::-1]
>>> z[:]
array([0, 1, 2, ..., 2, 1, 0], dtype=int32)

Setup a 2-dimensional array:

>>> z = zarr.zeros((10000, 10000), chunks=(1000, 1000), dtype='i4')
>>> z
Array((10000, 10000), int32, chunks=(1000, 1000), order=C)
  nbytes: 381.5M; nbytes_stored: 323; ratio: 1238390.1; initialized: 0/100
  compressor: Blosc(cname='lz4', clevel=5, shuffle=1)
  store: dict

Set all array elements to the same scalar value:

>>> z[:] = 42
>>> z[:]
array([[42, 42, 42, ..., 42, 42, 42],
       [42, 42, 42, ..., 42, 42, 42],
       [42, 42, 42, ..., 42, 42, 42],
       ...,
       [42, 42, 42, ..., 42, 42, 42],
       [42, 42, 42, ..., 42, 42, 42],
       [42, 42, 42, ..., 42, 42, 42]], dtype=int32)

Set a portion of the array:

>>> z[0, :] = np.arange(z.shape[1])
>>> z[:, 0] = np.arange(z.shape[0])
>>> z[:]
array([[   0,    1,    2, ..., 9997, 9998, 9999],
       [   1,   42,   42, ...,   42,   42,   42],
       [   2,   42,   42, ...,   42,   42,   42],
       ...,
       [9997,   42,   42, ...,   42,   42,   42],
       [9998,   42,   42, ...,   42,   42,   42],
       [9999,   42,   42, ...,   42,   42,   42]], dtype=int32)

resize(*args)¶

Change the shape of the array by growing or shrinking one or more dimensions.

Notes

When resizing an array, the data are not rearranged in any way.

If one or more dimensions are shrunk, any chunks falling outside the new array shape will be deleted from the underlying store.

Examples

>>> import zarr
>>> z = zarr.zeros(shape=(10000, 10000), chunks=(1000, 1000))
>>> z
Array((10000, 10000), float64, chunks=(1000, 1000), order=C)
  nbytes: 762.9M; nbytes_stored: 323; ratio: 2476780.2; initialized: 0/100
  compressor: Blosc(cname='lz4', clevel=5, shuffle=1)
  store: dict
>>> z.resize(20000, 10000)
>>> z
Array((20000, 10000), float64, chunks=(1000, 1000), order=C)
  nbytes: 1.5G; nbytes_stored: 323; ratio: 4953560.4; initialized: 0/200
  compressor: Blosc(cname='lz4', clevel=5, shuffle=1)
  store: dict
>>> z.resize(30000, 1000)
>>> z
Array((30000, 1000), float64, chunks=(1000, 1000), order=C)
  nbytes: 228.9M; nbytes_stored: 322; ratio: 745341.6; initialized: 0/30
  compressor: Blosc(cname='lz4', clevel=5, shuffle=1)
  store: dict

append(data, axis=0)¶

Append data to axis.

Parameters:	data : array_like Data to be appended. axis : int Axis along which to append.
Returns:	new_shape : tuple

Notes

The size of all dimensions other than axis must match between this array and data.

Examples

>>> import numpy as np
>>> import zarr
>>> a = np.arange(10000000, dtype='i4').reshape(10000, 1000)
>>> z = zarr.array(a, chunks=(1000, 100))
>>> z
Array((10000, 1000), int32, chunks=(1000, 100), order=C)
  nbytes: 38.1M; nbytes_stored: 1.9M; ratio: 20.3; initialized: 100/100
  compressor: Blosc(cname='lz4', clevel=5, shuffle=1)
  store: dict
>>> z.append(a)
(20000, 1000)
>>> z
Array((20000, 1000), int32, chunks=(1000, 100), order=C)
  nbytes: 76.3M; nbytes_stored: 3.8M; ratio: 20.3; initialized: 200/200
  compressor: Blosc(cname='lz4', clevel=5, shuffle=1)
  store: dict
>>> z.append(np.vstack([a, a]), axis=1)
(20000, 2000)
>>> z
Array((20000, 2000), int32, chunks=(1000, 100), order=C)
  nbytes: 152.6M; nbytes_stored: 7.5M; ratio: 20.3; initialized: 400/400
  compressor: Blosc(cname='lz4', clevel=5, shuffle=1)
  store: dict

view(shape=None, chunks=None, dtype=None, fill_value=None, filters=None, read_only=None, synchronizer=None)¶

Return an array sharing the same data.

Parameters:

shape : int or tuple of ints Array shape. chunks : int or tuple of ints, optional Chunk shape. dtype : string or dtype, optional NumPy dtype. fill_value : object Default value to use for uninitialized portions of the array. filters : sequence, optional Sequence of filters to use to encode chunk data prior to compression. read_only : bool, optional True if array should be protected against modification. synchronizer : object, optional Array synchronizer.

Notes

WARNING: This is an experimental feature and should be used with care. There are plenty of ways to generate errors and/or cause data corruption.

Examples

Bypass filters:

>>> import zarr
>>> import numpy as np
>>> np.random.seed(42)
>>> labels = [b'female', b'male']
>>> data = np.random.choice(labels, size=10000)
>>> filters = [zarr.Categorize(labels=labels,
...                                  dtype=data.dtype,
...                                  astype='u1')]
>>> a = zarr.array(data, chunks=1000, filters=filters)
>>> a[:]
array([b'female', b'male', b'female', ..., b'male', b'male', b'female'],
      dtype='|S6')
>>> v = a.view(dtype='u1', filters=[])
>>> v.is_view
True
>>> v[:]
array([1, 2, 1, ..., 2, 2, 1], dtype=uint8)

Views can be used to modify data:

>>> x = v[:]
>>> x.sort()
>>> v[:] = x
>>> v[:]
array([1, 1, 1, ..., 2, 2, 2], dtype=uint8)
>>> a[:]
array([b'female', b'female', b'female', ..., b'male', b'male', b'male'],
      dtype='|S6')

View as a different dtype with the same itemsize:

>>> data = np.random.randint(0, 2, size=10000, dtype='u1')
>>> a = zarr.array(data, chunks=1000)
>>> a[:]
array([0, 0, 1, ..., 1, 0, 0], dtype=uint8)
>>> v = a.view(dtype=bool)
>>> v[:]
array([False, False,  True, ...,  True, False, False], dtype=bool)
>>> np.all(a[:].view(dtype=bool) == v[:])
True

An array can be viewed with a dtype with a different itemsize, however some care is needed to adjust the shape and chunk shape so that chunk data is interpreted correctly:

>>> data = np.arange(10000, dtype='u2')
>>> a = zarr.array(data, chunks=1000)
>>> a[:10]
array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=uint16)
>>> v = a.view(dtype='u1', shape=20000, chunks=2000)
>>> v[:10]
array([0, 0, 1, 0, 2, 0, 3, 0, 4, 0], dtype=uint8)
>>> np.all(a[:].view('u1') == v[:])
True

Change fill value for uninitialized chunks:

>>> a = zarr.full(10000, chunks=1000, fill_value=-1, dtype='i1')
>>> a[:]
array([-1, -1, -1, ..., -1, -1, -1], dtype=int8)
>>> v = a.view(fill_value=42)
>>> v[:]
array([42, 42, 42, ..., 42, 42, 42], dtype=int8)

Note that resizing or appending to views is not permitted:

>>> a = zarr.empty(10000)
>>> v = a.view()
>>> try:
...     v.resize(20000)
... except PermissionError as e:
...     print(e)
not permitted for views

Groups (zarr.hierarchy)¶

zarr.hierarchy.group(store=None, overwrite=False, chunk_store=None, synchronizer=None, path=None)¶

Create a group.

Parameters:

store : MutableMapping or string Store or path to directory in file system. overwrite : bool, optional If True, delete any pre-existing data in store at path before creating the group. chunk_store : MutableMapping, optional Separate storage for chunks. If not provided, store will be used for storage of both chunks and metadata. synchronizer : object, optional Array synchronizer. path : string, optional Group path.

Returns:

g : zarr.hierarchy.Group

Examples

Create a group in memory:

>>> import zarr
>>> g = zarr.group()
>>> g
Group(/, 0)
  store: DictStore

Create a group with a different store:

>>> store = zarr.DirectoryStore('example')
>>> g = zarr.group(store=store, overwrite=True)
>>> g
Group(/, 0)
  store: DirectoryStore

zarr.hierarchy.open_group(store=None, mode='a', synchronizer=None, path=None)¶

Open a group using mode-like semantics.

Parameters:

store : MutableMapping or string Store or path to directory in file system. mode : {‘r’, ‘r+’, ‘a’, ‘w’, ‘w-‘} Persistence mode: ‘r’ means read only (must exist); ‘r+’ means read/write (must exist); ‘a’ means read/write (create if doesn’t exist); ‘w’ means create (overwrite if exists); ‘w-‘ means create (fail if exists). synchronizer : object, optional Array synchronizer. path : string, optional Group path.

Returns:

g : zarr.hierarchy.Group

Examples

>>> import zarr
>>> root = zarr.open_group('example', mode='w')
>>> foo = root.create_group('foo')
>>> bar = root.create_group('bar')
>>> root
Group(/, 2)
  groups: 2; bar, foo
  store: DirectoryStore
>>> root2 = zarr.open_group('example', mode='a')
>>> root2
Group(/, 2)
  groups: 2; bar, foo
  store: DirectoryStore
>>> root == root2
True

class zarr.hierarchy.Group(store, path=None, read_only=False, chunk_store=None, synchronizer=None)¶

Instantiate a group from an initialized store.

Parameters:	store : MutableMapping Group store, already initialized. path : string, optional Group path. read_only : bool, optional True if group should be protected against modification. chunk_store : MutableMapping, optional Separate storage for chunks. If not provided, store will be used for storage of both chunks and metadata. synchronizer : object, optional Array synchronizer.

Attributes

store	A MutableMapping providing the underlying storage for the group.
path	Storage path.
name	Group name following h5py convention.
read_only	A boolean, True if modification operations are not permitted.
chunk_store	A MutableMapping providing the underlying storage for array chunks.
synchronizer	Object used to synchronize write access to groups and arrays.
attrs	A MutableMapping containing user-defined attributes.

Methods

__len__()	Number of members.
__iter__()	Return an iterator over group member names.
__contains__(item)	Test for group membership.
__getitem__(item)	Obtain a group member.
group_keys()	Return an iterator over member names for groups only.
groups()	Return an iterator over (name, value) pairs for groups only.
array_keys()	Return an iterator over member names for arrays only.
arrays()	Return an iterator over (name, value) pairs for arrays only.
create_group(name[, overwrite])	Create a sub-group.
require_group(name[, overwrite])	Obtain a sub-group, creating one if it doesn’t exist.
create_groups(*names, **kwargs)	Convenience method to create multiple groups in a single call.
require_groups(*names)	Convenience method to require multiple groups in a single call.
create_dataset(name, **kwargs)	Create an array.
require_dataset(name, shape[, dtype, exact])	Obtain an array, creating if it doesn’t exist.
create(name, **kwargs)	Create an array.
empty(name, **kwargs)	Create an array.
zeros(name, **kwargs)	Create an array.
ones(name, **kwargs)	Create an array.
full(name, fill_value, **kwargs)	Create an array.
array(name, data, **kwargs)	Create an array.
empty_like(name, data, **kwargs)	Create an array.
zeros_like(name, data, **kwargs)	Create an array.
ones_like(name, data, **kwargs)	Create an array.
full_like(name, data, **kwargs)	Create an array.

__len__()¶

Number of members.

__iter__()¶

Return an iterator over group member names.

Examples

>>> import zarr
>>> g1 = zarr.group()
>>> g2 = g1.create_group('foo')
>>> g3 = g1.create_group('bar')
>>> d1 = g1.create_dataset('baz', shape=100, chunks=10)
>>> d2 = g1.create_dataset('quux', shape=200, chunks=20)
>>> for name in g1:
...     print(name)
bar
baz
foo
quux

__contains__(item)¶

Test for group membership.

Examples

>>> import zarr
>>> g1 = zarr.group()
>>> g2 = g1.create_group('foo')
>>> d1 = g1.create_dataset('bar', shape=100, chunks=10)
>>> 'foo' in g1
True
>>> 'bar' in g1
True
>>> 'baz' in g1
False

__getitem__(item)¶

Obtain a group member.

Parameters:	item : string Member name or path.

Examples

>>> import zarr
>>> g1 = zarr.group()
>>> d1 = g1.create_dataset('foo/bar/baz', shape=100, chunks=10)
>>> g1['foo']
Group(/foo, 1)
  groups: 1; bar
  store: DictStore
>>> g1['foo/bar']
Group(/foo/bar, 1)
  arrays: 1; baz
  store: DictStore
>>> g1['foo/bar/baz']
Array(/foo/bar/baz, (100,), float64, chunks=(10,), order=C)
  nbytes: 800; nbytes_stored: 290; ratio: 2.8; initialized: 0/10
  compressor: Blosc(cname='lz4', clevel=5, shuffle=1)
  store: DictStore

group_keys()¶

Return an iterator over member names for groups only.

Examples

>>> import zarr
>>> g1 = zarr.group()
>>> g2 = g1.create_group('foo')
>>> g3 = g1.create_group('bar')
>>> d1 = g1.create_dataset('baz', shape=100, chunks=10)
>>> d2 = g1.create_dataset('quux', shape=200, chunks=20)
>>> sorted(g1.group_keys())
['bar', 'foo']

groups()¶

Return an iterator over (name, value) pairs for groups only.

Examples

>>> import zarr
>>> g1 = zarr.group()
>>> g2 = g1.create_group('foo')
>>> g3 = g1.create_group('bar')
>>> d1 = g1.create_dataset('baz', shape=100, chunks=10)
>>> d2 = g1.create_dataset('quux', shape=200, chunks=20)
>>> for n, v in g1.groups():
...     print(n, type(v))
bar <class 'zarr.hierarchy.Group'>
foo <class 'zarr.hierarchy.Group'>

array_keys()¶

Return an iterator over member names for arrays only.

Examples

>>> import zarr
>>> g1 = zarr.group()
>>> g2 = g1.create_group('foo')
>>> g3 = g1.create_group('bar')
>>> d1 = g1.create_dataset('baz', shape=100, chunks=10)
>>> d2 = g1.create_dataset('quux', shape=200, chunks=20)
>>> sorted(g1.array_keys())
['baz', 'quux']

arrays()¶

Return an iterator over (name, value) pairs for arrays only.

Examples

>>> import zarr
>>> g1 = zarr.group()
>>> g2 = g1.create_group('foo')
>>> g3 = g1.create_group('bar')
>>> d1 = g1.create_dataset('baz', shape=100, chunks=10)
>>> d2 = g1.create_dataset('quux', shape=200, chunks=20)
>>> for n, v in g1.arrays():
...     print(n, type(v))
baz <class 'zarr.core.Array'>
quux <class 'zarr.core.Array'>

create_group(name, overwrite=False)¶

Create a sub-group.

Parameters:	name : string Group name. overwrite : bool, optional If True, overwrite any existing array with the given name.
Returns:	g : zarr.hierarchy.Group

Examples

>>> import zarr
>>> g1 = zarr.group()
>>> g2 = g1.create_group('foo')
>>> g3 = g1.create_group('bar')
>>> g4 = g1.create_group('baz/quux')

require_group(name, overwrite=False)¶

Obtain a sub-group, creating one if it doesn’t exist.

Parameters:	name : string Group name. overwrite : bool, optional Overwrite any existing array with given name if present.
Returns:	g : zarr.hierarchy.Group

Examples

>>> import zarr
>>> g1 = zarr.group()
>>> g2 = g1.require_group('foo')
>>> g3 = g1.require_group('foo')
>>> g2 == g3
True

create_groups(*names, **kwargs)¶

Convenience method to create multiple groups in a single call.

require_groups(*names)¶

Convenience method to require multiple groups in a single call.

create_dataset(name, **kwargs)¶

Create an array.

Parameters:

name : string Array name. data : array_like, optional Initial data. shape : int or tuple of ints Array shape. chunks : int or tuple of ints, optional Chunk shape. If not provided, will be guessed from shape and dtype. dtype : string or dtype, optional NumPy dtype. compressor : Codec, optional Primary compressor. fill_value : object Default value to use for uninitialized portions of the array. order : {‘C’, ‘F’}, optional Memory layout to be used within each chunk. synchronizer : zarr.sync.ArraySynchronizer, optional Array synchronizer. filters : sequence of Codecs, optional Sequence of filters to use to encode chunk data prior to compression. overwrite : bool, optional If True, replace any existing array or group with the given name. cache_metadata : bool, optional If True, array configuration metadata will be cached for the lifetime of the object. If False, array metadata will be reloaded prior to all data access and modification operations (may incur overhead depending on storage and data access pattern).

Returns:

a : zarr.core.Array

Examples

>>> import zarr
>>> g1 = zarr.group()
>>> d1 = g1.create_dataset('foo', shape=(10000, 10000),
...                        chunks=(1000, 1000))
>>> d1
Array(/foo, (10000, 10000), float64, chunks=(1000, 1000), order=C)
  nbytes: 762.9M; nbytes_stored: 323; ratio: 2476780.2; initialized: 0/100
  compressor: Blosc(cname='lz4', clevel=5, shuffle=1)
  store: DictStore

require_dataset(name, shape, dtype=None, exact=False, **kwargs)¶

Obtain an array, creating if it doesn’t exist. Other kwargs are as per zarr.hierarchy.Group.create_dataset().

Parameters:	name : string Array name. shape : int or tuple of ints Array shape. dtype : string or dtype, optional NumPy dtype. exact : bool, optional If True, require dtype to match exactly. If false, require dtype can be cast from array dtype.

create(name, **kwargs)¶

Create an array. Keyword arguments as per zarr.creation.create().

empty(name, **kwargs)¶

Create an array. Keyword arguments as per zarr.creation.empty().

zeros(name, **kwargs)¶

Create an array. Keyword arguments as per zarr.creation.zeros().

ones(name, **kwargs)¶

Create an array. Keyword arguments as per zarr.creation.ones().

full(name, fill_value, **kwargs)¶

Create an array. Keyword arguments as per zarr.creation.full().

array(name, data, **kwargs)¶

Create an array. Keyword arguments as per zarr.creation.array().

empty_like(name, data, **kwargs)¶

Create an array. Keyword arguments as per zarr.creation.empty_like().

zeros_like(name, data, **kwargs)¶

Create an array. Keyword arguments as per zarr.creation.zeros_like().

ones_like(name, data, **kwargs)¶

Create an array. Keyword arguments as per zarr.creation.ones_like().

full_like(name, data, **kwargs)¶

Create an array. Keyword arguments as per zarr.creation.full_like().

Storage (zarr.storage)¶

This module contains storage classes for use with Zarr arrays and groups. However, note that any object implementing the MutableMapping interface can be used as a Zarr array store.

zarr.storage.init_array(store, shape, chunks=None, dtype=None, compressor='default', fill_value=None, order='C', overwrite=False, path=None, chunk_store=None, filters=None)¶

initialize an array store with the given configuration.

Parameters:

store : MutableMapping A mapping that supports string keys and bytes-like values. shape : int or tuple of ints Array shape. chunks : int or tuple of ints, optional Chunk shape. If not provided, will be guessed from shape and dtype. dtype : string or dtype, optional NumPy dtype. compressor : Codec, optional Primary compressor. fill_value : object Default value to use for uninitialized portions of the array. order : {‘C’, ‘F’}, optional Memory layout to be used within each chunk. overwrite : bool, optional If True, erase all data in store prior to initialisation. path : string, optional Path under which array is stored. chunk_store : MutableMapping, optional Separate storage for chunks. If not provided, store will be used for storage of both chunks and metadata. filters : sequence, optional Sequence of filters to use to encode chunk data prior to compression.

Notes

The initialisation process involves normalising all array metadata, encoding as JSON and storing under the ‘.zarray’ key. User attributes are also initialized and stored as JSON under the ‘.zattrs’ key.

Examples

Initialize an array store:

>>> from zarr.storage import init_array
>>> store = dict()
>>> init_array(store, shape=(10000, 10000), chunks=(1000, 1000))
>>> sorted(store.keys())
['.zarray', '.zattrs']

Array metadata is stored as JSON:

>>> print(str(store['.zarray'], 'ascii'))
{
    "chunks": [
        1000,
        1000
    ],
    "compressor": {
        "clevel": 5,
        "cname": "lz4",
        "id": "blosc",
        "shuffle": 1
    },
    "dtype": "<f8",
    "fill_value": null,
    "filters": null,
    "order": "C",
    "shape": [
        10000,
        10000
    ],
    "zarr_format": 2
}

User-defined attributes are also stored as JSON, initially empty:

>>> print(str(store['.zattrs'], 'ascii'))
{}

Initialize an array using a storage path:

>>> store = dict()
>>> init_array(store, shape=100000000, chunks=1000000, dtype='i1',
...            path='foo')
>>> sorted(store.keys())
['.zattrs', '.zgroup', 'foo/.zarray', 'foo/.zattrs']
>>> print(str(store['foo/.zarray'], 'ascii'))
{
    "chunks": [
        1000000
    ],
    "compressor": {
        "clevel": 5,
        "cname": "lz4",
        "id": "blosc",
        "shuffle": 1
    },
    "dtype": "|i1",
    "fill_value": null,
    "filters": null,
    "order": "C",
    "shape": [
        100000000
    ],
    "zarr_format": 2
}

zarr.storage.init_group(store, overwrite=False, path=None, chunk_store=None)¶

initialize a group store.

Parameters:	store : MutableMapping A mapping that supports string keys and byte sequence values. overwrite : bool, optional If True, erase all data in store prior to initialisation. path : string, optional Path under which array is stored. chunk_store : MutableMapping, optional Separate storage for chunks. If not provided, store will be used for storage of both chunks and metadata.

class zarr.storage.DictStore(cls=<type 'dict'>)¶

Extended mutable mapping interface to a hierarchy of dicts.

Examples

>>> import zarr
>>> store = zarr.DictStore()
>>> store['foo'] = b'bar'
>>> store['foo']
b'bar'
>>> store['a/b/c'] = b'xxx'
>>> store['a/b/c']
b'xxx'
>>> sorted(store.keys())
['a/b/c', 'foo']
>>> store.listdir()
['a', 'foo']
>>> store.listdir('a/b')
['c']
>>> store.rmdir('a')
>>> sorted(store.keys())
['foo']

class zarr.storage.DirectoryStore(path)¶

Mutable Mapping interface to a directory. Keys must be strings, values must be bytes-like objects.

Parameters:	path : string Location of directory.

Examples

>>> import zarr
>>> store = zarr.DirectoryStore('example_store')
>>> store['foo'] = b'bar'
>>> store['foo']
b'bar'
>>> open('example_store/foo', 'rb').read()
b'bar'
>>> store['a/b/c'] = b'xxx'
>>> store['a/b/c']
b'xxx'
>>> open('example_store/a/b/c', 'rb').read()
b'xxx'
>>> sorted(store.keys())
['a/b/c', 'foo']
>>> store.listdir()
['a', 'foo']
>>> store.listdir('a/b')
['c']
>>> store.rmdir('a')
>>> sorted(store.keys())
['foo']
>>> import os
>>> os.path.exists('example_store/a')
False

class zarr.storage.TempStore(suffix='', prefix='zarr', dir=None)¶

Directory store using a temporary directory for storage.

class zarr.storage.ZipStore(path, compression=0, allowZip64=True, mode='a')¶

Mutable Mapping interface to a Zip file. Keys must be strings, values must be bytes-like objects.

Parameters:

path : string Location of file. compression : integer, optional Compression method to use when writing to the archive. allowZip64 : bool, optional If True (the default) will create ZIP files that use the ZIP64 extensions when the zipfile is larger than 2 GiB. If False will raise an exception when the ZIP file would require ZIP64 extensions. mode : string, optional One of ‘r’ to read an existing file, ‘w’ to truncate and write a new file, ‘a’ to append to an existing file, or ‘x’ to exclusively create and write a new file.

Notes

When modifying a ZipStore the close() method must be called otherwise essential data will not be written to the underlying zip file. The ZipStore class also supports the context manager protocol, which ensures the close() method is called on leaving the with statement.

Examples

>>> import zarr
>>> store = zarr.ZipStore('example.zip', mode='w')
>>> store['foo'] = b'bar'
>>> store['foo']
b'bar'
>>> store['a/b/c'] = b'xxx'
>>> store['a/b/c']
b'xxx'
>>> sorted(store.keys())
['a/b/c', 'foo']
>>> store.close()
>>> import zipfile
>>> zf = zipfile.ZipFile('example.zip', mode='r')
>>> sorted(zf.namelist())
['a/b/c', 'foo']

close()¶

Closes the underlying zip file, ensuring all records are written.

flush()¶

Closes the underlying zip file, ensuring all records are written, then re-opens the file for further modifications.

zarr.storage.migrate_1to2(store)¶

Migrate array metadata in store from Zarr format version 1 to version 2.

Parameters:	store : MutableMapping Store to be migrated.

Notes

Version 1 did not support hierarchies, so this migration function will look for a single array in store and migrate the array metadata to version 2.

Compressors and filters (zarr.codecs)¶

This module contains compressor and filter classes for use with Zarr.

Other codecs can be registered dynamically with Zarr. All that is required is to implement a class that provides the same interface as the classes listed below, and then to add the class to the codec_registry. See the source code of this module for details.

class zarr.codecs.Codec¶

Codec abstract base class.

encode(buf)¶

Encode data in buf.

Parameters:	buf : buffer-like Data to be encoded. May be any object supporting the new-style buffer protocol or array.array.
Returns:	enc : buffer-like Encoded data. May be any object supporting the new-style buffer protocol or array.array.

decode(buf, out=None)¶

Decode data in buf.

Parameters:	buf : buffer-like Encoded data. May be any object supporting the new-style buffer protocol or array.array. out : buffer-like, optional Buffer to store decoded data.
Returns:	out : buffer-like Decoded data. May be any object supporting the new-style buffer protocol or array.array.

get_config()¶

Return a dictionary holding configuration parameters for this codec. All values must be compatible with JSON encoding.

classmethod from_config(config)¶

Instantiate from a configuration object.

class zarr.codecs.Blosc(cname='lz4', clevel=5, shuffle=1)¶

Provides compression using the blosc meta-compressor.

Parameters:	cname : string, optional A string naming one of the compression algorithms available within blosc, e.g., ‘blosclz’, ‘lz4’, ‘zlib’ or ‘snappy’. clevel : integer, optional An integer between 0 and 9 specifying the compression level. shuffle : integer, optional Either 0 (no shuffle), 1 (byte shuffle) or 2 (bit shuffle).

class zarr.codecs.Zlib(level=1)¶

Provides compression using zlib via the Python standard library.

Parameters:	level : int Compression level.

class zarr.codecs.BZ2(level=1)¶

Provides compression using bzip2 via the Python standard library.

Parameters:	level : int Compression level.

class zarr.codecs.LZMA(format=1, check=-1, preset=None, filters=None)¶

Provides compression using lzma via the Python standard library (only available under Python 3).

Parameters:	format : integer, optional One of the lzma format codes, e.g., lzma.FORMAT_XZ. check : integer, optional One of the lzma check codes, e.g., lzma.CHECK_NONE. preset : integer, optional An integer between 0 and 9 inclusive, specifying the compression level. filters : list, optional A list of dictionaries specifying compression filters. If filters are provided, ‘preset’ must be None.

class zarr.codecs.Delta(dtype, astype=None)¶

Filter to encode data as the difference between adjacent values.

Parameters:	dtype : dtype Data type to use for decoded data. astype : dtype, optional Data type to use for encoded data.

Notes

If astype is an integer data type, please ensure that it is sufficiently large to store encoded values. No checks are made and data may become corrupted due to integer overflow if astype is too small. Note also that the encoded data for each chunk includes the absolute value of the first element in the chunk, and so the encoded data type in general needs to be large enough to store absolute values from the array.

Examples

>>> import zarr
>>> import numpy as np
>>> x = np.arange(100, 120, 2, dtype='i8')
>>> f = zarr.Delta(dtype='i8', astype='i1')
>>> y = f.encode(x)
>>> y
array([100,   2,   2,   2,   2,   2,   2,   2,   2,   2], dtype=int8)
>>> z = f.decode(y)
>>> z
array([100, 102, 104, 106, 108, 110, 112, 114, 116, 118])

class zarr.codecs.FixedScaleOffset(offset, scale, dtype, astype=None)¶

Simplified version of the scale-offset filter available in HDF5. Applies the transformation (x - offset) * scale to all chunks. Results are rounded to the nearest integer but are not packed according to the minimum number of bits.

Parameters:	offset : float Value to subtract from data. scale : int Value to multiply by data. dtype : dtype Data type to use for decoded data. astype : dtype, optional Data type to use for encoded data.

Notes

If astype is an integer data type, please ensure that it is sufficiently large to store encoded values. No checks are made and data may become corrupted due to integer overflow if astype is too small.

Examples

>>> import zarr
>>> import numpy as np
>>> x = np.linspace(1000, 1001, 10, dtype='f8')
>>> x
array([ 1000.        ,  1000.11111111,  1000.22222222,  1000.33333333,
        1000.44444444,  1000.55555556,  1000.66666667,  1000.77777778,
        1000.88888889,  1001.        ])
>>> f1 = zarr.FixedScaleOffset(offset=1000, scale=10, dtype='f8', astype='u1')
>>> y1 = f1.encode(x)
>>> y1
array([ 0,  1,  2,  3,  4,  6,  7,  8,  9, 10], dtype=uint8)
>>> z1 = f1.decode(y1)
>>> z1
array([ 1000. ,  1000.1,  1000.2,  1000.3,  1000.4,  1000.6,  1000.7,
        1000.8,  1000.9,  1001. ])
>>> f2 = zarr.FixedScaleOffset(offset=1000, scale=10**2, dtype='f8', astype='u1')
>>> y2 = f2.encode(x)
>>> y2
array([  0,  11,  22,  33,  44,  56,  67,  78,  89, 100], dtype=uint8)
>>> z2 = f2.decode(y2)
>>> z2
array([ 1000.  ,  1000.11,  1000.22,  1000.33,  1000.44,  1000.56,
        1000.67,  1000.78,  1000.89,  1001.  ])
>>> f3 = zarr.FixedScaleOffset(offset=1000, scale=10**3, dtype='f8', astype='u2')
>>> y3 = f3.encode(x)
>>> y3
array([   0,  111,  222,  333,  444,  556,  667,  778,  889, 1000], dtype=uint16)
>>> z3 = f3.decode(y3)
>>> z3
array([ 1000.   ,  1000.111,  1000.222,  1000.333,  1000.444,  1000.556,
        1000.667,  1000.778,  1000.889,  1001.   ])

class zarr.codecs.Quantize(digits, dtype, astype=None)¶

Lossy filter to reduce the precision of floating point data.

Parameters:	digits : int Desired precision (number of decimal digits). dtype : dtype Data type to use for decoded data. astype : dtype, optional Data type to use for encoded data.

Examples

>>> import zarr
>>> import numpy as np
>>> x = np.linspace(0, 1, 10, dtype='f8')
>>> x
array([ 0.        ,  0.11111111,  0.22222222,  0.33333333,  0.44444444,
        0.55555556,  0.66666667,  0.77777778,  0.88888889,  1.        ])
>>> f1 = zarr.Quantize(digits=1, dtype='f8')
>>> y1 = f1.encode(x)
>>> y1
array([ 0.    ,  0.125 ,  0.25  ,  0.3125,  0.4375,  0.5625,  0.6875,
        0.75  ,  0.875 ,  1.    ])
>>> f2 = zarr.Quantize(digits=2, dtype='f8')
>>> y2 = f2.encode(x)
>>> y2
array([ 0.       ,  0.109375 ,  0.21875  ,  0.3359375,  0.4453125,
        0.5546875,  0.6640625,  0.78125  ,  0.890625 ,  1.       ])
>>> f3 = zarr.Quantize(digits=3, dtype='f8')
>>> y3 = f3.encode(x)
>>> y3
array([ 0.        ,  0.11132812,  0.22265625,  0.33300781,  0.44433594,
        0.55566406,  0.66699219,  0.77734375,  0.88867188,  1.        ])

class zarr.codecs.PackBits¶

Filter to pack elements of a boolean array into bits in a uint8 array.

Notes

The first element of the encoded array stores the number of bits that were padded to complete the final byte.

Examples

>>> import zarr
>>> import numpy as np
>>> f = zarr.PackBits()
>>> x = np.array([True, False, False, True], dtype=bool)
>>> y = f.encode(x)
>>> y
array([  4, 144], dtype=uint8)
>>> z = f.decode(y)
>>> z
array([ True, False, False,  True], dtype=bool)

class zarr.codecs.Categorize(labels, dtype, astype='u1')¶

Filter encoding categorical string data as integers.

Parameters:	labels : sequence of strings Category labels. dtype : dtype Data type to use for decoded data. astype : dtype, optional Data type to use for encoded data.

Examples

>>> import zarr
>>> import numpy as np
>>> x = np.array([b'male', b'female', b'female', b'male', b'unexpected'])
>>> x
array([b'male', b'female', b'female', b'male', b'unexpected'],
      dtype='|S10')
>>> f = zarr.Categorize(labels=[b'female', b'male'], dtype=x.dtype)
>>> y = f.encode(x)
>>> y
array([2, 1, 1, 2, 0], dtype=uint8)
>>> z = f.decode(y)
>>> z
array([b'male', b'female', b'female', b'male', b''],
      dtype='|S10')

Synchronization (zarr.sync)¶

class zarr.sync.ThreadSynchronizer¶

Provides synchronization using thread locks.

class zarr.sync.ProcessSynchronizer(path)¶

Provides synchronization using file locks via the fasteners package.

Parameters:	path : string Path to a directory on a file system that is shared by all processes. N.B., this should be a different path to where you store the array.

Zarr storage specification version 1¶

This document provides a technical specification of the protocol and format used for storing a Zarr array. The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in RFC 2119.

{
    "chunks": [
        1000,
        1000
    ],
    "compression": "blosc",
    "compression_opts": {
        "clevel": 5,
        "cname": "lz4",
        "shuffle": 1
    },
    "dtype": "<f8",
    "fill_value": null,
    "order": "C",
    "shape": [
        10000,
        10000
    ],
    "zarr_format": 1
}

{
    "foo": 42,
    "bar": "apples",
    "baz": [1, 2, 3, 4]
}

>>> import zarr
>>> store = zarr.DirectoryStore('example.zarr')
>>> zarr.init_store(store, shape=(20, 20), chunks=(10, 10),
...                 dtype='i4', fill_value=42, compression='zlib',
...                 compression_opts=1, overwrite=True)

>>> import os
>>> sorted(os.listdir('example.zarr'))
['attrs', 'meta']

>>> print(open('example.zarr/meta').read())
{
    "chunks": [
        10,
        10
    ],
    "compression": "zlib",
    "compression_opts": 1,
    "dtype": "<i4",
    "fill_value": 42,
    "order": "C",
    "shape": [
        20,
        20
    ],
    "zarr_format": 1
}

>>> print(open('example.zarr/attrs').read())
{}

>>> z = zarr.Array(store)
>>> z[0:10, 0:10] = 1
>>> sorted(os.listdir('example.zarr'))
['0.0', 'attrs', 'meta']

>>> z[0:10, 10:20] = 2
>>> z[10:20, :] = 3
>>> sorted(os.listdir('example.zarr'))
['0.0', '0.1', '1.0', '1.1', 'attrs', 'meta']

>>> import zlib
>>> b = zlib.decompress(open('example.zarr/0.0', 'rb').read())
>>> import numpy as np
>>> a = np.frombuffer(b, dtype='<i4')
>>> a
array([1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
       1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
       1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
       1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
       1, 1, 1, 1, 1, 1, 1, 1], dtype=int32)

>>> z.attrs['foo'] = 42
>>> z.attrs['bar'] = 'apples'
>>> z.attrs['baz'] = [1, 2, 3, 4]
>>> print(open('example.zarr/attrs').read())
{
    "bar": "apples",
    "baz": [
        1,
        2,
        3,
        4
    ],
    "foo": 42
}

Zarr storage specification version 2¶

This document provides a technical specification of the protocol and format used for storing Zarr arrays. The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in RFC 2119.

{
    "chunks": [
        1000,
        1000
    ],
    "compressor": {
        "id": "blosc",
        "cname": "lz4",
        "clevel": 5,
        "shuffle": 1
    },
    "dtype": "<f8",
    "fill_value": "NaN",
    "filters": [
        {"id": "delta", "dtype": "<f8", "astype": "<f4"}
    ],
    "order": "C",
    "shape": [
        10000,
        10000
    ],
    "zarr_format": 2
}

{
    "foo": 42,
    "bar": "apples",
    "baz": [1, 2, 3, 4]
}

>>> import zarr
>>> store = zarr.DirectoryStore('example')
>>> a = zarr.create(shape=(20, 20), chunks=(10, 10), dtype='i4',
...                 fill_value=42, compressor=zarr.Zlib(level=1),
...                 store=store, overwrite=True)

>>> import os
>>> sorted(os.listdir('example'))
['.zarray', '.zattrs']

>>> print(open('example/.zarray').read())
{
    "chunks": [
        10,
        10
    ],
    "compressor": {
        "id": "zlib",
        "level": 1
    },
    "dtype": "<i4",
    "fill_value": 42,
    "filters": null,
    "order": "C",
    "shape": [
        20,
        20
    ],
    "zarr_format": 2
}

>>> print(open('example/.zattrs').read())
{}

>>> a[0:10, 0:10] = 1
>>> sorted(os.listdir('example'))
['.zarray', '.zattrs', '0.0']

>>> a[0:10, 10:20] = 2
>>> a[10:20, :] = 3
>>> sorted(os.listdir('example'))
['.zarray', '.zattrs', '0.0', '0.1', '1.0', '1.1']

>>> import zlib
>>> buf = zlib.decompress(open('example/0.0', 'rb').read())
>>> import numpy as np
>>> chunk = np.frombuffer(buf, dtype='<i4')
>>> chunk
array([1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
       1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
       1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
       1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
       1, 1, 1, 1, 1, 1, 1, 1], dtype=int32)