Storage (zarr.storage)

This module contains a single DirectoryStore class providing a MutableMapping interface to a directory on the file system. However, note that any object implementing the MutableMapping interface can be used as a Zarr array store.

zarr.storage.init_store(store, shape, chunks, dtype=None, compression='default', compression_opts=None, fill_value=None, order='C', overwrite=False)

Initialise an array store with the given configuration.

Parameters:

store : MutableMapping

A mapping that supports string keys and byte sequence values.

shape : int or tuple of ints

Array shape.

chunks : int or tuple of ints

Chunk shape.

dtype : string or dtype, optional

NumPy dtype.

compression : string, optional

Name of primary compression library, e.g., ‘blosc’, ‘zlib’, ‘bz2’, ‘lzma’.

compression_opts : object, optional

Options to primary compressor. E.g., for blosc, provide a dictionary with keys ‘cname’, ‘clevel’ and ‘shuffle’.

fill_value : object

Default value to use for uninitialised portions of the array.

order : {‘C’, ‘F’}, optional

Memory layout to be used within each chunk.

overwrite : bool, optional

If True, erase all data in store prior to initialisation.

Notes

The initialisation process involves normalising all array metadata, encoding as JSON and storing under the ‘meta’ key. User attributes are also initialised and stored as JSON under the ‘attrs’ key.

Examples

>>> import zarr
>>> store = dict()
>>> zarr.init_store(store, shape=(10000, 10000), chunks=(1000, 1000))
>>> sorted(store.keys())
['attrs', 'meta']
>>> print(str(store['meta'], 'ascii'))
{
    "chunks": [
        1000,
        1000
    ],
    "compression": "blosc",
    "compression_opts": {
        "clevel": 5,
        "cname": "lz4",
        "shuffle": 1
    },
    "dtype": "<f8",
    "fill_value": null,
    "order": "C",
    "shape": [
        10000,
        10000
    ],
    "zarr_format": 1
}
>>> print(str(store['attrs'], 'ascii'))
{}
class zarr.storage.DirectoryStore(path)

Mutable Mapping interface to a directory. Keys must be strings, values must be bytes-like objects.

Parameters:

path : string

Location of directory.

Examples

>>> import zarr
>>> store = zarr.DirectoryStore('example.zarr')
>>> zarr.init_store(store, shape=(10000, 10000), chunks=(1000, 1000),
...                 fill_value=0, overwrite=True)
>>> import os
>>> sorted(os.listdir('example.zarr'))
['attrs', 'meta']
>>> print(open('example.zarr/meta').read())
{
    "chunks": [
        1000,
        1000
    ],
    "compression": "blosc",
    "compression_opts": {
        "clevel": 5,
        "cname": "lz4",
        "shuffle": 1
    },
    "dtype": "<f8",
    "fill_value": 0,
    "order": "C",
    "shape": [
        10000,
        10000
    ],
    "zarr_format": 1
}
>>> print(open('example.zarr/attrs').read())
{}
>>> z = zarr.Array(store)
>>> z
zarr.core.Array((10000, 10000), float64, chunks=(1000, 1000), order=C)
  compression: blosc; compression_opts: {'clevel': 5, 'cname': 'lz4', 'shuffle': 1}
  nbytes: 762.9M; nbytes_stored: 313; ratio: 2555910.5; initialized: 0/100
  store: zarr.storage.DirectoryStore
>>> z[:] = 1
>>> len(os.listdir('example.zarr'))
102
>>> sorted(os.listdir('example.zarr'))[0:5]
['0.0', '0.1', '0.2', '0.3', '0.4']
>>> print(open('example.zarr/0.0', 'rb').read(10))
b'\x02\x01!\x08\x00\x12z\x00\x00\x80'