In memory hierarchy of arrays and attributes loaded from disk or to be saved to disk with Zarr.
-
If you need to store huge datasets that cannot fit uncompressed in memory consider using https://github.com/JuliaIO/HDF5.jl or https://github.com/JuliaIO/Zarr.jl
-
If you just want to serialize arbitrary Julia data consider using https://github.com/JuliaIO/JLD2.jl or https://github.com/invenia/JLSO.jl
-
Only Numpy types "b i u f c V" are supported.
-
Zarr filters are not supported.
ZGrouprepresents a tree with arrays as leaves.ZGroupleaf arrays are uncompressed but store metadata about how they should be compressed when saved to disk.ZGroupcan have JSON3 serializable attributes attached to any node or leaf.- Data can be quickly accessed and modified in
ZGroup. - No file open close semantics. Use the Julia garbage collector to clean memory up.
- Save and load
ZGroupin a directory or zip file in Zarr v2 format. ZGroupsaved to a directory or zip file can be read in other languages.
See test/test_simple-usage.jl for examples and documentation.