Reference¶
h5preserve is a thin wrapper around h5py, providing easier serialisation of native python types.
- copyright
2016 James Tocknell
- license
3-clause BSD
-
class
h5preserve.
DatasetContainer
(attrs=None, **kwargs)[source]¶ Representation of an hdf5 dataset for use in h5preserve.
- Parameters
attrs (Mapping) – mapping containing the attributes of the group
**kwargs – properties of the group, which get passed to create group
-
clear
() → None. Remove all items from D.¶
-
get
(k[, d]) → D[k] if k in D, else d. d defaults to None.¶
-
items
() → a set-like object providing a view on D’s items¶
-
keys
() → a set-like object providing a view on D’s keys¶
-
pop
(k[, d]) → v, remove specified key and return the corresponding value.¶ If key is not found, d is returned if given, otherwise KeyError is raised.
-
popitem
() → (k, v), remove and return some (key, value) pair¶ as a 2-tuple; but raise KeyError if D is empty.
-
setdefault
(k[, d]) → D.get(k,d), also set D[k]=d if k not in D¶
-
update
([E, ]**F) → None. Update D from mapping/iterable E and F.¶ If E present and has a .keys() method, does: for k in E: D[k] = E[k] If E present and lacks .keys() method, does: for (k, v) in E: D[k] = v In either case, this is followed by: for k, v in F.items(): D[k] = v
-
values
() → an object providing a view on D’s values¶
-
class
h5preserve.
DelayedContainer
[source]¶ Helper class for allowing delayed writing of containers to hdf5 files.
-
class
h5preserve.
GroupContainer
(attrs=None, **kwargs)[source]¶ Representation of an hdf5 group for use in h5preserve.
- Parameters
attrs (Mapping) – mapping containing the attributes of the group
**kwargs – datasets or subgroups to add to the group
-
clear
() → None. Remove all items from D.¶
-
get
(k[, d]) → D[k] if k in D, else d. d defaults to None.¶
-
items
() → a set-like object providing a view on D’s items¶
-
keys
() → a set-like object providing a view on D’s keys¶
-
pop
(k[, d]) → v, remove specified key and return the corresponding value.¶ If key is not found, d is returned if given, otherwise KeyError is raised.
-
popitem
() → (k, v), remove and return some (key, value) pair¶ as a 2-tuple; but raise KeyError if D is empty.
-
setdefault
(k[, d]) → D.get(k,d), also set D[k]=d if k not in D¶
-
update
([E, ]**F) → None. Update D from mapping/iterable E and F.¶ If E present and has a .keys() method, does: for k in E: D[k] = E[k] If E present and lacks .keys() method, does: for (k, v) in E: D[k] = v In either case, this is followed by: for k, v in F.items(): D[k] = v
-
values
() → an object providing a view on D’s values¶
-
class
h5preserve.
H5PreserveFile
(h5py_file, registries)[source]¶ Thin wrapper around
h5py.File
to automatically use h5preserve when accessing the file contents.Acts like
h5preserve.H5PreserveGroup
, but allows access to the associatedh5py.File
instance viah5py_file
.- Parameters
h5py_file (a
h5py.File
) – the hdf5 file to wrapregistries (RegistryContainer) – the collection of registries that you want to use to read from the hdf5 file
-
clear
() → None. Remove all items from D.¶
-
create_group
(name)¶ Creates a new group in the associated hdf5 file
- Parameters
name (string, or other identifier accepted by h5py) – name of the new group
- Returns
The new group wrapped by H5PreserveGroup
- Return type
-
get
(k[, d]) → D[k] if k in D, else d. d defaults to None.¶
-
property
h5py_file
¶ h5py.File: the instance of
h5py.File
whichH5PreserveFile
wraps
-
property
h5py_group
¶ h5py.Group: the instance of
h5py.Group
whichH5PreserveGroup
wraps
-
items
() → a set-like object providing a view on D’s items¶
-
keys
() → a set-like object providing a view on D’s keys¶
-
pop
(k[, d]) → v, remove specified key and return the corresponding value.¶ If key is not found, d is returned if given, otherwise KeyError is raised.
-
popitem
() → (k, v), remove and return some (key, value) pair¶ as a 2-tuple; but raise KeyError if D is empty.
-
require_group
(name)¶ Returns the group associated with
name
, creating it if necessary.- Parameters
name (string, or other identifier accepted by h5py) – name of the desired group
- Returns
The group wrapped by H5PreserveGroup
- Return type
-
setdefault
(k[, d]) → D.get(k,d), also set D[k]=d if k not in D¶
-
update
([E, ]**F) → None. Update D from mapping/iterable E and F.¶ If E present and has a .keys() method, does: for k in E: D[k] = E[k] If E present and lacks .keys() method, does: for (k, v) in E: D[k] = v In either case, this is followed by: for k, v in F.items(): D[k] = v
-
values
() → an object providing a view on D’s values¶
-
class
h5preserve.
H5PreserveGroup
(h5py_group, registries)[source]¶ Thin wrapper around
h5py.Group
to automatically use h5preserve when accessing the group contents.- Parameters
h5py_group (
h5py.Group
) –registries (RegistryContainer) – the collection of registries that you want to use to read from the hdf5 file
-
clear
() → None. Remove all items from D.¶
-
create_group
(name)[source]¶ Creates a new group in the associated hdf5 file
- Parameters
name (string, or other identifier accepted by h5py) – name of the new group
- Returns
The new group wrapped by H5PreserveGroup
- Return type
-
get
(k[, d]) → D[k] if k in D, else d. d defaults to None.¶
-
property
h5py_group
¶ h5py.Group: the instance of
h5py.Group
whichH5PreserveGroup
wraps
-
items
() → a set-like object providing a view on D’s items¶
-
keys
() → a set-like object providing a view on D’s keys¶
-
pop
(k[, d]) → v, remove specified key and return the corresponding value.¶ If key is not found, d is returned if given, otherwise KeyError is raised.
-
popitem
() → (k, v), remove and return some (key, value) pair¶ as a 2-tuple; but raise KeyError if D is empty.
-
require_group
(name)[source]¶ Returns the group associated with
name
, creating it if necessary.- Parameters
name (string, or other identifier accepted by h5py) – name of the desired group
- Returns
The group wrapped by H5PreserveGroup
- Return type
-
setdefault
(k[, d]) → D.get(k,d), also set D[k]=d if k not in D¶
-
update
([E, ]**F) → None. Update D from mapping/iterable E and F.¶ If E present and has a .keys() method, does: for k in E: D[k] = E[k] If E present and lacks .keys() method, does: for (k, v) in E: D[k] = v In either case, this is followed by: for k, v in F.items(): D[k] = v
-
values
() → an object providing a view on D’s values¶
-
class
h5preserve.
HardLink
(obj)[source]¶ Represent a h5py hard link to be created via h5preserve.
- Parameters
obj (string, h5py.Group or h5py.Dataset) – the h5py object that the hard link points to, can either be an h5py object, or a string with the absolute path of the object
-
property
h5py_obj
¶ h5py.Group or h5py.Dataset: the object which the hard link will point to
-
property
path
¶ The path this object points to
-
class
h5preserve.
OnDemandDatasetContainer
(attrs=None, **kwargs)[source]¶ Subclass of DatasetContainer which supports accessing dataset data on demand, rather that loading immediately.
-
clear
() → None. Remove all items from D.¶
-
get
(k[, d]) → D[k] if k in D, else d. d defaults to None.¶
-
items
() → a set-like object providing a view on D’s items¶
-
keys
() → a set-like object providing a view on D’s keys¶
-
pop
(k[, d]) → v, remove specified key and return the corresponding value.¶ If key is not found, d is returned if given, otherwise KeyError is raised.
-
popitem
() → (k, v), remove and return some (key, value) pair¶ as a 2-tuple; but raise KeyError if D is empty.
-
setdefault
(k[, d]) → D.get(k,d), also set D[k]=d if k not in D¶
-
update
([E, ]**F) → None. Update D from mapping/iterable E and F.¶ If E present and has a .keys() method, does: for k in E: D[k] = E[k] If E present and lacks .keys() method, does: for (k, v) in E: D[k] = v In either case, this is followed by: for k, v in F.items(): D[k] = v
-
values
() → an object providing a view on D’s values¶
-
-
class
h5preserve.
OnDemandGroupContainer
(attrs=None, **kwargs)[source]¶ Subclass of GroupContainer which supports accessing group members on demand, rather that loading immediately.
-
clear
() → None. Remove all items from D.¶
-
get
(k[, d]) → D[k] if k in D, else d. d defaults to None.¶
-
items
() → a set-like object providing a view on D’s items¶
-
keys
() → a set-like object providing a view on D’s keys¶
-
pop
(k[, d]) → v, remove specified key and return the corresponding value.¶ If key is not found, d is returned if given, otherwise KeyError is raised.
-
popitem
() → (k, v), remove and return some (key, value) pair¶ as a 2-tuple; but raise KeyError if D is empty.
-
setdefault
(k[, d]) → D.get(k,d), also set D[k]=d if k not in D¶
-
update
([E, ]**F) → None. Update D from mapping/iterable E and F.¶ If E present and has a .keys() method, does: for k in E: D[k] = E[k] If E present and lacks .keys() method, does: for (k, v) in E: D[k] = v In either case, this is followed by: for k, v in F.items(): D[k] = v
-
values
() → an object providing a view on D’s values¶
-
-
class
h5preserve.
OnDemandWrapper
(func)[source]¶ Wrapper which represents a container which can be accessed on demand.
-
class
h5preserve.
Registry
(name)[source]¶ Register of functions for converting between hdf5 and python.
This is the core of h5preserve, containing the information about how to convert to and from hdf5 files, what version to use, and the namespace of created data.
- Parameters
name (string) – name of registry for identification purposes
-
dumper
(cls, label, version)[source]¶ Decorator function to create a dumper function.
- Parameters
cls (any class) – the class which this dumper operates on
label (string) – the label or tag associated with this class
version (integer, None) – The version of the output that this function returns.
-
loader
(label, version)[source]¶ Decorator function to create a loader function.
- Parameters
label (string) – the label or tag associated with this class
version (integer, any, all, None) – The version of the output that this function reads.
-
property
name
¶ str: name of the registry
-
class
h5preserve.
RegistryContainer
(*registries)[source]¶ Ordered container of registries which manages interaction with the hdf5 file.
- Parameters
*registries (list of Registry) – the list of registries to be associated with this container
-
append
(value)¶ S.append(value) – append value to the end of the sequence
-
clear
() → None – remove all items from S¶
-
count
(value) → integer – return number of occurrences of value¶
-
dump
(obj)[source]¶ Dump native python object to h5preserve representation
- Parameters
obj – the object to dump
-
extend
(values)¶ S.extend(iterable) – extend sequence by appending elements from the iterable
-
from_file
(h5py_obj)[source]¶ Return an representation of a hdf5 object from a hdf5 file
- Parameters
h5py_obj (a
h5py
object, e.g. group, dataset) –
-
index
(value[, start[, stop]]) → integer – return first index of value.¶ Raises ValueError if the value is not present.
Supporting start and stop arguments is optional, but recommended.
-
load
(obj)[source]¶ Load native python object from h5preserve representation
- Parameters
obj – the object to load
-
lock_version
(cls, version)[source]¶ Lock output version for a specific class
- Parameters
cls (any class) – the class to lock the version of
version (integer, any, all, None) – the version which will always be used
-
pop
([index]) → item – remove and return item at index (default last).¶ Raise IndexError if list is empty or index is out of range.
-
property
registries
¶ Iterator over the registries contained in the order they were added.
-
remove
(value)¶ S.remove(value) – remove first occurrence of value. Raise ValueError if the value is not present.
-
reverse
()¶ S.reverse() – reverse IN PLACE
-
h5preserve.
new_registry_list
(*registries)[source]¶ Create a new list of registries which includes builtin registries.
- Parameters
*registries (list of Registry) – the list of registries to be associated with this container
-
h5preserve.
open
(filename, registries, *, mode, **kwargs)[source]¶ Open a hdf5 file wrapped with h5preserve.
- Parameters
filename (string, or other identifier accepted by
h5py.File
) –registries (RegistryContainer) – the collection of registries that you want to use to read from the hdf5 file
**kwargs – additional keyword arguments to pass to
h5py.File
-
h5preserve.
wrap_on_demand
(obj, key, val)[source]¶ Wrap val such that it can be used on demand.
wrap_on_demand returns either the original val if obj has not yet been dumped, or a wrapped version of val if obj has been dumped.
wrap_on_demand automatically deals with wrapping/unwrapping if needed, so it is save to repeatedly call on the same object.
- Parameters
obj (any dumpable object) – the object which val is a member of or an attribute of
key (string) – the key to be used when writing out val
val (any dumpable object) – the object to be wrapped