Skip to content

HDF5

ddacs.open_h5 and ddacs.inspect_h5 are the top level helpers for reading and inspecting a single simulation. Both are re-exported at the package root (ddacs.open_h5, ddacs.inspect_h5); the implementation lives in ddacs.h5_tools.

ddacs.open_h5

open_h5(sim_id: int, source: str | Path | None = None, data_dir: str | Path | None = DEFAULT_DATA_DIR, dataset=None, spec: DatasetSpec = DDACS_SPEC) -> h5py.File

Return an h5py.File for the requested simulation.

Looks the manifest up, walks the locally mapped zips and reads the h5 member matching <sim_id>.h5 into a BytesIO. The returned object is read-only, supports the with idiom and can be indexed like any other h5py.File.

Parameters:

Name Type Description Default
sim_id int

The simulation id (matches the h5 filename inside the zip).

 required
source str | Path | None

Override the Croissant manifest URL / path. Falls back to ddacs.croissant.resolve_source resolution.

 None
data_dir str | Path | None

Override the directory searched for already-downloaded zips. Defaults to ddacs.spec.DDACS_SPEC.default_data_dir. Pass None to skip the local lookup entirely.

 DEFAULT_DATA_DIR
dataset

A pre-loaded mlcroissant.Dataset (e.g. from ddacs.load). When given, source and data_dir are ignored: the manifest is not re-parsed and the caller's object is used. Useful when the caller has applied ddacs.add_view and wants the mutation to stay in scope.

 None

Raises:

Type Description
FileNotFoundError

No locally mapped zip contained the requested h5.
Source code in ddacs/h5_tools.py 
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
def open_h5(
    sim_id: int,
    source: str | Path | None = None,
    data_dir: str | Path | None = DEFAULT_DATA_DIR,
    dataset=None,
    spec: DatasetSpec = DDACS_SPEC,
) -> h5py.File:
    """Return an `h5py.File` for the requested simulation.

    Looks the manifest up, walks the locally mapped zips and reads the h5
    member matching `<sim_id>.h5` into a `BytesIO`. The returned object is
    read-only, supports the `with` idiom and can be indexed like any other
    `h5py.File`.

    Args:
        sim_id: The simulation id (matches the h5 filename inside the zip).
        source: Override the Croissant manifest URL / path. Falls back to
            `ddacs.croissant.resolve_source` resolution.
        data_dir: Override the directory searched for already-downloaded
            zips. Defaults to `ddacs.spec.DDACS_SPEC.default_data_dir`. Pass `None`
            to skip the local lookup entirely.
        dataset: A pre-loaded `mlcroissant.Dataset` (e.g. from `ddacs.load`).
            When given, `source` and `data_dir` are ignored: the manifest is
            not re-parsed and the caller's object is used. Useful when the
            caller has applied `ddacs.add_view` and wants the mutation to
            stay in scope.

    Raises:
        FileNotFoundError: No locally mapped zip contained the requested h5.
    """
    ds = (
        dataset
        if dataset is not None
        else _croissant.load(source=source, data_dir=data_dir, spec=spec)
    )
    h5_name = f"{spec.id_format.format(int(sim_id))}.h5"

    for zip_path in (ds.mapping or {}).values():
        zip_path = str(zip_path)
        if not zip_path.endswith(".zip"):
            continue
        try:
            with zipfile.ZipFile(zip_path) as zf:
                if h5_name in zf.namelist():
                    return h5py.File(io.BytesIO(zf.read(h5_name)), "r")
        except zipfile.BadZipFile:
            continue

    hint = (
        " (data_dir lookup was skipped — pass data_dir=...)"
        if data_dir is None
        else f" under {data_dir!r}. Run `{spec.prog} download` to fetch the dataset first."
    )
    raise FileNotFoundError(f"{h5_name} not found in any locally mapped zip{hint}")

ddacs.inspect_h5

inspect_h5(file_or_path: h5py.File | str | Path) -> None

Pretty-print the group/dataset hierarchy of an HDF5 file.

Accepts either an open h5py.File (e.g. from ddacs.open_h5) or a path on disk. Prints to stdout and returns nothing.

Source code in ddacs/h5_tools.py 
82
83
84
85
86
87
88
89
90
91
92
def inspect_h5(file_or_path: h5py.File | str | Path) -> None:
    """Pretty-print the group/dataset hierarchy of an HDF5 file.

    Accepts either an open `h5py.File` (e.g. from `ddacs.open_h5`) or a path
    on disk. Prints to stdout and returns nothing.
    """
    if isinstance(file_or_path, h5py.File):
        _print_tree(file_or_path)
    else:
        with h5py.File(file_or_path, "r") as f:
            _print_tree(f)