Croissant

ddacs.load and ddacs.add_view are the two entry points to the Croissant manifest. Both are re-exported at the top level (ddacs.load, ddacs.add_view); the implementation lives in ddacs.croissant.

ddacs.load

load(source: str | Path | None = None, data_dir: str | Path | None = DEFAULT_DATA_DIR, spec: DatasetSpec = DDACS_SPEC) -> mlc.Dataset

Return an :class:mlcroissant.Dataset for the DDACS manifest.

Local-first, URL-fallback resolution — see :func:resolve_source. When data_dir points at a directory that contains files referenced by the manifest (e.g. zips written by ddacs download), mlcroissant is told to use those local copies instead of refetching from DaRUS.

Pass data_dir=None to opt out of local-file discovery and force mlcroissant to download via its own cache.

Source code in ddacs/croissant.py

def load(
    source: str | Path | None = None,
    data_dir: str | Path | None = DEFAULT_DATA_DIR,
    spec: DatasetSpec = DDACS_SPEC,
) -> mlc.Dataset:
    """Return an :class:`mlcroissant.Dataset` for the DDACS manifest.

    Local-first, URL-fallback resolution — see :func:`resolve_source`. When
    ``data_dir`` points at a directory that contains files referenced by the
    manifest (e.g. zips written by ``ddacs download``), `mlcroissant` is told
    to use those local copies instead of refetching from DaRUS.

    Pass ``data_dir=None`` to opt out of local-file discovery and force
    `mlcroissant` to download via its own cache.
    """
    jsonld = resolve_source(source, data_dir, spec=spec)
    mapping = _build_mapping(jsonld, data_dir) if data_dir is not None else None
    return mlc.Dataset(jsonld=jsonld, mapping=mapping)

ddacs.add_view

add_view(ds: mlc.Dataset, name: str, fields: dict[str, FieldSpec]) -> mlc.Dataset

Add a custom RecordSet to a loaded dataset and rewire it in place.

Each fields entry references a manifest field by name. Bare ids resolve to field-map (the h5 source); qualified ids of the form "<record-set>/<field>" can pull from any other RecordSet, e.g. "process-parameters/sheet_metal_thickness":

• "name": "field_id" — whole field-map field
• "name": ("field_id", None | int | list[int]) — explicit, with optional timestep slicing
• "name": "process-parameters/<column>" — CSV column from the index

Timestep slicing is only valid on field-map sources. The published manifest on DaRUS is untouched — only the in-memory representation grows. ds is mutated in place and returned for optional chaining.

Source code in ddacs/croissant.py

def add_view(
    ds: mlc.Dataset,
    name: str,
    fields: dict[str, FieldSpec],
) -> mlc.Dataset:
    """Add a custom RecordSet to a loaded dataset and rewire it in place.

    Each ``fields`` entry references a manifest field by name. Bare ids
    resolve to ``field-map`` (the h5 source); qualified ids of the form
    ``"<record-set>/<field>"`` can pull from any other RecordSet, e.g.
    ``"process-parameters/sheet_metal_thickness"``:

    - ``"name": "field_id"``                            — whole field-map field
    - ``"name": ("field_id", None | int | list[int])``   — explicit, with optional timestep slicing
    - ``"name": "process-parameters/<column>"``          — CSV column from the index

    Timestep slicing is only valid on field-map sources. The published
    manifest on DaRUS is untouched — only the in-memory representation
    grows. ``ds`` is mutated in place and returned for optional chaining.
    """
    jsonld = _load_jsonld_dict(ds.jsonld)
    jsonld.setdefault("recordSet", []).append(_build_record_set(jsonld, name, fields))
    rebuilt = mlc.Dataset(jsonld=jsonld, mapping=ds.mapping)
    ds.metadata = rebuilt.metadata
    ds.operations = rebuilt.operations
    # Persist the mutated dict so subsequent add_view calls accumulate
    # instead of clobbering each other.
    ds.jsonld = jsonld
    return ds

Module reference

Croissant manifest access via mlcroissant.

Single source of truth for field/column descriptions, dataset name, etc. — nothing is duplicated in the Python package.

resolve_source(source=None, data_dir=None, spec=DDACS_SPEC)

Return the source string (path or URL) that :func:load would use.

Resolution order

1. source if given (local path or HTTP(S) URL).
2. <data_dir>/metadata.json if it exists locally.
3. The permanent DaRUS URL :data:METADATA_URL.

Source code in ddacs/croissant.py

def resolve_source(
    source: str | Path | None = None,
    data_dir: str | Path | None = None,
    spec: DatasetSpec = DDACS_SPEC,
) -> str:
    """Return the source string (path or URL) that :func:`load` would use.

    Resolution order:
        1. ``source`` if given (local path or HTTP(S) URL).
        2. ``<data_dir>/metadata.json`` if it exists locally.
        3. The permanent DaRUS URL :data:`METADATA_URL`.
    """
    if source is not None:
        return str(source)
    if data_dir is not None:
        local = Path(data_dir) / spec.metadata_file
        if local.is_file():
            return str(local)
    return metadata_url(spec)

field_map(dataset)

Map each HDF5 field name in the field-map RecordSet to its mlc Field.

Source code in ddacs/croissant.py

def field_map(dataset: mlc.Dataset) -> dict[str, Any]:
    """Map each HDF5 field name in the field-map RecordSet to its mlc Field."""
    rs = _record_set(dataset, FIELD_MAP_RECORD_SET)
    if rs is None:
        return {}
    return {f.name: f for f in rs.fields}

process_parameters_descriptions(dataset)

Map each process_parameters column to its human-readable description.

Pulled directly from the process-parameters RecordSet in the manifest.

Source code in ddacs/croissant.py

def process_parameters_descriptions(dataset: mlc.Dataset) -> dict[str, str]:
    """Map each process_parameters column to its human-readable description.

    Pulled directly from the ``process-parameters`` RecordSet in the manifest.
    """
    rs = _record_set(dataset, "process-parameters")
    if rs is None:
        return {}
    return {f.name: (f.description or "") for f in rs.fields}

dataset_name(dataset)

Source code in ddacs/croissant.py

def dataset_name(dataset: mlc.Dataset) -> str:
    return dataset.metadata.name or ""