mlrun.MLRunAbstractDataset

kedro_datasets_experimental.mlrun.MLRunAbstractDataset

MLRunAbstractDataset(
    key=None, load_args=None, save_args=None
)

Bases: AbstractDataset

Base class for MLRun datasets; use for generic artifacts (any serializable data).

Uses MLRun's log_artifact and get_artifact.

load_args and save_args accept any arguments supported by the corresponding MLRun API for your MLRun version; see the MLRun documentation.

Examples

Using the YAML API:

generic_artifact:
  type: kedro_datasets_experimental.mlrun.MLRunAbstractDataset
  key: my_artifact

Using the Python API:

from kedro_datasets_experimental.mlrun import MLRunAbstractDataset

dataset = MLRunAbstractDataset(key="config_data")
dataset.save({"param1": "value1", "param2": 42})
loaded = dataset.load()

Parameters:

• key (str | None, default: None ) –

  Artifact key for MLRun (defaults to catalog dataset name).

• load_args (dict[str, Any] | None, default: None ) –

  Passed to MLRun when loading; see MLRun docs for your version.

• save_args (dict[str, Any] | None, default: None ) –

  Passed to log_artifact; see MLRun docs for your version.

Source code in kedro_datasets_experimental/mlrun/abstract_artifact.py

def __init__(
    self,
    key: str | None = None,
    load_args: dict[str, Any] | None = None,
    save_args: dict[str, Any] | None = None,
) -> None:
    self._ctx_manager = MLRunContextManager()
    # None when not given; resolved by key property at save/load time
    self._key = key
    self._load_args = {**self.DEFAULT_LOAD_ARGS, **(load_args or {})}
    self._save_args = {**self.DEFAULT_SAVE_ARGS, **(save_args or {})}

DEFAULT_LOAD_ARGS class-attribute instance-attribute

DEFAULT_LOAD_ARGS = {}

DEFAULT_SAVE_ARGS class-attribute instance-attribute

DEFAULT_SAVE_ARGS = {}

_ctx_manager instance-attribute

_ctx_manager = MLRunContextManager()

_key instance-attribute

_key = key

_load_args instance-attribute

_load_args = {
    None: DEFAULT_LOAD_ARGS,
    None: load_args or {},
}

_save_args instance-attribute

_save_args = {
    None: DEFAULT_SAVE_ARGS,
    None: save_args or {},
}

key property

key

_describe

_describe()

Source code in kedro_datasets_experimental/mlrun/abstract_artifact.py

def _describe(self) -> dict[str, Any]:
    return {
        "key": self.key,
        "load_args": self._load_args,
        "save_args": self._save_args,
        "mlrun_project_name": self._ctx_manager.project.name,
    }

load

load()

Source code in kedro_datasets_experimental/mlrun/abstract_artifact.py

def load(self) -> mlrun.artifacts.Artifact | None:
    return self._ctx_manager.context.get_artifact(
        **{**self._load_args, "key": self.key}
    )

save

save(data)

Source code in kedro_datasets_experimental/mlrun/abstract_artifact.py

def save(self, data: Any) -> None:
    artifact = mlrun.artifacts.Artifact(key=self.key, body=data)
    self._ctx_manager.context.log_artifact(
        item=artifact, **{**self._save_args, "db_key": self.key}
    )