mlrun.MLRunDataframeDataset

kedro_datasets_experimental.mlrun.MLRunDataframeDataset

MLRunDataframeDataset(
    key=None, load_args=None, save_args=None
)

Bases: MLRunAbstractDataset

Dataset for saving/loading pandas DataFrames via MLRun.

Uses MLRun's log_artifact and get_artifact.

load_args and save_args accept any arguments supported by the corresponding MLRun API for your MLRun version; see the MLRun documentation.

Examples:

Using the YAML API:

user_data:
  type: kedro_datasets_experimental.mlrun.MLRunDataframeDataset
  key: generate-data-main_user_data

Using the Python API:

from kedro_datasets_experimental.mlrun import MLRunDataframeDataset
import pandas as pd

dataset = MLRunDataframeDataset(key="processed_df")
dataset.save(pd.DataFrame({"a": [1, 2], "b": [3, 4]}))
loaded_df = dataset.load()

Parameters:

• key (str | None, default: None ) –

  Artifact key for MLRun (defaults to catalog dataset name).

• load_args (dict[str, Any] | None, default: None ) –

  Passed to MLRun when loading; see MLRun docs for your version.

• save_args (dict[str, Any] | None, default: None ) –

  Passed to log_artifact; see MLRun docs for your version.

Source code in kedro_datasets_experimental/mlrun/df_dataset.py

def __init__(
    self,
    key: str | None = None,
    load_args: dict[str, Any] | None = None,
    save_args: dict[str, Any] | None = None,
) -> None:
    super().__init__(key=key, save_args=save_args, load_args=load_args)

load

load()

Source code in kedro_datasets_experimental/mlrun/df_dataset.py

def load(self):
    artifact = super().load()
    if not artifact:
        return None
    return artifact.to_dataitem().as_df()