Skip to content

langchain.PromptDataset

kedro_datasets_experimental.langchain.PromptDataset

PromptDataset(
    filepath,
    template="PromptTemplate",
    dataset=None,
    credentials=None,
    fs_args=None,
    metadata=None,
    **kwargs
)

Bases: AbstractDataset[Union[PromptTemplate, ChatPromptTemplate], Any]

A Kedro dataset for loading LangChain prompt templates from text, JSON, or YAML files.

This dataset wraps existing Kedro datasets (such as TextDataset, JSONDataset, or YAMLDataset) to load prompt configurations and convert them into LangChain PromptTemplate or ChatPromptTemplate objects.

Example usage for the YAML API:
my_prompt:
    type: kedro_datasets_experimental.langchain.PromptDataset
    filepath: data/prompts/my_prompt.json
    template: PromptTemplate
    dataset:
        type: json.JSONDataset
        fs_args:
            load_args:
                encoding: utf-8
            save_args:
                ensure_ascii: false
    credentials: dev_creds
    metadata:
        kedro-viz:
            layer: raw
Example usage for the Python API:
from kedro_datasets_experimental.langchain import PromptDataset

dataset = PromptDataset(
    filepath="data/prompts/my_prompt.json",
    template="PromptTemplate",
    dataset={"type": "json.JSONDataset"},
)
prompt = dataset.load()
print(prompt.format(name="Kedro"))

Parameters:

  • filepath (str | PathLike) –

    Path to the prompt file

  • template (str, default: 'PromptTemplate' ) –

    Name of the LangChain template class ("PromptTemplate" or "ChatPromptTemplate")

  • dataset (dict[str, Any] | str | None, default: None ) –

    Configuration for the underlying Kedro dataset

  • credentials (dict[str, Any] | None, default: None ) –

    Credentials passed to the underlying dataset unless already defined

  • fs_args (dict[str, Any] | None, default: None ) –

    Extra arguments passed to the filesystem, if supported

  • metadata (dict[str, Any] | None, default: None ) –

    Arbitrary metadata

  • **kwargs (Any, default: {} ) –

    Additional arguments (ignored)

Source code in kedro_datasets_experimental/langchain/prompt_dataset.py
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
def __init__(  # noqa: PLR0913
    self,
    filepath: str | os.PathLike,
    template: str = "PromptTemplate",
    dataset: dict[str, Any] | str | None = None,
    credentials: dict[str, Any] | None = None,
    fs_args: dict[str, Any] | None = None,
    metadata: dict[str, Any] | None = None,
    **kwargs: Any,
):
    """
    Initialize the LangChain prompt dataset.

    Args:
        filepath: Path to the prompt file
        template: Name of the LangChain template class ("PromptTemplate" or "ChatPromptTemplate")
        dataset: Configuration for the underlying Kedro dataset
        credentials: Credentials passed to the underlying dataset unless already defined
        fs_args: Extra arguments passed to the filesystem, if supported
        metadata: Arbitrary metadata
        **kwargs: Additional arguments (ignored)
    """
    super().__init__()

    self.metadata = metadata
    self._filepath = get_filepath_str(Path(filepath), kwargs.get("protocol"))

    try:
        self._template_name = template
        self._create_template_function = getattr(self, self.TEMPLATES[template])
    except KeyError:
        raise DatasetError(
            f"Invalid template '{template}'. Must be one of: {list(self.TEMPLATES)}"
        )

    # Infer dataset type if not explicitly provided
    dataset_config = self._build_dataset_config(dataset)

    # Handle credentials
    self._credentials = deepcopy(credentials or {})
    self._fs_args = deepcopy(fs_args or {})

    if self._credentials:
        if CREDENTIALS_KEY in dataset_config:
            self._logger.warning(
                "Top-level credentials will not propagate into the underlying dataset "
                "since credentials were explicitly defined in the dataset config."
            )
        else:
            dataset_config[CREDENTIALS_KEY] = deepcopy(self._credentials)

    if self._fs_args:
        if "fs_args" in dataset_config:
            self._logger.warning(
                "Top-level fs_args will not propagate into the underlying dataset "
                "since fs_args were explicitly defined in the dataset config."
            )
        else:
            dataset_config["fs_args"] = deepcopy(self._fs_args)

    try:
        dataset_class, dataset_kwargs = parse_dataset_definition(dataset_config)
        self._dataset = dataset_class(**dataset_kwargs)
    except Exception as e:
        raise DatasetError(f"Failed to create underlying dataset: {e}")

TEMPLATES class-attribute instance-attribute

TEMPLATES = {
    "PromptTemplate": "_create_prompt_template",
    "ChatPromptTemplate": "_create_chat_prompt_template",
}

VALID_DATASETS class-attribute instance-attribute

VALID_DATASETS = {
    "text.TextDataset",
    "json.JSONDataset",
    "yaml.YAMLDataset",
}

metadata instance-attribute

metadata = metadata

load

load()

Loads the underlying dataset and converts the data into a LangChain prompt template.

This method retrieves raw prompt data from the underlying dataset (e.g., a JSON or YAML file) and constructs the corresponding LangChain template — either a PromptTemplate or ChatPromptTemplate — depending on the dataset configuration.

Raises:

  • DatasetError –

    If the dataset cannot be loaded, contains no data, or cannot be converted into the expected prompt template.

Returns:

  • PromptTemplate | ChatPromptTemplate –

    A fully initialized LangChain prompt object created from the dataset contents.

Example

dataset.load() ChatPromptTemplate.from_messages([ ("system", "You are a helpful assistant."), ("human", "{input}") ])

Source code in kedro_datasets_experimental/langchain/prompt_dataset.py
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
def load(self) -> PromptTemplate | ChatPromptTemplate:
    """
    Loads the underlying dataset and converts the data into a LangChain prompt template.

    This method retrieves raw prompt data from the underlying dataset (e.g., a JSON or YAML file)
    and constructs the corresponding LangChain template — either a `PromptTemplate` or
    `ChatPromptTemplate` — depending on the dataset configuration.

    Raises:
        DatasetError: If the dataset cannot be loaded, contains no data, or cannot be
            converted into the expected prompt template.

    Returns:
        PromptTemplate | ChatPromptTemplate:
            A fully initialized LangChain prompt object created from the dataset contents.

    Example:
        >>> dataset.load()
        ChatPromptTemplate.from_messages([
            ("system", "You are a helpful assistant."),
            ("human", "{input}")
        ])
    """
    try:
        raw_data = self._dataset.load()
    except Exception as e:
        raise DatasetError(f"Failed to load data from {self._filepath}: {e}")

    if raw_data is None:
        raise DatasetError(f"No data loaded from {self._filepath}")

    try:
        return self._create_template_function(raw_data)
    except Exception as e:
        raise DatasetError(f"Failed to create {self._template_name}: {e}")

preview

preview()

Generate a JSON-compatible preview of the underlying prompt data for Kedro-Viz.

Returns:

  • JSONPreview –

    A Kedro-Viz-compatible object containing a serialized JSON string of the processed data. If an exception occurs during processing, the returned JSONPreview contains an error message instead of the dataset content.

Example: >>> dataset.preview() JSONPreview('{"messages": [{"role": "system", "content": "You are..."}]}')

Source code in kedro_datasets_experimental/langchain/prompt_dataset.py
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
def preview(self) -> JSONPreview:
    """
    Generate a JSON-compatible preview of the underlying prompt data for Kedro-Viz.

    Returns:
        JSONPreview:
            A Kedro-Viz-compatible object containing a serialized JSON string of the
            processed data. If an exception occurs during processing, the returned
            JSONPreview contains an error message instead of the dataset content.
    Example:
        >>> dataset.preview()
        JSONPreview('{"messages": [{"role": "system", "content": "You are..."}]}')
    """
    try:
        data = self._dataset.load()

        if isinstance(data, str):
            # Wrap plain text in a dictionary or Viz doesn't render it
            data = {"text": data}

        return JSONPreview(json.dumps(data))

    except Exception as e:
        return JSONPreview(f"Error generating preview: {e}")

save

save(data)
Source code in kedro_datasets_experimental/langchain/prompt_dataset.py
297
298
def save(self, data: Any) -> None:
    raise DatasetError("Saving is not supported for PromptDataset")