kedro_datasets

Description

kedro_datasets is where you can find all of Kedro’s data connectors.

Classes

kedro_datasets.api.APIDataSet(url[, method, ...])

APIDataSet loads/saves data from/to HTTP(S) APIs.

kedro_datasets.biosequence.BioSequenceDataSet(...)

BioSequenceDataSet loads and saves data to a sequence file.

kedro_datasets.dask.ParquetDataSet(filepath)

ParquetDataSet loads and saves data to parquet file(s).

kedro_datasets.databricks.ManagedTableDataSet(table)

ManagedTableDataSet loads and saves data into managed delta tables on Databricks.

kedro_datasets.email.EmailMessageDataSet(...)

EmailMessageDataSet loads/saves an email message from/to a file using an underlying filesystem (e.g.: local, S3, GCS).

kedro_datasets.geopandas.GeoJSONDataSet(filepath)

GeoJSONDataSet loads/saves data to a GeoJSON file using an underlying filesystem (eg: local, S3, GCS).

kedro_datasets.holoviews.HoloviewsWriter(...)

HoloviewsWriter saves Holoviews objects to image file(s) in an underlying filesystem (e.g.

kedro_datasets.json.JSONDataSet(filepath[, ...])

JSONDataSet loads/saves data from/to a JSON file using an underlying filesystem (e.g.: local, S3, GCS).

kedro_datasets.matplotlib.MatplotlibWriter(...)

MatplotlibWriter saves one or more Matplotlib objects as image files to an underlying filesystem (e.g.

kedro_datasets.networkx.GMLDataSet(filepath)

GMLDataSet loads and saves graphs to a GML file using an underlying filesystem (e.g.: local, S3, GCS).

kedro_datasets.networkx.GraphMLDataSet(filepath)

GraphMLDataSet loads and saves graphs to a GraphML file using an underlying filesystem (e.g.: local, S3, GCS).

kedro_datasets.networkx.JSONDataSet(filepath)

NetworkX JSONDataSet loads and saves graphs to a JSON file using an underlying filesystem (e.g.: local, S3, GCS).

kedro_datasets.pandas.CSVDataSet(filepath[, ...])

CSVDataSet loads/saves data from/to a CSV file using an underlying filesystem (e.g.: local, S3, GCS).

kedro_datasets.pandas.DeltaTableDataSet([...])

DeltaTableDataSet loads/saves delta tables from/to a filesystem (e.g.: local, S3, GCS), Databricks unity catalog and AWS Glue catalog respectively.

kedro_datasets.pandas.ExcelDataSet(filepath)

ExcelDataSet loads/saves data from/to a Excel file using an underlying filesystem (e.g.: local, S3, GCS).

kedro_datasets.pandas.FeatherDataSet(filepath)

FeatherDataSet loads and saves data to a feather file using an underlying filesystem (e.g.: local, S3, GCS).

kedro_datasets.pandas.GBQQueryDataSet([sql, ...])

GBQQueryDataSet loads data from a provided SQL query from Google BigQuery.

kedro_datasets.pandas.GBQTableDataSet(...[, ...])

GBQTableDataSet loads and saves data from/to Google BigQuery.

kedro_datasets.pandas.GenericDataSet(...[, ...])

pandas.GenericDataSet loads/saves data from/to a data file using an underlying filesystem (e.g.: local, S3, GCS).

kedro_datasets.pandas.HDFDataSet(filepath, key)

HDFDataSet loads/saves data from/to a hdf file using an underlying filesystem (e.g.

kedro_datasets.pandas.JSONDataSet(filepath)

JSONDataSet loads/saves data from/to a JSON file using an underlying filesystem (e.g.: local, S3, GCS).

kedro_datasets.pandas.ParquetDataSet(filepath)

ParquetDataSet loads/saves data from/to a Parquet file using an underlying filesystem (e.g.: local, S3, GCS).

kedro_datasets.pandas.SQLQueryDataSet([sql, ...])

SQLQueryDataSet loads data from a provided SQL query.

kedro_datasets.pandas.SQLTableDataSet(...[, ...])

SQLTableDataSet loads data from a SQL table and saves a pandas dataframe to a table.

kedro_datasets.pandas.XMLDataSet(filepath[, ...])

XMLDataSet loads/saves data from/to a XML file using an underlying filesystem (e.g.: local, S3, GCS).

kedro_datasets.pickle.PickleDataSet(filepath)

PickleDataSet loads/saves data from/to a Pickle file using an underlying filesystem (e.g.: local, S3, GCS).

kedro_datasets.pillow.ImageDataSet(filepath)

ImageDataSet loads/saves image data as numpy from an underlying filesystem (e.g.: local, S3, GCS).

kedro_datasets.plotly.JSONDataSet(filepath)

JSONDataSet loads/saves a plotly figure from/to a JSON file using an underlying filesystem (e.g.: local, S3, GCS).

kedro_datasets.plotly.PlotlyDataSet(...[, ...])

PlotlyDataSet generates a plot from a pandas DataFrame and saves it to a JSON file using an underlying filesystem (e.g.: local, S3, GCS).

kedro_datasets.polars.CSVDataSet(filepath[, ...])

CSVDataSet loads/saves data from/to a CSV file using an underlying filesystem (e.g.: local, S3, GCS).

kedro_datasets.redis.PickleDataSet(key[, ...])

PickleDataSet loads/saves data from/to a Redis database.

kedro_datasets.snowflake.SnowparkTableDataSet(...)

SnowparkTableDataSet loads and saves Snowpark dataframes.

kedro_datasets.spark.DeltaTableDataSet(filepath)

DeltaTableDataSet loads data into DeltaTable objects.

kedro_datasets.spark.SparkDataSet(filepath)

SparkDataSet loads and saves Spark dataframes.

kedro_datasets.spark.SparkHiveDataSet(...[, ...])

SparkHiveDataSet loads and saves Spark dataframes stored on Hive.

kedro_datasets.spark.SparkJDBCDataSet(url, table)

SparkJDBCDataSet loads data from a database table accessible via JDBC URL url and connection properties and saves the content of a PySpark DataFrame to an external database table via JDBC.

kedro_datasets.spark.SparkStreamingDataSet([...])

SparkStreamingDataSet loads data to Spark Streaming Dataframe objects.

kedro_datasets.svmlight.SVMLightDataSet(filepath)

SVMLightDataSet loads/saves data from/to a svmlight/libsvm file using an underlying filesystem (e.g.: local, S3, GCS).

kedro_datasets.tensorflow.TensorFlowModelDataSet(...)

TensorFlowModelDataSet loads and saves TensorFlow models.

kedro_datasets.text.TextDataSet(filepath[, ...])

TextDataSet loads/saves data from/to a text file using an underlying filesystem (e.g.: local, S3, GCS)

kedro_datasets.tracking.JSONDataSet(filepath)

JSONDataSet saves data to a JSON file using an underlying filesystem (e.g.: local, S3, GCS).

kedro_datasets.tracking.MetricsDataSet(filepath)

MetricsDataSet saves data to a JSON file using an underlying filesystem (e.g.: local, S3, GCS).

kedro_datasets.video.VideoDataSet(filepath)

VideoDataSet loads / save video data from a given filepath as sequence of PIL.Image.Image using OpenCV.

kedro_datasets.yaml.YAMLDataSet(filepath[, ...])

YAMLDataSet loads/saves data from/to a YAML file using an underlying filesystem (e.g.: local, S3, GCS).