kedro_datasets

Description

kedro_datasets is where you can find all of Kedro’s data connectors.

Classes

kedro_datasets.api.APIDataset(*, url[, ...])

APIDataset loads/saves data from/to HTTP(S) APIs.

kedro_datasets.biosequence.BioSequenceDataset(*, ...)

BioSequenceDataset loads and saves data to a sequence file.

kedro_datasets.dask.ParquetDataset(*, filepath)

ParquetDataset loads and saves data to parquet file(s).

kedro_datasets.databricks.ManagedTableDataset(*, ...)

ManagedTableDataset loads and saves data into managed delta tables on Databricks.

kedro_datasets.email.EmailMessageDataset(*, ...)

EmailMessageDataset loads/saves an email message from/to a file using an underlying filesystem (e.g.: local, S3, GCS).

kedro_datasets.geopandas.GeoJSONDataset(*, ...)

GeoJSONDataset loads/saves data to a GeoJSON file using an underlying filesystem (eg: local, S3, GCS).

kedro_datasets.holoviews.HoloviewsWriter(*, ...)

HoloviewsWriter saves Holoviews objects to image file(s) in an underlying filesystem (e.g. local, S3, GCS).

kedro_datasets.huggingface.HFDataset(*, ...)

HFDataset loads Hugging Face datasets using the datasets library.

kedro_datasets.huggingface.HFTransformerPipelineDataset(*)

HFTransformerPipelineDataset loads pretrained Hugging Face transformers using the transformers library.

kedro_datasets.json.JSONDataset(*, filepath)

JSONDataset loads/saves data from/to a JSON file using an underlying filesystem (e.g.: local, S3, GCS).

kedro_datasets.matlab.MatlabDataset(filepath)

MatlabDataSet loads and saves data from/to a MATLAB file using scipy.io.

kedro_datasets.matplotlib.MatplotlibWriter(*, ...)

MatplotlibWriter saves one or more Matplotlib objects as image files to an underlying filesystem (e.g. local, S3, GCS).

kedro_datasets.netcdf.NetCDFDataset(*, filepath)

NetCDFDataset loads/saves data from/to a NetCDF file using an underlying filesystem (e.g.: local, S3, GCS).

kedro_datasets.networkx.GMLDataset(*, filepath)

GMLDataset loads and saves graphs to a GML file using an underlying filesystem (e.g.: local, S3, GCS).

kedro_datasets.networkx.GraphMLDataset(*, ...)

GraphMLDataset loads and saves graphs to a GraphML file using an underlying filesystem (e.g.: local, S3, GCS).

kedro_datasets.networkx.JSONDataset(*, filepath)

NetworkX JSONDataset loads and saves graphs to a JSON file using an underlying filesystem (e.g.: local, S3, GCS).

kedro_datasets.pandas.CSVDataset(*, filepath)

CSVDataset loads/saves data from/to a CSV file using an underlying filesystem (e.g.: local, S3, GCS).

kedro_datasets.pandas.DeltaTableDataset(*[, ...])

DeltaTableDataset loads/saves delta tables from/to a filesystem (e.g.: local, S3, GCS), Databricks unity catalog and AWS Glue catalog respectively.

kedro_datasets.pandas.ExcelDataset(*, filepath)

ExcelDataset loads/saves data from/to a Excel file using an underlying filesystem (e.g.: local, S3, GCS).

kedro_datasets.pandas.FeatherDataset(*, filepath)

FeatherDataset loads and saves data to a feather file using an underlying filesystem (e.g.: local, S3, GCS).

kedro_datasets.pandas.GBQQueryDataset([sql, ...])

GBQQueryDataset loads data from a provided SQL query from Google BigQuery.

kedro_datasets.pandas.GBQTableDataset(*, ...)

GBQTableDataset loads and saves data from/to Google BigQuery.

kedro_datasets.pandas.GenericDataset(*, ...)

pandas.GenericDataset loads/saves data from/to a data file using an underlying filesystem (e.g.: local, S3, GCS).

kedro_datasets.pandas.HDFDataset(*, ...[, ...])

HDFDataset loads/saves data from/to a hdf file using an underlying filesystem (e.g. local, S3, GCS).

kedro_datasets.pandas.JSONDataset(*, filepath)

JSONDataset loads/saves data from/to a JSON file using an underlying filesystem (e.g.: local, S3, GCS).

kedro_datasets.pandas.ParquetDataset(*, filepath)

ParquetDataset loads/saves data from/to a Parquet file using an underlying filesystem (e.g.: local, S3, GCS).

kedro_datasets.pandas.SQLQueryDataset([sql, ...])

SQLQueryDataset loads data from a provided SQL query.

kedro_datasets.pandas.SQLTableDataset(*, ...)

SQLTableDataset loads data from a SQL table and saves a pandas dataframe to a table.

kedro_datasets.pandas.XMLDataset(*, filepath)

XMLDataset loads/saves data from/to a XML file using an underlying filesystem (e.g.: local, S3, GCS).

kedro_datasets.partitions.IncrementalDataset(*, ...)

IncrementalDataset inherits from PartitionedDataset, which loads and saves partitioned file-like data using the underlying dataset definition.

kedro_datasets.partitions.PartitionedDataset(*, ...)

PartitionedDataset loads and saves partitioned file-like data using the underlying dataset definition.

kedro_datasets.pickle.PickleDataset(*, filepath)

PickleDataset loads/saves data from/to a Pickle file using an underlying filesystem (e.g.: local, S3, GCS).

kedro_datasets.pillow.ImageDataset(*, filepath)

ImageDataset loads/saves image data as numpy from an underlying filesystem (e.g.: local, S3, GCS).

kedro_datasets.plotly.JSONDataset(*, filepath)

JSONDataset loads/saves a plotly figure from/to a JSON file using an underlying filesystem (e.g.: local, S3, GCS).

kedro_datasets.plotly.PlotlyDataset(*, ...)

PlotlyDataset generates a plot from a pandas DataFrame and saves it to a JSON file using an underlying filesystem (e.g.: local, S3, GCS).

kedro_datasets.polars.CSVDataset(*, filepath)

CSVDataset loads/saves data from/to a CSV file using an underlying filesystem (e.g.: local, S3, GCS).

kedro_datasets.polars.EagerPolarsDataset(*, ...)

polars.EagerPolarsDataset loads/saves data from/to a data file using an underlying filesystem (e.g.: local, S3, GCS).

kedro_datasets.polars.LazyPolarsDataset(*, ...)

LazyPolarsDataset loads/saves data from/to a data file using an underlying filesystem (e.g.: local, S3, GCS).

kedro_datasets.redis.PickleDataset(*, key[, ...])

PickleDataset loads/saves data from/to a Redis database.

kedro_datasets.snowflake.SnowparkTableDataset(*, ...)

SnowparkTableDataset loads and saves Snowpark dataframes.

kedro_datasets.spark.DeltaTableDataset(*, ...)

DeltaTableDataset loads data into DeltaTable objects.

kedro_datasets.spark.SparkDataset(*, filepath)

SparkDataset loads and saves Spark dataframes.

kedro_datasets.spark.SparkHiveDataset(*, ...)

SparkHiveDataset loads and saves Spark dataframes stored on Hive.

kedro_datasets.spark.SparkJDBCDataset(*, ...)

SparkJDBCDataset loads data from a database table accessible via JDBC URL url and connection properties and saves the content of a PySpark DataFrame to an external database table via JDBC.

kedro_datasets.spark.SparkStreamingDataset(*)

SparkStreamingDataset loads data to Spark Streaming Dataframe objects.

kedro_datasets.svmlight.SVMLightDataset(*, ...)

SVMLightDataset loads/saves data from/to a svmlight/libsvm file using an underlying filesystem (e.g.: local, S3, GCS).

kedro_datasets.tensorflow.TensorFlowModelDataset(*, ...)

TensorFlowModelDataset loads and saves TensorFlow models.

kedro_datasets.text.TextDataset(*, filepath)

TextDataset loads/saves data from/to a text file using an underlying filesystem (e.g.: local, S3, GCS)

kedro_datasets.tracking.JSONDataset(*, filepath)

JSONDataset saves data to a JSON file using an underlying filesystem (e.g.: local, S3, GCS).

kedro_datasets.tracking.MetricsDataset(*, ...)

MetricsDataset saves data to a JSON file using an underlying filesystem (e.g.: local, S3, GCS).

kedro_datasets.video.VideoDataset(*, filepath)

VideoDataset loads / save video data from a given filepath as sequence of PIL.Image.Image using OpenCV.

kedro_datasets.yaml.YAMLDataset(*, filepath)

YAMLDataset loads/saves data from/to a YAML file using an underlying filesystem (e.g.: local, S3, GCS).