kedro_datasets

Description

kedro_datasets is where you can find all of Kedro’s data connectors.

Classes

api.APIDataset(*, url[, method, load_args, ...])

APIDataset loads/saves data from/to HTTP(S) APIs.

biosequence.BioSequenceDataset(*, filepath)

BioSequenceDataset loads and saves data to a sequence file.

dask.CSVDataset(filepath[, load_args, ...])

CSVDataset loads and saves data to comma-separated value file(s).

dask.ParquetDataset(*, filepath[, ...])

ParquetDataset loads and saves data to parquet file(s).

databricks.ManagedTableDataset(*, table[, ...])

ManagedTableDataset loads and saves data into managed delta tables in Databricks.

email.EmailMessageDataset(*, filepath[, ...])

EmailMessageDataset loads/saves an email message from/to a file using an underlying filesystem (e.g.: local, S3, GCS).

geopandas.GenericDataset(*, filepath[, ...])

GenericDataset loads/saves data to a file using an underlying filesystem (eg: local, S3, GCS).

holoviews.HoloviewsWriter(*, filepath[, ...])

HoloviewsWriter saves Holoviews objects to image file(s) in an underlying filesystem (e.g. local, S3, GCS).

huggingface.HFDataset(*, dataset_name[, ...])

HFDataset loads Hugging Face datasets using the datasets library.

huggingface.HFTransformerPipelineDataset(*)

HFTransformerPipelineDataset loads pretrained Hugging Face transformers using the transformers library.

ibis.FileDataset(filepath[, file_format, ...])

FileDataset loads/saves data from/to a specified file format.

ibis.TableDataset(*, table_name[, database, ...])

TableDataset loads/saves data from/to Ibis table expressions.

json.JSONDataset(*, filepath[, save_args, ...])

JSONDataset loads/saves data from/to a JSON file using an underlying filesystem (e.g.: local, S3, GCS).

matlab.MatlabDataset(filepath[, save_args, ...])

MatlabDataSet loads and saves data from/to a MATLAB file using scipy.io.

matplotlib.MatplotlibDataset(*, filepath[, ...])

MatplotlibDataset saves one or more Matplotlib objects as image files to an underlying filesystem (e.g. local, S3, GCS).

matplotlib.MatplotlibWriter(**kwargs)

MatplotlibWriter saves one or more Matplotlib objects as image files to an underlying filesystem (e.g. local, S3, GCS).

networkx.GMLDataset(*, filepath[, ...])

GMLDataset loads and saves graphs to a GML file using an underlying filesystem (e.g.: local, S3, GCS).

networkx.GraphMLDataset(*, filepath[, ...])

GraphMLDataset loads and saves graphs to a GraphML file using an underlying filesystem (e.g.: local, S3, GCS).

networkx.JSONDataset(*, filepath[, ...])

NetworkX JSONDataset loads and saves graphs to a JSON file using an underlying filesystem (e.g.: local, S3, GCS).

pandas.CSVDataset(*, filepath[, load_args, ...])

CSVDataset loads/saves data from/to a CSV file using an underlying filesystem (e.g.: local, S3, GCS).

pandas.DeltaTableDataset(*[, filepath, ...])

DeltaTableDataset loads/saves delta tables from/to a filesystem (e.g.: local, S3, GCS), Databricks unity catalog and AWS Glue catalog respectively.

pandas.ExcelDataset(*, filepath[, engine, ...])

ExcelDataset loads/saves data from/to a Excel file using an underlying filesystem (e.g.: local, S3, GCS).

pandas.FeatherDataset(*, filepath[, ...])

FeatherDataset loads and saves data to a feather file using an underlying filesystem (e.g.: local, S3, GCS).

pandas.GBQQueryDataset([sql, project, ...])

GBQQueryDataset loads data from a provided SQL query from Google BigQuery.

pandas.GBQTableDataset(*, dataset, table_name)

GBQTableDataset loads and saves data from/to Google BigQuery.

pandas.GenericDataset(*, filepath, file_format)

pandas.GenericDataset loads/saves data from/to a data file using an underlying filesystem (e.g.: local, S3, GCS).

pandas.HDFDataset(*, filepath, key[, ...])

HDFDataset loads/saves data from/to a hdf file using an underlying filesystem (e.g. local, S3, GCS).

pandas.JSONDataset(*, filepath[, load_args, ...])

JSONDataset loads/saves data from/to a JSON file using an underlying filesystem (e.g.: local, S3, GCS).

pandas.ParquetDataset(*, filepath[, ...])

ParquetDataset loads/saves data from/to a Parquet file using an underlying filesystem (e.g.: local, S3, GCS).

pandas.SQLQueryDataset([sql, credentials, ...])

SQLQueryDataset loads data from a provided SQL query.

pandas.SQLTableDataset(*, table_name, ...[, ...])

SQLTableDataset loads data from a SQL table and saves a pandas dataframe to a table.

pandas.XMLDataset(*, filepath[, load_args, ...])

XMLDataset loads/saves data from/to a XML file using an underlying filesystem (e.g.: local, S3, GCS).

partitions.IncrementalDataset(*, path, dataset)

IncrementalDataset inherits from PartitionedDataset, which loads and saves partitioned file-like data using the underlying dataset definition.

partitions.PartitionedDataset(*, path, dataset)

PartitionedDataset loads and saves partitioned file-like data using the underlying dataset definition.

pickle.PickleDataset(*, filepath[, backend, ...])

PickleDataset loads/saves data from/to a Pickle file using an underlying filesystem (e.g.: local, S3, GCS).

pillow.ImageDataset(*, filepath[, ...])

ImageDataset loads/saves image data as numpy from an underlying filesystem (e.g.: local, S3, GCS).

plotly.HTMLDataset(*, filepath[, save_args, ...])

HTMLDataset saves a plotly figure to an HTML file using an underlying filesystem (e.g.: local, S3, GCS).

plotly.JSONDataset(*, filepath[, load_args, ...])

JSONDataset loads/saves a plotly figure from/to a JSON file using an underlying filesystem (e.g.: local, S3, GCS).

plotly.PlotlyDataset(*, filepath, plotly_args)

PlotlyDataset generates a plot from a pandas DataFrame and saves it to a JSON file using an underlying filesystem (e.g.: local, S3, GCS).

polars.CSVDataset(*, filepath[, load_args, ...])

CSVDataset loads/saves data from/to a CSV file using an underlying filesystem (e.g.: local, S3, GCS).

polars.EagerPolarsDataset(*, filepath, ...)

polars.EagerPolarsDataset loads/saves data from/to a data file using an underlying filesystem (e.g.: local, S3, GCS).

polars.LazyPolarsDataset(*, filepath, ...[, ...])

LazyPolarsDataset loads/saves data from/to a data file using an underlying filesystem (e.g.: local, S3, GCS).

redis.PickleDataset(*, key[, backend, ...])

PickleDataset loads/saves data from/to a Redis database.

snowflake.SnowparkTableDataset(*, table_name)

SnowparkTableDataset loads and saves Snowpark DataFrames.

spark.DeltaTableDataset(*, filepath[, metadata])

DeltaTableDataset loads data into DeltaTable objects.

spark.SparkDataset(*, filepath[, ...])

SparkDataset loads and saves Spark dataframes.

spark.SparkHiveDataset(*, database, table[, ...])

SparkHiveDataset loads and saves Spark dataframes stored on Hive.

spark.SparkJDBCDataset(*, url, table[, ...])

SparkJDBCDataset loads data from a database table accessible via JDBC URL url and connection properties and saves the content of a PySpark DataFrame to an external database table via JDBC.

spark.SparkStreamingDataset(*[, filepath, ...])

SparkStreamingDataset loads data to Spark Streaming Dataframe objects.

svmlight.SVMLightDataset(*, filepath[, ...])

SVMLightDataset loads/saves data from/to a svmlight/libsvm file using an underlying filesystem (e.g.: local, S3, GCS).

tensorflow.TensorFlowModelDataset(*, filepath)

TensorFlowModelDataset loads and saves TensorFlow models.

text.TextDataset(*, filepath[, version, ...])

TextDataset loads/saves data from/to a text file using an underlying filesystem (e.g.: local, S3, GCS)

yaml.YAMLDataset(*, filepath[, save_args, ...])

YAMLDataset loads/saves data from/to a YAML file using an underlying filesystem (e.g.: local, S3, GCS).