MatlabDataset¶
MatlabDataset loads and saves data to MATLAB files.
kedro_datasets.matlab.MatlabDataset ¶
MatlabDataset(
filepath,
save_args=None,
version=None,
credentials=None,
fs_args=None,
metadata=None,
)
Bases: AbstractVersionedDataset[ndarray, ndarray]
MatlabDataSet loads and saves data from/to a MATLAB file using scipy.io.
Warning
MAT v5 files can contain pickled Python objects.
scipy.io.loadmat() will deserialize these, which can execute
arbitrary code if the file is untrusted. Only load .mat files from
sources you trust.
Examples:
Using the YAML API:
cars:
type: matlab.MatlabDataset
filepath: gcs://your_bucket/cars.mat
fs_args:
project: my-project
credentials: my_gcp_credentials
Using the Python API:
>>> import numpy as np
>>> from kedro_datasets.matlab import MatlabDataset
>>>
>>> data = np.array([1, 2, 3])
>>>
>>> dataset = MatlabDataset(filepath=tmp_path / "test.mat")
>>> dataset.save(data)
>>> reloaded = dataset.load()
>>> assert (data == reloaded["data"]).all()
Parameters:
-
filepath(str | PathLike) –Filepath in POSIX format to a Matlab file prefixed with a protocol like
s3://. If prefix is not provided,fileprotocol (local filesystem) will be used. The prefix should be any protocol supported byfsspec. Note:http(s)doesn't support versioning. -
save_args(dict[str, Any] | None, default:None) –.mat options for saving .mat files.
-
version(Version | None, default:None) –If specified, should be an instance of
kedro.io.core.Version. If itsloadattribute is None, the latest version will be loaded. If itssaveattribute is None, save version will be autogenerated. -
credentials(dict[str, Any] | None, default:None) –Credentials required to get access to the underlying filesystem. E.g. for
GCSFileSystemit should look like{"token": None}. -
fs_args(dict[str, Any] | None, default:None) –Extra arguments to pass into underlying filesystem class constructor (e.g.
{"project": "my-project"}forGCSFileSystem), as well as to pass to the filesystem'sopenmethod through nested keysopen_args_loadandopen_args_save. Here you can find all available arguments foropen: https://filesystem-spec.readthedocs.io/en/latest/api.html#fsspec.spec.AbstractFileSystem.open All defaults are preserved, exceptmode, which is set towbwhen saving. -
metadata(dict[str, Any] | None, default:None) –Any arbitrary metadata. This is ignored by Kedro, but may be consumed by users or external plugins.
Source code in kedro_datasets/matlab/matlab_dataset.py
63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 | |
DEFAULT_FS_ARGS
class-attribute
instance-attribute
¶
DEFAULT_FS_ARGS = {'open_args_save': {'mode': 'wb'}}
load ¶
load()
Access the specific variable in the .mat file, e.g, data['variable_name']
Source code in kedro_datasets/matlab/matlab_dataset.py
133 134 135 136 137 138 139 140 | |
save ¶
save(data)
Source code in kedro_datasets/matlab/matlab_dataset.py
142 143 144 145 146 | |