kedro.config.ConfigLoader¶
- class kedro.config.ConfigLoader(conf_source, env=None, runtime_params=None, config_patterns=None, *, base_env='base', default_run_env='local')[source]¶
Recursively scan directories (config paths) contained in
conf_source
for configuration files with ayaml
,yml
,json
,ini
,pickle
,xml
orproperties
extension, load them, and return them in the form of a config dictionary.The first processed config path is the
base
directory insideconf_source
. The optionalenv
argument can be used to specify a subdirectory ofconf_source
to process as a config path afterbase
.When the same top-level key appears in any 2 config files located in the same (sub)directory, a
ValueError
is raised.When the same key appears in any 2 config files located in different (sub)directories, the last processed config path takes precedence and overrides this key.
For example, if your
conf_source
looks like this:. `-- conf |-- README.md |-- base | |-- catalog.yml | |-- logging.yml | `-- experiment1 | `-- parameters.yml `-- local |-- catalog.yml |-- db.ini |-- experiment1 | |-- parameters.yml | `-- model_parameters.yml `-- experiment2 `-- parameters.yml
You can access the different configurations as follows:
import logging.config from kedro.config import ConfigLoader from kedro.framework.project import settings conf_path = str(project_path / settings.CONF_SOURCE) conf_loader = ConfigLoader(conf_source=conf_path, env="local") conf_logging = conf_loader["logging"] logging.config.dictConfig(conf_logging) # set logging conf conf_catalog = conf_loader["catalog"] conf_params = conf_loader["parameters"]
Attributes
Property method to return deduplicated configuration paths.
Methods
clear
()copy
()fromkeys
(iterable[, value])get
(*patterns)Retrieve the corresponding layout by the string key.
items
()keys
()pop
(k[,d])If key is not found, d is returned if given, otherwise KeyError is raised.
popitem
()as a 2-tuple; but raise KeyError if D is empty.
setdefault
(k[,d])update
([E, ]**F)If E present and has a .keys() method, does: for k in E: D[k] = E[k] If E present and lacks .keys() method, does: for (k, v) in E: D[k] = v In either case, this is followed by: for k, v in F.items(): D[k] = v
values
()- __init__(conf_source, env=None, runtime_params=None, config_patterns=None, *, base_env='base', default_run_env='local')[source]¶
Instantiates a
ConfigLoader
.- Parameters
conf_source – Path to use as root directory for loading configuration.
env – Environment that will take precedence over base.
runtime_params – Extra parameters passed to a Kedro run.
config_patterns – Regex patterns that specify the naming convention for configuration files so they can be loaded. Can be customised by supplying config_patterns as in CONFIG_LOADER_ARGS in settings.py.
base_env – Name of the base environment. Defaults to “base”. This is used in the conf_paths property method to construct the configuration paths.
default_run_env – Name of the default run environment. Defaults to “local”. This is used in the conf_paths property method to construct the configuration paths. Can be overriden by supplying the env argument.
- clear() None. Remove all items from D. ¶
- property conf_paths¶
Property method to return deduplicated configuration paths.
- copy()¶
- classmethod fromkeys(iterable, value=None)¶
- get(*patterns)[source]¶
Retrieve the corresponding layout by the string key.
When there isn’t an exact match, all the existing keys in the layout map will be treated as a regex and map against the input key again. The first match will be returned, based on the key insertion order. Return None if there isn’t any match found.
- Parameters
key – the string key as the query for the layout.
- Returns
Corresponding layout based on the query.
- items() a set-like object providing a view on D's items ¶
- keys() a set-like object providing a view on D's keys ¶
- pop(k[, d]) v, remove specified key and return the corresponding value. ¶
If key is not found, d is returned if given, otherwise KeyError is raised.
- popitem() (k, v), remove and return some (key, value) pair ¶
as a 2-tuple; but raise KeyError if D is empty.
- setdefault(k[, d]) D.get(k,d), also set D[k]=d if k not in D ¶
- update([E, ]**F) None. Update D from mapping/iterable E and F. ¶
If E present and has a .keys() method, does: for k in E: D[k] = E[k] If E present and lacks .keys() method, does: for (k, v) in E: D[k] = v In either case, this is followed by: for k, v in F.items(): D[k] = v
- values() an object providing a view on D's values ¶