Alexis Drakopoulos
08/30/2024, 2:20 PMValueError: Pipeline input(s) {'params:features', 'params:sources', 'params:general', 'params:feature_extraction'} not found in the DataCatalog
Elena Khaustova
08/30/2024, 2:35 PMAlexis Drakopoulos
08/30/2024, 3:06 PMAlexis Drakopoulos
08/30/2024, 3:07 PMParameters not found in your Kedro project config.
No files of YAML or JSON format found in
/dbfs/Shared/<REMOVED>/conf or
/dbfs/Shared/<REMOVED>/conf/da
tabricks matching the glob pattern(s): ['parameters*',
'parameters*/**']
ah it's not finding the params in confAlexis Drakopoulos
08/30/2024, 3:08 PMAlexis Drakopoulos
08/30/2024, 3:08 PMfrom kedro.config import OmegaConfigLoader # noqa: E402
CONFIG_LOADER_CLASS = OmegaConfigLoader
CONFIG_LOADER_ARGS = {
"base_env": "base",
"default_run_env": "local",
"config_patterns": {
"spark" : ["spark*/"],
"parameters": ["parameters*", "parameters*/**", "**/parameters*"],
}
}
looks fine to me I'll check if we're doing something weird in the hooks maybeElena Khaustova
08/30/2024, 3:17 PMkedro ipython
parameters = context.config_loader.get("parameters")
parameters
Elena Khaustova
08/30/2024, 3:18 PMElena Khaustova
08/30/2024, 3:18 PMkedro ipython
catalog
Nok Lam Chan
08/30/2024, 4:03 PM/dbfs/Shared/<REMOVED>/conf
Is this where your project live in? if you are on databricks, it's most likely you need to use CONF_SOURCE
to customise where you put your configuration.