Jonas Kemper
03/15/2023, 12:03 PMdata_science:
active_modelling_pipeline:
model_options:
test_size: 0.2...
and I load it via
conf_loader = kedro.config.ConfigLoader(".")
parameters = conf_loader['parameters']
that returns me
{'data_science': {'active_modelling_pipeline': {'model_options': {'test_size': 0.2,
. When in another place, I
data_catalog = DataCatalog.from_config(catalog, credentials)
data_catalog.add_feed_dict(parameters)
this won't work, because eventually that'll land me
ValueError: Pipeline input(s) {'params:data_science.candidate_modelling_pipeline.model_options.random_state', ...} not found in the DataCatalog
What's the intermediate step that I'm missing?Merel
03/15/2023, 12:09 PMJonas Kemper
03/15/2023, 12:10 PMMerel
03/15/2023, 12:13 PMdata_catalog = DataCatalog.from_config(catalog, credentials)
data_catalog.add_feed_dict(parameters)
Jonas Kemper
03/15/2023, 12:14 PMrunner.run(pipeline, data_catalog)
somehow, don't I?pipeline = register_pipelines()[pipeline_name]
)Merel
03/15/2023, 12:18 PMJonas Kemper
03/15/2023, 12:18 PMNok Lam Chan
03/15/2023, 12:32 PMJonas Kemper
03/15/2023, 12:32 PMNok Lam Chan
03/15/2023, 12:33 PMparams:
syntax is something handled by KedroContext
(search for KedroContext._get_feed_dict)
When you use add_feed_dict
, it really just store the dictionary as A MemoryDataSetJonas Kemper
03/15/2023, 12:40 PMNok Lam Chan
03/15/2023, 12:41 PMJonas Kemper
03/15/2023, 12:42 PMfrom kedro.framework.context.context import _get_feed_dict
?Nok Lam Chan
03/15/2023, 12:51 PMKedroContext
and let it create the catalog
etcKedroContext
to just use the context._get_feed_dict
method or just copy that piece of code_Jonas Kemper
03/15/2023, 12:53 PMNok Lam Chan
03/15/2023, 1:00 PM