Afaque Ahmad
11/24/2022, 9:43 AMcache
made available inside the _load
method of multiple Kedro Datasets. How to go about it? Can we use hooks? or anything simpler?marrrcin
11/24/2022, 9:57 AMdatajoely
11/24/2022, 10:05 AMAfaque Ahmad
11/24/2022, 10:08 AMredis
cache having metadata updated dynamically based on which data is loaded.datajoely
11/24/2022, 10:09 AMAfaque Ahmad
11/24/2022, 10:13 AMmarrrcin
11/24/2022, 10:24 AMbefore_dataset_loaded
hook most probablydatajoely
11/24/2022, 10:27 AMbefore_pipeline_run
since you have access to everythingAfaque Ahmad
11/24/2022, 10:35 AMbefore_pipeline_run
, the datasets would be loaded by that time? I need to inject the values before loading the dataset, seems I've to use before_dataset_loaded
?
before_dataset_loaded
is called each time before loading the dataset or only once before all datasets are loaded?Nok Lam Chan
11/24/2022, 11:08 AMAfaque Ahmad
11/24/2022, 11:55 AMbefore_pipeline_run
. Where can I find all the inputs to the hook before_pipeline_run
? In the example I see only run_params
pipeline.inputs
attr will be helpful hereAfaque Ahmad
11/24/2022, 11:58 AM