Jan
06/28/2024, 8:01 AMElena Khaustova
06/28/2024, 8:17 AMafter_node_run
hook to log your metrics upon node/nodes execution as in the example: https://docs.kedro.org/en/stable/hooks/examples.html#add-metrics-tracking-to-your-modelJan
06/28/2024, 8:24 AMElena Khaustova
06/28/2024, 8:41 AMJan
06/28/2024, 8:46 AMElena Khaustova
06/28/2024, 9:10 AMkedro run experiment_traking_pipleline + target_subset
Edit: the syntax above is not possible - it’s just to give an idea of splitting the pipeline. But you can sum pipeline objects, see an example below.Elena Khaustova
06/28/2024, 9:12 AMafter_node_run
hook to save your metrics rather than make it in a separate nodeJan
06/28/2024, 9:54 AM--from-nodes
?
For the second option I don't think this is possible. Or will modifications to the catalog object be reflected in the rest of the execution? Because the return type of the function is None?Elena Khaustova
06/28/2024, 10:25 AM--from-nodes
in case your metrics tracking nodes remain in the pipeline after the slice. You do not need to split your pipeline for this.
2. The solution that I mean is to split your pipeline into two:
custom_pipeline = (
experiment_traking_pipleline() + main_pipeline()
)
To further filter nodes you can apply tags: https://docs.kedro.org/en/stable/nodes_and_pipelines/nodes.html#how-to-tag-a-node
So you run kedro run -p custom_pipeline -t tracking, tag_a
Elena Khaustova
06/28/2024, 10:28 AMJan
06/28/2024, 1:17 PM--from-nodes
filter. Putting that one in another pipeline will probably fix that indeed. Thanks a lot for the support 🙂