```Hi team, I would like to discuss a feature idea...
# questionsd
Dustin
01/20/2023, 4:05 AM Copy code
Hi team, I would like to discuss a feature idea (or this is already implemented?) to seek your thought :)
Context:
It is common in practice to know the consuming time of the whole pipeline and the consuming time of each node in the pipeline.
I assume the stakeholder/engineers would like to understand the performance of pipline and which part can be optimized.
Features:
1. Is it possible to show consuming time (in second/minutes/) of each node in the pipeline?
1.1 by default, it is shown in the console as part of logging and you can configure to turn it off
2. Given feature 1, is it possible to show the consuming time of each pipeline?
2.1 by default, it is shown in the console as part of logging at the end of each pipeline running
2.2 in case there are multiple pipeline, show it for each pipeline, you can configure to turn it off
3. Given feature 2, is it possible to show the consuming time of all pipelines in total
3.1 by default, it is shown in the console at the end of all pipelines running and you can configured
Deepyaman Datta
01/20/2023, 4:35 AMI agree that this is useful, and have used something similar internally when I used to work at QuantumBlack. In case you're looking for how to get started, a basic implementation could look a lot like https://kedro.readthedocs.io/en/stable/hooks/examples.html#add-memory-consumption-tracking (but tracking node times instead/as well).
d
Dustin
01/20/2023, 6:03 AMAha, thanks Deepyaman for your comment. My bad, I gave a wrong expression, was asking whether QuantumBlack team would consider adding it so it can be properly maintained
j
Jannic Holzer
01/20/2023, 8:44 AMHey @Dustin, thanks for the suggestion 😃 generally the way to get a feature added is to open an issue (https://github.com/kedro-org/kedro/issues) for the framework team to review 🙂
d
Dustin
01/20/2023, 9:06 AMthanks Jannic, will do 😁
2 Views