Hi all, what would be the best way to time-profile a kedro pipeline?

Thanks, Joel!

yeah so the point is you can start a timer in one hook and pick it up in another

Great. I will proceed to actually read it now

Hooks expose the lifecycle of a run so you can do lots of cool stuff with them

Awesome. I may need to simplify this to be able to run in databricks

you don't need the extra infrastructure for just profiling. same thing can be replaced by a simple dictionary

Makes sense. What would be the best way/hook to report out the run times after pipeline run?

Essentially, you record start/end for each node, and reporting that at the end of the pipeline using

after_pipeline_run

. For really simple case, it's fine to just print it out as a dictionary. If you want to take this further then maybe it's easier to make this a DataFrame so you can sort it easily, highlighting etc.