Hi, I have a pipeline that uses a lot of memory. I profiled it with

memory_profiler

, and there is a huge memory usage, before the first node runs. This is critical for me because I am running it on a kubernetes pods, and if I use too much memory, the pod gets killed. Attached is the profile of a

dummy

pipeline that I created that prints a parameter and does nothing else. We can see that it uses nearly 200MiB of memory. I am not sure what I should look for to improve this memory usage. Is it the catalog? If so, how can I reduce it? Is it something else from kedro? Is it outside of kedro?

thanks a lot @FlorianGD, this is really helpful. could you open an issue with this same message and screenshot? and if you can share the "dummy" project, much better. we've been keeping an eye on the long startup times but never really looked at the memory (btw I'm supposed to be a maintainer of memory-profiler but Fabian has moved on and I've been neglecting the issue tracker... glad it still works)

OK, will do!