Hello, we have a kedro pipeline with 450+ nodes and, as you can imagine, we're struggling with kedro-viz. It lags, freezes and takes a lot of time for a simple filter. Is there anything we can do to improve kedro-viz performance? Maybe in the settings or in the source code. Thanks

I'm not sure if there is some modifications you can do to SC of viz to improve performance @Rashida Kanchwala. But if your project has multiple pipelines, loading only the one you need reduces the graph complexity.

kedro viz --pipeline=your_specific_pipeline

There is also a

--lite

mode that only effects not needed dependencies might not have any benefit to the rendering, https://docs.kedro.org/projects/kedro-viz/en/v11.0.2/kedro-viz_visualisation.html#visualise-a-kedro-project-without-installing-project-dependencies.

The solution we have for this is namespaces as it allows you to communicate the complexity of your pipeline but wrapping up nodes into super nodes In terms of performance this is an On3 complexity problem so the layout engine with always struggle at a certain size. Fun fact the layout engine was written by a former game engine developer so it’s as optimized as we can make it

Thanks both! I'll give it a go with namespaces and let you know 👍