Hi All, In developing modular pipelines the kedro-viz tool is quite indispensable, without the viz its really quite hard to see whether inputs, outputs and parameters are connected properly. However, the most obvious workflow for an established project to develop a new pipeline/refactor a pipeline based on the documentation, requires several steps which could be streamlined. # The issues: 1. To have more control over over the interactions between the pipelines you want to work on, one can adjust the pipeline_registry to only return the pipelines you want. Which renders the other pipelines unusable (and could lead to bugs down the road). (this can be partly solved by filtering the pipelines in the kedro viz CLI command) 2. For every change you want to visualise, you’ll need to: a. save the file b. stop the current kedro viz c. run kedro viz d. go to the browser window to view the pipeline (which is quite annoying if one has only one monitor at their disposal) 3. Lastly, this does not make for easy debugging/viewing the python objects in the pipelines/nodes. # Half-way solution: using run_viz in a jupyter notebook is great, and solves some issues. I personally combine it with nbdev which allows me to convert the notebook to a python file, and then do run_viz on the entire project. This still suffers from issue 1 even more, but issue 2 and 3 are drastically reduced. Mostly because i just save/commit and then rerun the notebook and get the viz in the notebook. # Better solution Running the

run_viz

command almost begs for the ability to do

run_viz(pipeline)

. Where pipeline is an actual pipeline object. (although it would also be nice as to be able to pass the name of a pipeline to filter like the CLI command w.r.t issue 1). This way, one doesn’t need nbdev (which is a slightly controversial tool), one can develop pipelines easier, without any adjustment to the original project. Since kedro viz can already filter, i can imagine such changes being possible. Also, debugging kedro pipelines from the vscode notebook cell debugger is actually quite nice (i would arque a lot nicer than using the debug configs). Has anyone faced similar issues, or thought of a different solution?

I am very interested what is your experience using it with nbdev

its quite good actually, especially for documenting and testing nodes, i was working on a post about some of the tool combinations we were using, so ill include that.

have you tried using the

--autoreload

flag while running Viz? that should solve your second point in your "# The issues:" section. https://github.com/kedro-org/kedro-viz#cli-usage

oh, okay, that makes life a lot easier!

you're welcome!

@Hugo Evers please do share when it is ready 😀