I am cooking up a feature that helps with debugging Kedro Node. Would something like this be useful to you? What more would like to see? The function will prepare the necessary inputs and import for you, so you only focus on debugging the function and less hustle with loading the datasets.

that is very cool

wow

What does it do?

just tell you the list of functions avaialble

Look awesome @Nok Lam Chan. Would there be a path back from notebook into the project’s files (as it was with

kedro jupyter convert

)?

Maybe something like

%update_node

? 🤔

For now, it's only one way from code -> notebook, as the notebook is mean to be short-live and mainly used for debugging. Enable the other way round means we encourage people to actually do the development work in notebook, it's also much harder to get everything right because it's quite risky to edit source file. It's also relatively easy to do this manually (just copy paste the whole cell back to a .py file). That said it's up for discussion, but I think it deserves its own github issue and separate discussion. • What if it's a packaged project (i.e. in

site_packages

?) • Permission problem (may not have access to touch the source code, i.e. Databricks Repo) • Should it works only when the node is already written in a source file? i.e. it has a specific file to live in, or we also need to enable developing a new node in notebook? • Does it only need to handle the node? Or it need to handles the corresponding

catalog.py

entry and

pipeline.py

?

I think it’s excellent in its current format - printing the file path where this was retrieved from is an MVP solution of sorts too

HI. very nice! is it something that could work in kedro ipython also direclty ?

It won't work for IPython, how would you edit multiple cell in a IPython terminal? (not sure if it is possible). In addition, due to the backend support, it will only works for notebook >7.0 (or jupyter lab)

ok thanks . yes youre right for the multiple cells! Ipython is nice to explore with ephemeral code, i often use it to debug/understand without creating any extra file. but thats not the same use case here

It’s still a new feature so we are very open for changes, if you can think of how it can also be useful in a ipython terminal I am happy to explore too!

I think this is awesome, I've been waiting for that for years and now it's becoming a reality thanks to @Nok Lam Chan! 🙇 An IPython version will also be awesome, maybe one that just loads the datasets for you and the imports

what happens when you re-run the command again? does it add the relevant code again in new fields?

It creates new cells. You are not supposed to run that cell twice because once your code is in the notebook, you will be debugging there until you are done and you copy the code back to the source. Are you thinking of something like "reload" the source code? Could you elaborate how you are going to use it?

Sometimes when the kernel hangs it's useful to kill it and then do run all cells to restart and resume w work and I wonder how it behaves then

Good point - not sure if it's feasible and I am not sure if editing the cell that you are running feels too intrusive to users. It's not a bad idea.

🔥

I am sure most people have seen the announcement already, but just in case this is out last week: https://kedro-org.slack.com/archives/C03RKAQ0MGQ/p1709221377663529