This is a very open ended question, but I'm having trouble finding any reference to someone using nbdev to develop a kedro pipeline. All I can seem to dig up is this repo from @Nok Lam Chan, but I think it was for the kedro repo not a pipeline.... 🤷 Does anyone know of a project where that's been done or have any advice on the subject? Brand new to nbdev so I'm just trying to figure out if there are any good templates out there to help me get started w/ nbdev & kedro

I’m not sure I’ve seen anyone do this - it’s an interesting idea certainly

I'm still new to kedro so maybe I'll feel differently after some time, but I find the need to use

kedro ipython

or

kedro jupyter

a lot to write my code anyways because I can't think through what all I want to do in a node without "playing with" the data... feels like embracing the nbdev approach might streamline things for me but maybe I'm overthinking it 🤷 I'm gonna see if I can figure out how to work it into my kedro workflow and will share details if I have any success

@Nok Lam Chan is currently prototyping this https://kedro-org.slack.com/archives/C03R8N2M8KT/p1705422918899009 Would it be useful in your workflow?

@Mark Druffel I view developing a pipeline mostly "one node at a time", so developing in notebook or the new feature would help. I am a fan of nbdev, if you have suggestions how we can adopt similar approaches i would love to hear more

@datajoely

%load_node

would be very useful. I would definitely use if for debugging and making changes to nodes. That said, it does present a chicken / egg problem & still requires copy / paste. 1. Chicken / egg -

%load_node

only works if you have a node. Maybe

%new_node

could be used to address that?

%new_node <input_node(s)> <input_params(s)>

🤔 2. Copy / paste - any changes would have to be copy / pasted from the notebook to the node. This is a big part of what nbdev tries to solve so potentially that's a good fit to resolve that challenge 🤷 @Nok Lam Chan I'm new to nbdev so I doubt I'll be able to suggest anything you couldn't think through yourself, but I just added it to my repo and am testing it out. I'll share any relevant findings.

🙏 appreciate the feedback, any crazy ideas or suggestions please let us know