Hi, I’d like to share a crazy story that happened to me yesterday, about how Open AI’s chatbot taught me something in Kedro I NEVER SAW ANYWHERE ELSE 🤯 So I was toying around with the chatbot asking him. I asked the chatbot : “How to implement a pipeline that reads an Excel file with arbitrary amount of sheets in the Python framework Kedro ?” (recent issue I was facing). Sadly, The chatbot come with an answer that was totally wrong. Code didn’t work, wrong syntax, bad usage of the framework, etc. Weirdly enough, it used some weird decorator syntax to define a node with Kedro. The first screenshot is how he used it. I didn’t think about it much longer (apart from “Well, looks like I’ll still have a job for a few more years 😅 “). BUT THEN. This morning, it struck me ! With a few tweaks, this syntax totally works ! 💥 Indeed, a node can be seen as a decorator, since it’s a function that takes a function as an input. On the second screenshot, I show my implementation of this chatbot’s inspired syntax. It’s crazy because everything in Kedro works the same way. I personally love this syntax. • Having nodes’ inputs and outputs right next to the transformation function definition makes it very clear what type is what. Much less error prone and back and forths between

node.py

and

pipeline.py

. • It also opens up the possibility of generating the pipeline automatically (by scanning the

node.py

file for every function decorated by node_decorator), making

pipeline.py

totally optional. • I could even see the inputs and outputs being automatically inferred from the function definition, a bit like FastAPI does using Pydantic models. Did you every see or use this decorator syntax for defining Kedro nodes ? What’s your experience with OpenAI’s chatbot ? Could this be implemented in Kedro ? (we would just need to make so

func

can be a positional argument)

So I think that was introduced originally to make notebook life easier, but I quite like the way you’ve string it together

Sounds interesting, do you have a link to a github repo with those prototypes ?

not yet… but soon 🙂

What if you want to reuse a node? Think that's pretty common. Would you apply the decorator twice?

You’re right Deepyaman, this doesn’t work when you want to reuse a node. You could just use the current technique without the decorator But it would be nice to be able to stack node_decorators on top of each others 🤔

referenced this thread in https://github.com/kedro-org/kedro/issues/2408 ⭐