Or, is there such collections/gallery for complex/...
# questions
u
Or, is there such collections/gallery for complex/real-world kedro applications?
💡 2
j
the closest thing is https://github.com/kedro-org/awesome-kedro but admittedly is not very structured
d
Hi folks, I'm wondering if there are any benchmark suite that consists complex pipelines written in kedro?
I would also love this! Very hard to show the value of a performance-improving plug-in on Iris lol.
👀 1
m
I don’t think kedro would necessarily improve pipeline performance. Rather, it improves developer productivity, which, in itself is hard to measure/showcase with just an example repo. But I do see the value of figuring out the best way to demonstrate this to new users.
u
Thank you, @Juan Luis. I have gone through the list but seems these blogs are mainly focused on the usage.
Hi @Matthias Roels just like @Deepyaman Datta said, what I'm finding is some real use cases (from real-world or being synthesized) that can run in our environments and can be used to measure the workload performance and identify the problems or optimization opportunities in the kedro part, e.g., which dataset should I use? Will introducing S3 to my application has big performance impact in Kedro? Or even more complex questions like what should I expect if I deploy complex pipelines to a distributed environment like Kubernetes (with argo) or Dask cluster? As well as questions like how should be justify the gain and pain if I developed a Kedro plugin (e.g., kedro-accelerator)? Maybe the community could have such a list for kedro developers?