Is it possible to have this PR code as plugin to integrate with kedro? Not sure how general plugins work and i don't know whether it would work as a plugin or needs to be merged into main repo in order to work
07/31/2023, 11:53 AM
Hi Fazil, so generally plugins are maintained by the creator and not the Kedro core team. Our docs give a bit more detail to the plugins: https://docs.kedro.org/en/stable/extend_kedro/plugins.html
I think the topic of versioning is very important and something we’d like to design in more detail as a team. It’s not currently something we’re working on, so any new features would take a while still. That said, I do think it’s worth prioritising the discussion on our side.
Fazil B. Topal
07/31/2023, 11:55 AM
Yes, I checked the website and plugin examples very a bit basic. I wanted to know if I could develop a plugin which can have this versioning behavior. Maybe it's impossible to change the core of how kedro works at the moment. Not an expert in kedro yet so just making sure whether it is possible or not before i put too much effort on this one.
07/31/2023, 12:01 PM
I guess it depends on how you want to make the custom versioning works. The PR linked touches a lot of the Kedro internals, so that probably wouldn’t work as a plugin. There’s different ways of extending Kedro and most involve using Hooks and custom datasets, but without designing this properly I can’t say if that works for this use case. It does seem more involved. You could have a look at some of the community plugins to see if there’s any that are somewhat similar to what you’d like to achieve: https://docs.kedro.org/en/stable/extend_kedro/plugins.html#community-developed-plugins
Fazil B. Topal
08/01/2023, 9:29 AM
I went through the plugins and versioning related one doesnt exist yet. I wonder if I can just simply define a templated path and after catalog is loaded, fill in those values 🤔
Nok Lam Chan
08/03/2023, 8:23 PM
Missed this message! I am happy to discuss. This PR is an interesting one, but the code change is a lot consider the use case is simply customise the time stamp