Hey, everybody. Recently I started to look into the possibility of integrating kedro as a fundamental framework for development teams, but I've come across the fact that one of the modules that currently exist (Micro-packaging) will be deprecated in version 0.20.0. Can anyone tell me if there are other alternatives to continue using this approach? The idea is that code generated within one pipeline can be passed as an independent library to another development team or can be used in the future in other projects to minimize the need to rewrite the same code. The problem I see with using only the kedro package module is that if you only use this option, you will eventually have to deploy multiple projects to get a particular functionality that is needed and has already been developed in other projects, which will make the deployment process laborious. I will be glad to receive any information and comments. Thanks in advance!

Someone will reply to you shortly. In the meantime, this might help:

So something that I’ve come to realize is that you don’t need to package isolated bits of code, you just

kedro package

the whole project, and deploy it multiple times with different run logic It’s the same performance and execution plan, just less complexity. Code is cheap who cares if you package it twice.

The code that you should shared as a library is the function - not necessary the pipeline. As the complexity grow you would also likely develop some common utils, which could be a separate Python package by itself. That will be pure python and nothing specific about Kedro (which I think is a good thing). The different bit in deployment of the same/similar pipeline will most likely the configuration(i.e. dataset).

I may not have expressed the basic idea correctly. The idea is to use Kedro as a standard for developing data-related projects within a company. Considering that it is a standard for all projects, the ability to use parts of the pipeline from other projects would make life a lot easier and reduce development time. You are right Nok, you can take individual functions or already created functionality and create your own library of tools, in this case it will really not be related to Kedro and I think it will be the best alternative for Kedro Micro-packaging. I will be glad to hear other opinions. Thanks @Nok Lam Chan and @datajoely.

yeah in any case I agree you should decouple your business logic into standard independently tested packages rather than coupling to the flow logic on kedro