Hi guys, are you able to package a Kedro project t...
# questions
m
Hi guys, are you able to package a Kedro project to enable loading from the catalog in another application? My use case is that not only I have several datasets (stored in GCP), but also Kedro parameters that I would like to embed and reuse in a front-end application. If there is a way for me to package my project (along with the
conf
directory) so I can easily access parameters and datasets with
catalog.load
or the
ConfigLoader
class, instead of configuring my connection to GCP + recreate the parameters from the ground-up in the new application? Thanks in advance and keep up with the good work, this project is awesome!
y
In case you are using mlflow and want to reuse a machine learning pipeline, you can check https://github.com/Galileo-Galilei/kedro-mlflow-tutorial to get an example on how to do it with a custom mlflow model
👍 1
In case you want something more general and customisable you can check https://github.com/takikadiri/kedro-boot
a
Hey Manoel, in addition to what Yolan said, just the regular
kedro package
command will create a
dist/
folder with the
.whl
file as well as the
conf-<project_name>.tar.gz
which is your compressed configuration (not the
local
) Kedro accepts a
.tar.gz
file as the
conf_source
You can run the project with
kedro run --conf-source <compressed_file_path>
👍 2
m
Hi Ankita, thanks for your tips! Either with MLFlow, Kedro Boot or the
.tar.gz
file for the
conf
folder, all three seem to enable Kedro usage by running pipelines from another application, am I right? My case would be much simpler, because it encompasses only reading datasets and parameters from the
catalog.yml
and
parameters.yml
files, I suppose this is not supported by any of these solutions?