Hi all, trying to understand if it's possible to integrate kedro with meta-flow. Are there any existing plug-ins? Has anybody done this in the past?
n
Nok Lam Chan
03/22/2024, 1:54 PM
Is Metaflow a pre-requisite?
Nok Lam Chan
03/22/2024, 1:56 PM
Personally I am not aware of any existing plugin, I think this should be possible but I also not super update with Metaflow.
In general decorator approaches works quite well with Kedro since you can just put that on a function, IIRC Metaflow use something like @step and @flow. Happy to discuss more in details
Nok Lam Chan
03/22/2024, 2:54 PM
cc @Deepyaman Datta as I recall we had some discussion about Metaflow but I can't find the discussion now
d
Deepyaman Datta
03/22/2024, 2:56 PM
Yeah, don't think there's anything out of the box. Outerbounds had expressed some interest in an integration, but nobody got around to building it. 🙂
It should be pretty similar to other orchestrator plugins, where you will map pipelines or nodes to steps.
m
Manali Kadam
03/22/2024, 3:51 PM
Thanks all. Metaflow unfortunately is a pre-requisite. But, I can look into how we can use decorators here. Also, agree with you that it should be similar to other orchestrator plugins. Just wanted to understand if it has been done before