Hi Team, I have written Kedro pipelines for data processing, model training, and scoring. To deploy a trained model for realtime inference, I want to see if it's a good idea to reuse data processing and scoring pipelines. To minimize the latency, what's the best way to utilize nodes and pipes written in Kedro?

Hi @Jonghyun Yun, one perspective on that I personally like is the one taken by the kedro-mlflow plugin: https://kedro-mlflow.readthedocs.io/en/stable/source/04_pipeline_as_model/index.html

Thank you @Guillaume Tauzin, it is interesting stuff, and definitely look into it

Hi @Jonghyun Yunand @Guillaume Tauzin . I happened to have the exact same question today and was glad someone else had the same one. I also have a batch training pipeline defined in Kedro, which is very lean and trains on DataFrames. The models that it outputs should be used for reatime inference. I was under the assumption that I could define a separate 'inference' pipeline with raw input and then reuse the Transform Nodes on that before invoking model.predict(). But alas, the Nodes are written from the DataFrame perspective, and not individual (string) input. I think my strategies are: 1. Take the string -> DataFrame conversion overhead per instance. This is easiest but may not perform well 2. Rewrite the transformations steps as Nodes that don't use DataFrames and build a custom inference script; this is faster but doesnt leverage code reuse as much 3. Maybe there is another option? The MLFlow route as above seems a bit overkill for this usecase. Also it seems Kedro Pipelines are designed for training only and not so much for the inference part? Any ideas on this?

Hi @Martin van Hensbergen May I ask what you mean by "Nodes are written from the DataFrame perspective and not individual string input?" As for the last point, Kedro helps you author generic pipelines, I do think they can be used for training and inference but it all depends on how you run/orchestrate them.

Hi @Guillaume Tauzin thanks for replying. To be honest, I wrote that at the end of a long day and when i drove home I thought: "I can put in those nodes whatever I want so if DataFrames are not what I need just don't use them"... so I got that sorted out now 😉 As for the latter issue I asked a separate question to which you just gave an answer; i'll continue that discussion on that thread. Thanks again for your help!