Hi everyone. Has anyone used Kedro with Tensorflow? If yes, how was your experience? or maybe is better to use another framework like TFX? Thanks!
04/01/2023, 2:42 PM
I think TFX pipelines solves a different problem, and is more focused around the production deployment of pipelines (more similar to the way you deploy to something like Kubeflow Pipelines, which TFX is well-suited for). If you look at https://github.com/tensorflow/tfx/blob/master/docs/tutorials/tfx/components_keras.ipynb, for example, each of the components are a full module file with many functions, and you don't see that lower-level orchestration (just the orchestration of the high-level components).
The way I would personally go about it, if using both together, is to see if each component file can define a Kedro pipeline that is the function called by TFX. That way, can add a bit more structure to the lower-level modules.
Disclaimer: I'm basing this suggestion on more familiarity with other frameworks and looking quickly at TFX, so definitely may be some experts/people with hands-on experience.
04/02/2023, 11:04 AM
Hi Sebastian, I’ve used TF quite a bit with Kedro and haven’t had many issues so far. TFX would require a bit of custom code to integrate with Kedro - I’ve always felt that both kinda try to solve the same problems rather than complement each other so I haven’t really tried to use them together.
Is there any particular question you have around using TF with Kedro?