https://kedro.org/ logo
#questions
Title
# questions
h

Harsh Maheshwari

04/01/2023, 9:02 PM
Hi everyone, I am trying to run Kedro with Ray, considering the potential of ray distributed and Kedro pipelines an integration would help a lot. Has someone tried something similar, Need some guidance on how to proceed!
d

Deepyaman Datta

04/02/2023, 5:22 AM
Yeah, people have. At a high level, the deployment pattern can be pretty similar to how it is for Dask (which I know you've experimented with). https://github.com/kedro-org/kedro/pull/995/files#diff-dbf04ed0f48883285bfb145a14ce9d7a1d33f6808ba3a375d239f4ba8949a742 is a publicly-available project with Ray execution, but it also includes significant changes to the Kedro framework code (since that was the purpose of the hack project). There are people who have proven it out within QuantumBlack, too, but I can't find them on this channel.
j

Juan Luis

04/02/2023, 6:19 PM
for future reference, I see you also chimed in https://github.com/kedro-org/kedro/issues/479 👍🏼