Hi, is anyone here using Kedro and (PyTorch)Lightn...
# questions
s
Hi, is anyone here using Kedro and (PyTorch)Lightning together? It seems to me that Kedro has a lot of advantages like the DAG, and the fact that you can keep all of your parameters and files in one yaml-file etc. At the same time, I worry that Kedro and Lightning will fight over the control of the hardware. Lightning automatically detects GPUs and takes care of distributed training and so on. I don't want Kedro to interfere with this. Can I for instance say to Kedro: "let Lightning do the hardware managment"?
👍🏼 1
a
Hey @Sven-Arne Quist I’ll let the community chime in with their experiences but I think Kedro doesn’t concern itself with the hardware but simply running whatever code you’ve written in your nodes in a topologically sorted order so in theory, it shouldn’t be a problem!
y
I strongly suggest you use the default SequentialRunner instead of Thread runner or ParallelRunner which can interfere though
👍 1