Hey all. Is there a way to get a single node to ex...
# questionsa
Aaron Niskin (amniskin)
04/07/2023, 4:48 PMHey all. Is there a way to get a single node to execute in, say, aws batch?
d
datajoely
04/07/2023, 4:49 PMkedro run --node {node_name}a
Aaron Niskin (amniskin)
04/07/2023, 4:50 PMCan I run the whole pipeline locally, but have one node run in batch, or is it an all or nothing kinda thing?
d
datajoely
04/07/2023, 4:51 PMdatajoely
04/07/2023, 4:51 PMthere are lots of ways to specify which parts of a project get run
a
Aaron Niskin (amniskin)
04/07/2023, 4:52 PMHmm. But I want to run the whole project. I just want to farm out the compute for one node. It doesn’t seem like this would do that, or am I misunderstanding?
d
datajoely
04/07/2023, 4:52 PMoh as part of the same pipeline run
datajoely
04/07/2023, 4:52 PMnot really, there is stuff coming down the line for that
👍 1
a
Aaron Niskin (amniskin)
04/07/2023, 4:53 PMBasically I want to run the whole thing, but have a node specific runner
d
datajoely
04/07/2023, 4:53 PMyou could maybe design a custom runner today
a
Aaron Niskin (amniskin)
04/07/2023, 4:53 PMYeah I couldn’t find it in the docs, but I figured I should ask
d
datajoely
04/07/2023, 4:53 PMyeah it’s a good push
datajoely
04/07/2023, 4:53 PMwe don’t have a great solution today in truth
a
Aaron Niskin (amniskin)
04/07/2023, 4:54 PMI’ll keep a lookout for it!
d
datajoely
04/07/2023, 4:54 PM💪
a
Aaron Niskin (amniskin)
04/07/2023, 4:54 PMThanks for the help!
Aaron Niskin (amniskin)
04/07/2023, 4:54 PMAnd great work here!
d
datajoely
04/07/2023, 4:54 PM🙏 thanks! Shout if we can help with other questions
👍 1