Abhishek Bhatia
06/12/2023, 1:06 PMMemoryDataSet
from nodes. By default kedro, deep copies the memoery dataset which leads to loss of information so I created a catalog entry with copy_mode
set to assign
. This solves our basic problem of objects being retained as is but messes up the DAG order displayed in kedro viz. Any solutions?Nok Lam Chan
06/12/2023, 1:10 PMAbhishek Bhatia
06/12/2023, 1:13 PMNok Lam Chan
06/12/2023, 1:14 PMAbhishek Bhatia
06/12/2023, 1:17 PMatw.dataset1
, atw.dataset2
...
So these got grouped together and all nodes showed up at the same level without dependancy/connections 🤦datajoely
06/12/2023, 1:44 PM-
instead of your .
prefix delimiter, I think this is interfering with a diff feature in Viz ?Abhishek Bhatia
06/12/2023, 1:49 PMNok Lam Chan
06/12/2023, 1:51 PM.
has special meaning of modular pipeline in Kedrodatajoely
06/12/2023, 1:51 PMNok Lam Chan
06/12/2023, 1:51 PMModularPipeline
, it’s just a pipeline afterall.
Something like my_pipeline.abc
is totally valid in catalog.yml