Rennan Haro
08/15/2023, 2:05 PMviz
? Curious to hear some ideas š
The context is that we have a couple of Kedro projects in the client, but having a single viz
graph to visualize all of them would help us track upstream/downstream dependencies.catalog
.
One project is exclusively for data ingestion/preprocessing ā getting data from multiple sources
and storing them in the silver
layer. The other projects are downstream consumers, meaning that they get data exclusively from silver
datajoely
08/15/2023, 2:06 PMRennan Haro
08/15/2023, 2:08 PMs3
(and a couple of SQLQueryDataSets
as sources to the ingestion project)datajoely
08/15/2023, 2:11 PMNok Lam Chan
08/15/2023, 2:13 PMfrom xxx import register_pipelines as register_x
from yyy import register_pipelines as register_y
pipelines = {}
pipelines["x"] = register_x()
pipelines=["y"] = register_y()
pipelines["__default__"] = sum(pipelines)
Additionally, it may requires installing both project or adding both project into PYTHONPATH
datajoely
08/15/2023, 2:15 PMkedro viz --save-file
twice, get the json, modify it and then kedro viz --load-file
Rennan Haro
08/15/2023, 2:16 PMdatajoely
08/15/2023, 2:22 PMNok Lam Chan
08/15/2023, 2:27 PMWorking with the json exports was the first thing that came to mindIn that case a meta project may work but that requires you having all the dependencies installed. Let us know if you have success with concating the JSON.
Rennan Haro
08/15/2023, 2:36 PM