Lim H.
09/30/2023, 4:12 PMNok Lam Chan
09/30/2023, 6:04 PMdatajoely
10/01/2023, 10:19 AMJuan Luis
10/01/2023, 10:23 AMLim H.
10/01/2023, 11:16 AMdatajoely
10/01/2023, 3:24 PMLim H.
10/01/2023, 3:41 PMpipeline.py
with a bunch of
func_a(a: DataCatalog.A) -> DataCatalog.B
func_b(a: DataCatalog.B) -> DataCatalog.C
and it automatically works out this is a pipeline of A -> func_a -> B -> func_b
the main problem is it lost the pure function nature of the node but I guess you can just do that under the hooddatajoely
10/01/2023, 3:42 PMLim H.
10/01/2023, 3:50 PMnode(func_a, inputs=[DataCatalog.A], outputs=[DataCatalog.B])
is fine tooJuan Luis
10/02/2023, 8:01 AMdatajoely
10/02/2023, 8:35 AMJuan Luis
10/02/2023, 8:42 AMcatalog = DataCatalog.from_config(OmegaConfigLoader.load_file("catalog.yml"))
without having to use the Kedro project template at all @Lim H.?
(context and discussion: https://github.com/kedro-org/kedro/issues/2819)Ivan Danov
10/03/2023, 12:45 PMConfigLoader
can be replaced with a simple dictionary object, the DataCatalog
with a dictionary-like object and then everything should workcatalog.load("dataset") -> catalog["dataset"]
catalog.save("dataset", data) -> catalog["dataset"] = data
load
/ save
/ exists
.