Toni
09/09/2022, 1:21 PMset_targets = ['a', 'b', 'c']
, we can loop the same pipeline for each value of that list without "copying" that pipeline? We may have a different length and names for that "`set_of_targets`", and thus we want to avoid manual work... Also, we need the outputs to have "dynamic" names in the catalog in order to save all the outputs (score_{{target}}
... score_a
, score_b
, score_c
)... I think this could be done with jinja
, but no idea where to start...
Thank you very much!Shubham Gupta
09/09/2022, 1:44 PMToni
09/09/2022, 2:11 PMnamespaces
is a tool to keep in mind, I think that would fit my problem if I would know the set of targets through which I have to iterate and run the pipeline "beforehand".
The problem is that I may not have that set of targets, and that I cannot add the namespace parameters and catalog entries
until I have the set (which may be different from one project to another).
The PO would like to have a "dynamic template", that only changing a set_of_targets parameter
, it would automatically create those namespaces
Nok Lam Chan
09/09/2022, 3:54 PM