https://kedro.org/ logo
#questions
Title
# questions
p

Panos P

05/10/2023, 5:08 PM
Hello Kedro folks, I have a question on dynamic generation of nodes via parameters in my catalog. I have this paramaters.yml file:
Copy code
params:
 - p1
 - p2

p1: value1
p2: value2
I want to create a pipeline with 2 nodes that each node take as input one of these params. e.g.
Copy code
nodes = [node(func, f"params:{p}", f"output_{p}" for p in params]
pipeline(nodes)
Is that possible and how?
j

Juan Luis

05/11/2023, 10:04 AM
hello @Panos P, my colleagues from the engineering team can probably give a more qualified answer, but generally speaking, generating pipelines dynamically is not something Kedro supports well
n

Nok Lam Chan

05/11/2023, 11:05 AM
Params are meant to be passed into a node (function). It’s probably a good idea to separate these “pipeline_parameters” into a separate file. Then you can read these “pipeline_parameters” and construct the pipeline as you intended.
p

Panos P

05/11/2023, 2:05 PM
I just did what @Nok Lam Chan suggested…
I was hoping I could control pipeline creation via parameters