Yury Fedotov
04/14/2024, 1:08 AMparameters
references will not be namespaced, but params:
references will.
Do parameters
here refer to a keyword argument of pipeline
wrapper, or it means that in node definition, I can use e.g. "learning_rate": "parameters:learning_rate"
instead of "learning_rate": "params:learning_rate"
and it will not be namespaced?
P.S. if the answer is that it's about pipeline
wrapper, then the follow up question is... Is there any way to somehow prohibit namespacing params at the node
definition? The background for the question is the following. I'm using namespace pipelines to process different datasets via the same multi-node logic, but parameters should be the same. And without prohibiting namespacing those, each time I use pipeline
wrapper I need to map a crazy number of parameters to prohibit namespacing those since I want them to be reused by all modular pipelines.Elena Khaustova
04/15/2024, 10:50 PMpipeline
wrapper is meant. So, those parameters that are referenced in the parameters
argument won’t be namespaced.
Is it correct that the main problem is that you don’t want to provide the whole set of parameters via this argument as it is too extensive?
Also, from your description, I'm using namespace pipelines to process different datasets via the same multi-node logic
, using different configuration environment might be helpful. You create two environments pointing to different datasets and switch between them at the runtime.Yury Fedotov
04/15/2024, 10:52 PMpipeline(…, parameters={})
it’s clear
2. “Is it correct that the main problem…” - yes
3. Thanks I’ll have a look at config environments!