Hello kedro experts, Is it possible to create dynamic pipelines based on params? For example: My parameters.yml file:

pipelines:
 - test1
 - test2

I want to return 2 pipelines like this:

pipes = []
for pipeline in pipelines:
   pipes.append(Pipeline([
      node(do_something, [f"params:{pipeline}"], [f"output_{pipeline}"], tags=pipeline)
   ]))

In the older versions of Kedro I was able to get the params before the creation of pipelines and then work from there. Like this:

def get_kedro_env() -> str:
    """Get the kedro --env parameter or local

    Returns:
        The kedro --env parameter
    """
    return os.getenv("KEDRO_ENV", "local")


def _get_config() -> ConfigLoader:
    """Get the kedro configuration context
    Returns:
        The kedro configuration context
    """
    try:
        return get_current_session().load_context().config_loader
    except Exception:  # NOQA
        env = get_kedro_env()
        return ConfigLoader(["./conf/base", f"./conf/{env}"])


def get_params() -> Dict[str, Any]:
    """Get all the parameter values from the parameters.yml as a dictionary.
    Returns:
        The parameter values
    """
    return _get_config().get("parameters*", "parameters*/**", "**/parameters*")

It seems like in kedro 0.18.3 There is no more load_context() any thoughts?

Hi Panos. This is removed in 0.18 indeed, the more Kedro way to do it is via the after_context_hook to get access of the parameters

yet that is called after the pipelines have already been created

I don’t have a good solution yet, parameters wasn’t designed for constructing the pipeline and we always try to avoid dynamic pipelines. A relevant thread here 🧵 https://kedro-org.slack.com/archives/C03RKP2LW64/p1667910377475889