Hi! If I have 2 configuration environments (

local

and

prod

), is it possible to know during pipeline creation which environment is run? Or how should I do this using proper Kedro patterns. Eg. I want to do something like:

def create_pipeline(env: str = "local") -> Pipeline:
    if env == "prod":
        return create_pipeline_prod()
    else:
        return create_pipeline_local()

Hey Arnout, I don’t believe you can access the run environment from the pipelines like this in a straightforward way. What you could try doing is configuring the environment through env variables like this: https://docs.kedro.org/en/stable/configuration/configuration_basics.html#how-to-specify-additional-configuration-environments

export KEDRO_ENV=dev

And use the env variable in the

create_pipeline()

to determine which pipelines should be created.

Okay, thanks! I considered this solution but thought the setup would be confusing when eg. someone runs the pipeline with

kedro run --env=dev

and does not get the same behavior. But I'll guess I'll do it this way 🙂

yeah, the

--env

passed through the CLI does override the environment variable. It’s not ideal but dynamic pipelines are generally not supported by Kedro. Another thing I could think of was maybe passing params specified in the configuration environments which modify the behaviour of the node functions on that level but the pipeline structure itself remains the same.

actually I had the same question while taking a pandas-based project to Databricks. I now find it interesting that one can change the datasets with the Kedro environment but not the code (local with pandas and dbx with PySpark).