Kedro is an open-sourced Python framework for creating maintainable and modular data science code.

Kedro

Hi Kedro team!

I have a scenario where I need to load a file from multiple environments and aggregate them. What is the best practice for doing so?
Should I create a custom dataset or are there better ways of doing it?

When you say multiple environments want do you mean?

So i have multiple data catalogs and I have multiple `envs` to access them.

a handpicked list of them &amp; aggregate them in to a single input to my node.

intentionally so - it’s been designed in a way where they are isolated in accordance with the <https://12factor.net/config|12factor app principles>

I think creating an `aggregate` environment with that sort of thing is the best way to do it

If you want to do this dynamically I think the `after_context_created` hook you could dynamically play around with things before the catalog is created with the `env` logic applied, but it’s a bit complicated

Thanks for your suggestions. I'll look into them. :raised_hands: