Hey team! Not a kedro question per se. What is th...
# questions
a
Hey team! Not a kedro question per se. What is the go to tooling for configuration management in data science projects outside of kedro (with OmegaConf)? Is Hydra the most popular choice? I am looking at the following features: 1. Global configs 2. Clear patterns for config type a. Static vs Dynamic b. Global vs Granular c. Constant vs Overridable 3. Param overriding with Globals 4. Param overriding within config file 5. Support for environment variables 6. Storing environment wise configs - DEV / STG / UAT / PROD etc 7. Interpolation with basic text concat 8. (Optional) Python function as resolvers in config (OmegaConf) 9. Config compilation artifact (i.e. I want to see how my config looks after resolving) 10. Invoking python scripts with arbitrary / alternate config paths 11. Invoking python scripts with specific param value Most of the above features are already there in kedro, but I need this functionality outside kedro. Eager to here the community's recommendation here! 🙂
d
So hydra and omegaconf were developed by Facebook but neither maintainers are still at the company. I don’t have a great answer for you, other than I don’t think hydra is the horse to back given the lack of direction with the project
j
Agreed with Joel, OmegaConf is unmaintained and Hydra is built on top of it. I think many people have jumped into the Pydantic bandwagon for config, rather than dealing with complex YAMLs
d
yup
I've also just discovered https://docs.pydantic.dev/latest/concepts/pydantic_settings/ which just makes config a doddle