Juan Luis
12/01/2023, 5:56 PMdataset:
filepath: ${custom_kedro_resolver_magic:project_root}/data/01_raw/iris.csv
? has anybody come up with a custom resolver that can do this dynamically, rather than having to, say, hardcode it in globals.yml
?Juan Luis
12/01/2023, 8:28 PMJuan Luis
12/01/2023, 8:29 PMTakieddine Kadiri
12/04/2023, 8:20 PMpackage_path
settings.py
from kedro.config import OmegaConfigLoader
CONFIG_LOADER_CLASS = OmegaConfigLoader
CONFIG_LOADER_ARGS = {
"custom_resolvers": {
"package_path": lambda: os.path.basename(__file__),
}
}
catalog.yml
dataset:
filepath: ${package_path:}
We used to resolve package_path
inside a CustomConfigLoader (before custom resolver was a thing) this let us have a sort of application conf, where we bundle some artifact alongside the src code (stopwords, referentials, …) and having them versioned and released with the python package. A dataset would point to them relative to package_path
.
Maybe this is outside of the scope of the question, since it’s about projet_root. I don’t know how to do that for project rootJuan Luis
12/04/2023, 11:27 PMsettings.py
, the place where this resolver would go, doesn't have enough info to know where project_root
is. one could hack something around but it would be brittle I guess.