Nicolò Vallana
09/13/2024, 8:40 AMHuong Nguyen
09/13/2024, 10:40 AMNok Lam Chan
09/13/2024, 11:23 AMsettings.py
• globals.yml
and it's path
• your catalog.yml
or parameters.yml
that referencing the globals config?Nok Lam Chan
09/13/2024, 11:23 AMNicolò Vallana
09/13/2024, 11:51 AM"""Project settings. There is no need to edit this file unless you want to change values
from the Kedro defaults. For further information, including these default values, see
<https://docs.kedro.org/en/stable/kedro_project_setup/settings.html|https://docs.kedro.org/en/stable/kedro_project_setup/settings.html>."""
# Instantiated project hooks.
from adviser.hooks import (
SparkHooks,
PDBNodeDebugHook,
PDBPipelineDebugHook,
) # noqa: E402
# Hooks are executed in a Last-In-First-Out (LIFO) order.
HOOKS = (SparkHooks(),PDBNodeDebugHook(),PDBPipelineDebugHook())
# Installed plugins for which to disable hook auto-registration.
# DISABLE_HOOKS_FOR_PLUGINS = ("kedro-viz",)
from pathlib import Path # noqa: E402
from kedro_viz.integrations.kedro.sqlite_store import SQLiteStore # noqa: E402
# Class that manages storing KedroSession data.
SESSION_STORE_CLASS = SQLiteStore
# Keyword arguments to pass to the `SESSION_STORE_CLASS` constructor.
SESSION_STORE_ARGS = {"path": str(Path(__file__).parents[2])}
# Directory that holds configuration.
CONF_SOURCE = "conf"
# Class that manages how configuration is loaded.
from kedro.config import OmegaConfigLoader # noqa: E402
CONFIG_LOADER_CLASS = OmegaConfigLoader
# Keyword arguments to pass to the `CONFIG_LOADER_CLASS` constructor.
CONFIG_LOADER_ARGS = {
"base_env": "base",
"default_run_env": "local",
"config_patterns": {
"spark": ["spark*", "spark*/**"],
}
}
# Class that manages Kedro's library components.
# from kedro.framework.context import KedroContext
# CONTEXT_CLASS = KedroContext
# Class that manages the Data Catalog.
# from <http://kedro.io|kedro.io> import DataCatalog
# DATA_CATALOG_CLASS = DataCatalog
My globals file:
# User Configurations
USER: admin
DATA_FILEPATH: /project_name/data
# Sensor Configurations
SENSOR: XVF
Path of the globals file: /project_name/conf/base/globals.yml
My catalog.yml:
master_table:
type: abstractdataset
parameters_list:
- ${globals:SENSOR}
Juan Luis
09/13/2024, 12:58 PMomegaconf.errors.InterpolationResolutionError: Globals key 'DATA_FILEPATH' not found ...
Nok Lam Chan
09/16/2024, 4:09 PM