Hi everyone! Is there a way to access `namespace` ...
# questions
m
Hi everyone! Is there a way to access
namespace
in a python script the same way I can access e.g., parameters? For example, I can access parameters in a
py
script as such:
Copy code
from kedro.framework.project import settings
CONFIG_LOADER = settings.CONFIG_LOADER_CLASS(conf_source=settings.CONF_SOURCE, **settings.CONFIG_LOADER_ARGS)
params = CONFIG_LOADER["parameters"]
But how can I access the namespace I pass when running
kedro run --pipeline data_preprocessing --namespace example_space
to for example - in the simplest form - be able to log "namespace is set to: example_space"? Thank you!!!
h
Someone will reply to you shortly. In the meantime, this might help:
d
I think you can retrieve this information from a hook and then set an environment variable. The nodes intentionally are not aware of IO or execution, they are designed to be pure python functions with no side effects
m
Thank you! I realised after your response that an initial hook I had previously set up was not even working and am now struggling to get hooks to work in general. I've followed the documentation - and am a bit lost as how to fix this problem as the hooks are correctly registered but still not triggered when I run the pipeline. Do you have any idea why hooks would be registered but still not be triggered when running the pipeline? Do namespaces influence
before/after_pipeline_run
hooks in any way? Some more context:
hooks.py
Copy code
class PipelineStatusHookNew:
    def __init__(self):
        logger.info("PipelineStatusHookNew instantiated.")
        logger.info(f"Class instance: {self}")

...

    @hook_impl
    def before_pipeline_run(self, run_params, **kwargs):
        logger.info("before_pipeline_run triggered with run_params: %s", run_params)
        self.start_time = datetime.now(tz=timezone.utc)
        logger.info("Logged run start time for state management table update.")
...
CLI
You can see that that the
def __init__(self):
logs are being displayed but then the pipeline and nodes start running without triggering the
before_pipeline_run
method. And I've tried before/after pipeline/node etc. - logging things but also writing to an output file but nothing's happening.
Copy code
kedro run --pipeline sample_testing_pipe --namespace industry

[03/07/25 15:33:04] INFO     Using 'conf/logging.yml' as logging configuration. You can change this by setting the KEDRO_LOGGING_CONFIG        __init__.py:270
                             environment variable accordingly.
2025-03-07 15:33:19,052 - hook_registry.hooks - INFO - PipelineStatusHookNew instantiated.
[03/07/25 15:33:19] INFO     PipelineStatusHookNew instantiated.                                                                                   hooks.py:16
2025-03-07 15:33:19,055 - hook_registry.hooks - INFO - Class instance: <hook_registry.hooks.PipelineStatusHookNew object at 0x162526620>
                    INFO     Class instance: <hook_registry.hooks.PipelineStatusHookNew object at 0x162526620>                                     hooks.py:17
                    WARNING  /Users/Marie_Cordes/.pyenv/versions/3.10.9/lib/python3.10/site-packages/kedro_viz/integrations/kedro/hooks.py:13: warnings.py:109
                             KedroDeprecationWarning: 'TRANSCODING_SEPARATOR' has been moved to 'kedro.pipeline.transcoding', and the alias
                             will be removed in Kedro 0.20.0
                               from kedro.pipeline.pipeline import TRANSCODING_SEPARATOR, _strip_transcoding

                    INFO     Kedro project classifications-v2                                                                                   session.py:329
                    INFO     Using synchronous mode for loading and saving data. Use the --async flag for potential performance gains. sequential_runner.py:74
                             <https://docs.kedro.org/en/stable/nodes_and_pipelines/run_a_pipeline.html#load-and-save-asynchronously>
                    INFO     Loading data from model_config_dev (SnowparkTableDataset)...                                                  data_catalog.py:390
[03/07/25 15:33:20] INFO     Loading data from params:industry.model_configs (MemoryDataset)...                                            data_catalog.py:390
                    INFO     Running node: fetch_model_configs_from_snowflake_node:                                                                node.py:367
                             fetch_model_configs([model_config_dev;params:industry.model_configs]) -> [industry.classifier_profiles]
...

<< proceeds to run the pipeline >>
src/classifications_v2/settings.py
Copy code
from hook_registry.hooks import PipelineStatusHookNew
HOOKS = (PipelineStatusHookNew(),)
d
so it's a little to tell from these logs
all I can suggest is to place a
breakpoint()
in the hook and make sure it actually gets called