https://kedro.org/ logo
Join Slack
Powered by
Hi, all, I have a profiling hook that uses the `be...
# questions
r
Hi, all, I have a profiling hook that uses the 
before_pipeline_run
spec to load 
run_params
. However, for some reason the 
run_params
dictionary is empty (I would like to use the 
pipeline_name
key). I'm checking this via a debugging session after calling kedro in the CLI, and also looking at the hook outputs. I'm wondering how else I could troubleshoot this? I'm using kedro 0.19.3.
n
(deleted)
If the 
pipeline_name
is 
None
it's using 
__default__
, you can confirm this with 
kedro run -p <some_pipeline>
Copy code
name = pipeline_name or "__default__"
        record_data = {
            "session_id": session_id,
            "project_path": self._project_path.as_posix(),
            "env": context.env,
            "kedro_version": kedro_version,
            "tags": tags,
            "from_nodes": from_nodes,
            "to_nodes": to_nodes,
            "node_names": node_names,
            "from_inputs": from_inputs,
            "to_outputs": to_outputs,
            "load_versions": load_versions,
            "extra_params": extra_params,
            "pipeline_name": pipeline_name,
            "namespace": namespace,
            "runner": getattr(runner, "__name__", str(runner)),
        }
For some reason the data record here is the argument that get pass in, not the final value. By default if 
pipeline_name=None
then it use 
__default__
IMO it's a bug
r
That's the thing, I'm passing a pipeline name but it's still taking an empty dict
👀 1
I am going to play around with it some tomorrow and report back
Hi @Nok Lam Chan I tested with the following dummy hook and have the same issue:
Copy code
class DummyHooks:
    @hook_impl
    def before_pipeline_run(self, run_params: Dict[str, Any] = dict()):
        breakpoint()
        print(run_params)
n
What is the result?
Do you have a typo? Run_params you have run_parmas
r
😅 I hand-typed it from another machine, it's not a typo
Hmmm...it seems that the default arg is overriding
Updated code that works:
Copy code
class DummyHooks:
    @hook_impl
    def before_pipeline_run(self, run_params: Dict[str, Any]):
        breakpoint()
        print(run_params)
I'm not familiar with pluggy but it seems the default args breaks this functionality somehow
n
Putting 
dict
/`list` in argument is not advised ( an anti-pattern in Python). What you should do is instead:
Copy code
if not arg:
  arg = dict()
Although it's also not clear to me why this override 
pluggy
in this case.
👍 1
Copy code
class DummyHooks:
    @hook_impl
    def before_pipeline_run(self, run_params: Dict[str, Any] = 'dummy'):
        breakpoint()
        print(run_params)
I change this example, and seems that if you give a value to the argument, 
pluggy
no longer treat it as a hook argument for some reason.
Furthermore, each hookimpl or hookspec decorator can configure the underlying call-time behavior of each hook object by providing special options passed as keyword arguments.
https://pluggy.readthedocs.io/en/stable/#marking-hooks It seems like keywords argument are expected to be call-time behavior so it may be reserved for something else. The docs didn't mention this explicitly so I am not sure. I am following up with an issue here: https://github.com/pytest-dev/pluggy/issues/522
👌 1
https://github.com/pytest-dev/pluggy/issues/522#issuecomment-2293770102 > Currently default args are ignored. The discussion on support for them for forward or backward compatibility has not yet yielded implementations So I think the conclusion is, you cannot use default argument as it's not supported (it was never supported I guess). @Richard Purvis
r
There we go. Thanks!
12 Views
Open in Slack
PreviousNext
Powered by