Hi all, I’m trying to implement a CLI hook (`befor...
# questions
l
Hi all, I’m trying to implement a CLI hook (
before_command_run
) but it never seems to execute no matter what kind of CLI command I try (
kedro run
,
kedro info
,
kedro ipython
, etc). I’ve looked at examples in the docs, kedro-telemetry, and plugins from the community but can’t figure it out. Does anyone know what I'm missing or have an example they could share? added to
setup.py
Copy code
setup(
entry_points={"kedro.cli_hooks": ["test_plugin = plugin:cli_hooks"]},
)
`plugin.py`(currently at the same level as
setup.py
)
Copy code
import logging
from typing import List
from kedro.framework.cli.hooks import cli_hook_impl
from kedro.framework.startup import ProjectMetadata

logger = logging.getLogger(name)

class ProjectCLIHooks:
  @cli_hook_impl
  def before_command_run(
    self, 
    project_metadata: ProjectMetadata, 
    command_args: List[str],
  ):
    <http://logger.info|logger.info>(
      "Command %s will be run for project %s", command_args, project_metadata
    )

cli_hooks = ProjectCLIHooks()
d
so have you
pip install -e
the plug-in?
n
If I rmb even if you do pip install -e, if you register a new entrypoint you have to install it again.
What are you trying to do with
before_cli_run
hook?
l
So I don't actually have a full plugin written, I was hoping to use it the same way as regular project hooks & just register a single hook by including a
plugin.py
file. Seems like I'm thinking about this incorrectly?
My goal with
before_command_run
was to parse the runtime arguments and add them to a globals.py file for the config loader to pick up during context creation. (I saw in the code that
OmegaConfigLoader
adds runtime args to the parameters key automatically, but I need them in the catalog key).
d
Ah okay I think there is an easier way to achieve that than writing a full plug-in You can actually introduce your own
cli.py
that will override the run command https://docs.kedro.org/en/0.18.12/development/commands_reference.html#customise-or-override-project-specific-kedro-commands
then I think you can use the
extra_params
argument to make this accessible to the configloader/catalog
f
You can just do this and this moves what you pass to params via CLI. I found it be very simple
‼️ 2
❤️ 2
d
That’s beautiful @Fazil B. Topal should include this in the docs!
The omega conf stuff is new, how are you finding it?
n
@datajoely I think you tag the wrong person : p
👍 1
btw @Fazil B. Topal nice creative use of resolver!
🥳 1
l
Thank you all so much for the responses!!
d
Yeah the omegaconf approach is super elegant
@Fazil B. Topal would you mind showing us what a sample CLI command looks like?
f
sure, this is the command i run
Copy code
kedro run --env test --params=countryCode='cz',executionDate='2023-08-13'
I use this in my config files:
Copy code
${runtimeParam:executionDate}/${runtimeParam:countryCode}
Once omegaconf is tries to load them, it parses using what i sent. screenshot is how i use this in the python testing 🙂
d
This is honestly a work of art
thankyou 2
super nice
a
@Fazil B. Topal Registering custom resolvers through
settings.py
is also a feature we’ve merged already and will be out in the next release! (https://github.com/kedro-org/kedro/issues/2622) and docs for it - https://docs.kedro.org/en/latest/configuration/advanced_configuration.html#how-to-use-resolvers-in-the-omegaconfigloader
🥳 2
f
yes, i am waiting for the next release to just pass them in settings directly 🙂
🥳 1
hey @Ankita Katiyar Im using the latest globals from master and i noticed a one subtle error/bug in the code. If i define a globals value and set it to 0 code complains because of the if checks. Also by design i can't pass default as null/None which i think is a limitation. Is it possible to fix these cases? 😄 Example global conf:
Copy code
maxLookBackDays: 0  --> Doesnt work
maxLookBackDays: 1  --> It works
👀 2
a
Thanks @Fazil B. Topal! Let me fix it before the release 😄
🥳 1
f
If we can get a null/None support that would also be nice as i wanted to pass null as default in some cases but it wasn't possible 😄
https://github.com/kedro-org/kedro/blob/main/kedro/config/omegaconf_config.py#L335 this line needs to change for default null support and https://github.com/kedro-org/kedro/blob/main/kedro/config/omegaconf_config.py#L334C26-L334C26 for the 0 support but im sure you already know what needs to be done 🙂
I applied this and it works for 0 and uses default as null
n
Cc @Fazil B. Topal https://github.com/kedro-org/kedro/commit/dfee6436737fe8f4e907dbd9d45ca9a56441296e , we end up going with this approach. 0.18.13 Release is coming either today or tomorrow.
🥳 1
f
ah, thanks for notifying! Will try to migrate to latest version when it's out thankyou
K 1