https://kedro.org/ logo
#questions
Title
# questions
f

Francis Duval

01/17/2024, 6:32 PM
Hello! This is my grid parameter:
Copy code
tuning_options:
  grid:
    batch_size: [64]
    optimizer__lr: [0.0001]
    module__act: [nn.ReLU]
When I want to retrieve this value in my node, I do:
Copy code
param_grid = tuning_options.get('grid')
and then I can use it for grid search:
Copy code
GridSearchCV(estimator=something, param_grid=param_grid, scoring=something2)
The problem is that the value of
param_grid['module__act']
is
'nn.ReLU'
. But I would like to have the class, no the string! Thanks!
j

Juan Luis

01/17/2024, 6:46 PM
hi @Francis Duval! a similar case is covered in the docs, you'd need a custom resolver https://docs.kedro.org/en/stable/configuration/advanced_configuration.html#how-to-use-custom-resolvers-with-omegaconfigloader
f

Francis Duval

01/17/2024, 6:48 PM
thanks! Didn't know it existed!
This part goes in pipeline_registry.py?:
Copy code
import polars as pl
from datetime import date

from kedro.config import OmegaConfigLoader

custom_resolvers = {"polars": lambda x: getattr(pl, x),
                    "today": lambda: date.today()}

# Register custom resolvers
config_loader = OmegaConfigLoader(conf_source=".", custom_resolvers=custom_resolvers)
>>> print(config_loader["parameters"])
{'polars_float64': Float64, 'today': datetime.date(2023, 11, 23)}
j

Juan Luis

01/17/2024, 6:58 PM
actually in
settings.py
, let me open an issue about that
❤️ 1
f

Francis Duval

01/17/2024, 7:26 PM
Works beautifully: In settings.py:
Copy code
import torch
custom_resolvers = {'act_resolver': lambda liste: [getattr(torch.nn, element) for element in liste]}

# Register custom resolvers
config_loader = OmegaConfigLoader(conf_source='.', custom_resolvers=custom_resolvers)
In parameters.yml:
Copy code
grid:
  batch_size: [64]
  optimizer__lr: [0.0001, 0.001]
  module__act: '${act_resolver: [ReLU, Tanh]}'
j

Juan Luis

01/17/2024, 7:27 PM
awesome! 👏🏼
f

Francis Duval

01/17/2024, 8:08 PM
I put the following lines of code at the end of setting.py, and it seems to have broken something because the command "kedro viz run" isn't working anymore (but the pipeline run well):
Copy code
import torch
custom_resolvers = {'act_resolver': lambda liste: [getattr(torch.nn, element) for element in liste]}

# Register custom resolvers
config_loader = OmegaConfigLoader(conf_source='.', custom_resolvers=custom_resolvers)
Do you have an idea why? I have this error:
ValueError: [TypeError("'builtin_function_or_method' object is not iterable"), TypeError('vars() argument must have __dict__ attribute')]
j

Juan Luis

01/17/2024, 11:27 PM
hmm first time I see it, cc @Rashida Kanchwala
r

Rashida Kanchwala

01/18/2024, 12:34 PM
Me too. Could u pls share the traceback?
f

Francis Duval

01/18/2024, 1:54 PM
Hard to do so because my company prevents me from sending snippets or screenshots, but here it is, summarized: ERROR Exception in ASGI application Traceback: C\Users\XG04377\.conda\envs\monenv\lib\site packages\uvicorn\protocols\http\httptools impl.py435 in run_asgi C\Users\XG04377\.conda\envs\monenv\lib\site packages\uvicorn\middleware\proxy headers.py78 in call C\Users\XG04377\.conda\envs\monenv\lib\site packages\uvicorn\middleware\message logger.py86 in call C\Users\XG04377\.conda\envs\monenv\lib\site packages\uvicorn\middleware\message logger.py82 in call C\Users\XG04377\.conda\envs\monenv\lib\site packages\fastapi\applications.py1054 in call C\Users\XG04377\.conda\envs\monenv\lib\site packages\starlette\applications.py116 in call C\Users\XG04377\.conda\envs\monenv\lib\site packages\starlette\middleware\errors.py186 in call C\Users\XG04377\.conda\envs\monenv\lib\site packages\starlette\middleware\errors.py164 in call C\Users\XG04377\.conda\envs\monenv\lib\site packages\starlette\middleware\exceptions.py62 in call C\Users\XG04377\.conda\envs\monenv\lib\site packages\starlette\ exception handler.py55 in wrapped_app C\Users\XG04377\.conda\envs\monenv\lib\site packages\starlette\ exception handler.py44 in wrapped_app C\Users\XG04377\.conda\envs\monenv\lib\site packages\starlette\routing.py746 in call C\Users\XG04377\.conda\envs\monenv\lib\site packages\starlette\routing.py288 in handle C\Users\XG04377\.conda\envs\monenv\lib\site packages\starlette\routing.py75 in app C\Users\XG04377\.conda\envs\monenv\lib\site packages\starlette\ exception handler.py55 in wrapped_app C\Users\XG04377\.conda\envs\monenv\lib\site packages\starlette\ exception handler.py44 in wrapped_app C\Users\XG04377\.conda\envs\monenv\lib\site packages\starlette\routing.py70 in app C\Users\XG04377\.conda\envs\monenv\lib\site packages\fastapi\routing.py315 in app C\Users\XG04377\.conda\envs\monenv\lib\site packages\fastapi\routing.py170 in serialize_response C\Users\XG04377\.conda\envs\monenv\lib\site packages\fastapi\encoders.py235 in jsonable_encoder C\Users\XG04377\.conda\envs\monenv\lib\site packages\fastapi\encoders.py287 in jsonable_encoder C\Users\XG04377\.conda\envs\monenv\lib\site packages\fastapi\encoders.py301 in jsonable_encoder C\Users\XG04377\.conda\envs\monenv\lib\site packages\fastapi\encoders.py287 in jsonable_encoder And it goes on with jsonable_encoder all the way down.
This whole traceback is repeated many times, and then: Traceback (most recent call last): File "C:\Users\XG04377\.conda\envs\monenv\lib\site-packages\kedro_viz\launchers\cli.py", line 187, in run _wait_for(func=_check_viz_up, host=host, port=port) File "C:\Users\XG04377\.conda\envs\monenv\lib\site-packages\kedro_viz\launchers\utils.py", line 58, in _wait_for raise WaitForException( kedro_viz.launchers.utils.WaitForException: func: <function _check_viz_up at 0x0000025B5F794AF0>, didn't return True within specified timeout: 60 kedro.framework.cli.utils.KedroCliError: func: <function _check_viz_up at 0x0000025B5F794AF0>, didn't return True within specified timeout: 60 Run with --verbose to see the full exception
r

Rashida Kanchwala

01/18/2024, 4:55 PM
Thank you. This is going to require some investigation. Can i please request you to create a bug report on github and we can look into it.
f

Francis Duval

01/18/2024, 6:16 PM
Yes sure, I will do this as soon as I can!
Done!
3 Views