https://kedro.org/ logo
#questions
Title
# questions
c

Clay Smyth

10/02/2023, 6:50 PM
Hi, I apologize because this has probably been asked but I couldn’t find adequate documentation on this (I’m new to Kedro). For our use case, we use Hydra to modularly develop function handles, but I would like to wrap the pipelining of the function handles with Kedro. This seems to require running Kedro within script, instead of the CLI. The only documentation I could find on running within script is https://dev.to/waylonwalker/using-kedro-in-scripts-6l2. Is this not the recommended use case, or is there other documentation in which I can learn more about developing Kedro pipelines within script?
j

Juan Luis

10/03/2023, 6:53 AM
hello @Clay Smyth! you can absolutely run Kedro programmatically. the instructions have changed a bit since 2021: https://docs.kedro.org/en/stable/kedro_project_setup/session.html#create-a-session
let us know if you have any further questions
c

Clay Smyth

10/03/2023, 4:59 PM
Thanks!
f

fmfreeze

11/24/2023, 10:07 AM
I have a question on this: What is the difference between such a
KedroSession
object and the ipython/notebook
session
object?
I have a use case where a data scientist wants to run a session in a jupyter notebook and pass "runtime_params" to it (like one can do in CLI via
--params=key:value
). That is possible via
params
argument of a
KedroSession
but not via an ipython
session
object?
see better formulated comment below, which is also posted in #questions as recommended by @Juan Luis
j

Juan Luis

11/24/2023, 10:18 AM
@fmfreeze for visibility, better to post a new thread or "also send to #questions " 🙏🏼
👍 1
f

fmfreeze

11/24/2023, 11:00 AM
I have a question on this: Is there a difference between such a
KedroSession
object and the ipython/notebook
session
object? And how does one implement parameter overriding programmatically? According to @Juan Luis’s link above
the instructions have changed a bit since 2021:
https://docs.kedro.org/en/stable/kedro_project_setup/session.html#create-a-session
it is possible to set
runtime_params
via
params
argument of a
KedroSession
, but that is not documented for not via an ipython
session
object. (see https://docs.kedro.org/en/latest/notebooks_and_ipython/kedro_and_notebooks.html#session) I tried with the latest kedro release (0.18.14) and an ipython
session
seems to be a
kedro.framework.session.KedroSession
object. But this has no argument like
params
,
extra_params
or
runtime_params
to override yaml config parameters
, like one could do with CLI usage (e.g.
kedro run --params=key:value
).
l

Lukas Innig

11/24/2023, 11:11 AM
You would put the params into the %reload_kedro call when working in a notebook
f

fmfreeze

11/24/2023, 11:20 AM
You would put the params into the %reload_kedro call when working in a notebook
thank you, with your hook I found it here: https://docs.kedro.org/en/stable/kedro.ipython.magic_reload_kedro.html remains the question: how does it work with the pure python mechanism?
Copy code
bootstrap_project(Path.cwd())
with KedroSession.create() as session:
    session.run()
I made a dumb mistake 🙈
KedroSession.create()
does have the argument
extra_params
to pass a dictionary to inject parameters at runtime. My fault, i hope this helps future readers.
thankyou 2
n

Nok Lam Chan

11/25/2023, 7:54 AM
Not a dumb mistake at all, I think we can make it more obvious that the goto solution is reload_kedro magic, if not using the KedroSession.create() method