Hi, I apologize because this has probably been asked but I couldn’t find adequate documentation on this (I’m new to Kedro). For our use case, we use Hydra to modularly develop function handles, but I would like to wrap the pipelining of the function handles with Kedro. This seems to require running Kedro within script, instead of the CLI. The only documentation I could find on running within script is https://dev.to/waylonwalker/using-kedro-in-scripts-6l2. Is this not the recommended use case, or is there other documentation in which I can learn more about developing Kedro pipelines within script?

hello @Clay Smyth! you can absolutely run Kedro programmatically. the instructions have changed a bit since 2021: https://docs.kedro.org/en/stable/kedro_project_setup/session.html#create-a-session

Thanks!

I have a question on this: What is the difference between such a

KedroSession

object and the ipython/notebook

session

object? I have a use case where a data scientist wants to run a session in a jupyter notebook and pass "runtime_params" to it (like one can do in CLI via

--params=key:value

). That is possible via

params

argument of a

KedroSession

but not via an ipython

session

object? see better formulated comment below, which is also posted in #C03RKP2LW64 as recommended by @Juan Luis

@fmfreeze for visibility, better to post a new thread or "also send to #C03RKP2LW64 " 🙏🏼

I have a question on this: Is there a difference between such a

KedroSession

object and the ipython/notebook

session

object? And how does one implement parameter overriding programmatically? According to @Juan Luis’s link above

the instructions have changed a bit since 2021:

https://docs.kedro.org/en/stable/kedro_project_setup/session.html#create-a-session

it is possible to set

runtime_params

via

params

argument of a

KedroSession

, but that is not documented for not via an ipython

session

object. (see https://docs.kedro.org/en/latest/notebooks_and_ipython/kedro_and_notebooks.html#session) I tried with the latest kedro release (0.18.14) and an ipython

session

seems to be a

kedro.framework.session.KedroSession

object. But this has no argument like

params

,

extra_params

or

runtime_params

to override yaml config parameters, like one could do with CLI usage (e.g.

kedro run --params=key:value

).

You would put the params into the %reload_kedro call when working in a notebook

You would put the params into the %reload_kedro call when working in a notebook

thank you, with your hook I found it here: https://docs.kedro.org/en/stable/kedro.ipython.magic_reload_kedro.html remains the question: how does it work with the pure python mechanism?

bootstrap_project(Path.cwd())
with KedroSession.create() as session:
    session.run()

Not a dumb mistake at all, I think we can make it more obvious that the goto solution is reload_kedro magic, if not using the KedroSession.create() method