Hey all, I'm trying to run my Kedro project programmatically but I want to update some variables for the run, but there doesn't seem to be an easy way to do that. The variables I have are specified as

runtime_params

but

session.run()

doesn't take

runtime_params

, it seems like those should be put in

KedroContext

but creating a new context looses all of the other information and there doesn't seem to be an easy way to create a new context from an existing one or just add

extra_params

. Is there something I'm missing or is this a weird use-case?

How did you create the session? Can you show some code snippets

I'm doing it in a notebook so I'm currently using the

%load_ext kedro.Ipython

to create the session, but in future iterations I imagine I would use bootstrap

Hi, this is totally a legitimate use case, and unfortunately there is no other way for now that recreating the session each time you run it, or to use the low level API (e.g handle runner, pipeline, hooks, catalog manually).

Using extra_params should work. KedroSession.create

Yes sure, but you have to recreate a session each time,you cannot pass it to session.run()

Yes - I understand this is not convenient. For the record I voted for not locking up the session as I used to do a lot of interactive workflows in notebook/partial re-run pipeline etc. In case anyone found this thread, if it's a one-time usage.

with KedroSession.create(xxx, extra_params) as session:
  session.run()

You need to re-create a new session everytime. For a slightly more hacky solution, I think this should work:

def update_runtime_params(session, runtime_params: dict):
      session._store["extra_params"] = runtime_params

update_runtime_params(session, params)
session.run()

The way how session getting params from the session store is weird and session store is tight to the experiment tracking feature a lot which I am not a fan of.

I think your recommendation about adding them to the session store is exactly what I was looking for, why is it hacky other than accessing a private attribute?

changing a private attribute could break in any version as we don't guarantee the API