I just want to double check something. I 'think' when a node function accesses the parameters that it only uses a copy of the parameters. Is that correct? I was tinkering with Kedro v1.0.0 and I was not able to change the value of a given parameter. IMO it is desirable for these node functions to be unable to modify the parameters.

Hi Galen! Do you mean the parameters defined in the

parameters.yml

file? If that's the case, yes, what you're getting when you call the

params

argument in a node function is a copy, not a reference. If you look at the way the config loader class fetches the parameters from the .yaml files in the

conf

directory in your project, you can see that they are loaded into a python dictionary that'll belong to that one instance of the config loader that's created when you run your pipeline. So they cannot alter the ones that you have in the configuration files. Also, any parameters you pass at runtime via the CLI, for example, with

kedro run --params='{"key": "value"}',

will override the parameters defined in the .yaml files for that run. But those overrides will only apply during the execution and do not persist back to your files.

Are you asking if mutating a parameter in a node will affect other nodes? If so, this is quite complicated to explain, but the short answer is No. In Kedro, we user

MemoryDataset

for any undeclared dataset, and by default

params:

are loaded as MemoryDataset. In memory dataset, you can control how it is copy via a

copy_mode

def _infer_copy_mode(data: Any) -> str:
    # Skip ...
    if pd and isinstance(data, pd.DataFrame) or np and isinstance(data, np.ndarray):
        copy_mode = "copy"
    elif type(data).__name__ == "DataFrame" or ibis and isinstance(data, ibis.Table):
        copy_mode = "assign"
    else:
        copy_mode = "deepcopy"
    return copy_mode

So by default it uses

deepcopy

, this guarantee you cannot mutate the parameters inside parameters. If the question is about dataset, then it depends on what

copy_mode

. For parameters, I don't think there is a way to modify

copy_mode

from user code.

@Nok Lam Chan Thanks! Your explanation answers my question. :)