Hey Team, QQ regarding versioning. I think I am c...
# questions
Hey Team, QQ regarding versioning. I think I am clear regarding versioned datasets. Searching the docs I could not find anything regarding versioned parameters. Given that I trigger a pipeline run, I create versioned datasets (if I choose to do so), but can I also create a versioned save of the used parameters (from one or more
files?) Or am I thinking about this the wrong way and there is a good reason that this is not possible? Thanks!
There's no automated way to do this, but
is an automatically-created
, and you can persist the data to some physical versioned data set. Alternatively, you can use something like
to log parameters for runs.
Thanks for the advice! Say I want to go for the first option: how do I do that? I seem to be unable to add it to the catalogue with a filepath:
Copy code
__init__() got an unexpected keyword argument 'filepath'.
DataSet 'parameters' must only contain arguments valid for the constructor of 'kedro.io.memory_dataset.MemoryDataSet'.
You would have to create a node to write it to a dataset (e.g. with catalog entry name
); you cannot control the
catalog entry itself.
(well, I guess you can, by modifying the catalog Kedro creates and creating a
out of it or something, but unless you have a strong requirement for doing that, let's do it the simple way)