Hello, I am running a pipeline with versioned data...
# questions
j
Hello, I am running a pipeline with versioned datasets. Can you tell me how I can specify different timestamps for different files while running?
1
d
the timestamp is shared across all files
so you can link them to a particular version
j
I have run pipeline A at timestamp 1, and pipeline B at timestamp 2
You can specify timestamp by dataset
j
The syntax is not clear, have tried many differnent ways:
Copy code
--load-versions=aaaaa:2024-02-06T14.04.36.575Z, bbbb:2024-02-05T18.12.21.682Z, ccccc:2024-02-05T18.12.21.682Z --to-nodes=get_combine_data_for_reporting
where aaaa, bbb, ccc are different datasets
is this possible to achieve? and is the question clear?
d
Why doesn't it work? I assume you get an error? If specifying multiple, I would guess you need to
--load-versions="aaa:...,bbb:...,ccc:..."
j
Copy code
kedro.framework.cli.utils.KedroCliError: Expected the form of 'load_version' to be 'dataset_name:YYYY-MM-DDThh.mm.ss.sssZ',found  instead
Run with --verbose to see the full exception
Error: Expected the form of 'load_version' to be 'dataset_name:YYYY-MM-DDThh.mm.ss.sssZ',found  instead
d
did you put it in quotes?
👍 1
j
Trying that now
I believe this is working, thank you
🙌 3