https://kedro.org/ logo
#questions
Title
# questions
n

noam

05/04/2023, 2:55 PM
Hi Kedro users, Does anyone know how one would implement versioning for a PartitionedDataSet? In other words, does anyone have a convenient solution for enabling versioning (i.e. setting “versioned: True” in the data catalog) for a PartitionedDataSet the same way one can for a PickleDataSet? The following is my code in conf/data/local/catalog.yml:
Copy code
validation_data:
  type: kedro.io.PartitionedDataSet
  path: data/03_primary/validation_data/
  dataset: pickle.PickleDataSet
  filename_suffix: ".df"
  versioned: True
Thanks in advance!
a

Antony Milne

05/04/2023, 4:33 PM
I think this might not be possible - @Deepyaman Datta will know though 😄
d

Deepyaman Datta

05/04/2023, 8:02 PM
As @Antony Milne mentioned, this isn't currently feasible, although that PR works. I think (with sufficient interest) this could be revisited; TBH I'm not 100% sure why it got stuck, since it was just around some decisions for implementation.