https://kedro.org/ logo
#questions
Title
# questions
p

Pranav Khurana

06/20/2023, 11:32 AM
Hi folks I'm trying to create a custom kedro dataset (inherited from AbstractVersionedDataSet) I have to write a few tests similar to the ones existing for CsvDataSet However I'm witnessing that a few tests are failing. Need some advice around the same. happy to hop on a call to discuss the details
j

Juan Luis

06/20/2023, 11:32 AM
hi @Pranav Khurana! what version of
kedro
and
kedro-datasets
are you using, and what tests are failing?
p

Pranav Khurana

06/20/2023, 11:34 AM
kedro version
0.18.8
test_versioning_existing_dataset
basically, according to this, I should get an error when I try to overwrite an unversioned dataset with a versioned dataset
but for my custom dataset
I'm not getting this error
p

Pranav Khurana

06/20/2023, 11:52 AM
Yes I'm inheriting from AbstractVersionedDataset
j

Juan Luis

06/20/2023, 11:55 AM
how does your
_save
method look like?
p

Pranav Khurana

06/20/2023, 5:35 PM
Precisely Juan Both the functions you mentioned, raise error for different reasons, for the path being invalid and for overwriting an already existing dataset with a new one
However, when I try to replicate these tests in my custom dataset , they’re not working as expected
i.e my custom dataset is allowing data files to be overwritten and it is also allowing passing an invalid path
which is why I’m confused because both, the CsvDataSet and my Custom dataset are inheriting the AbstractVersionedDataset class
Maybe there’s something missing in my code
j

Juan Luis

06/20/2023, 9:32 PM
Maybe there’s something missing in my code
looks like it. for example, passing an invalid path would fail in the
get_protocol_and_path
function that CSVDataSet uses: https://github.com/kedro-org/kedro-plugins/blob/031b737337ac2023e29009142a156d1f8969a31e/kedro-datasets/kedro_datasets/pandas/csv_dataset.py#L112 unfortunately creating a dataset currently requires some copy-pasting https://github.com/kedro-org/kedro/issues/1936#issuecomment-1597474696
p

Pranav Khurana

06/21/2023, 7:49 AM
I figured it out!
Thanks a lot juan, this was really helpful
j

Juan Luis

06/21/2023, 7:51 AM
🙌🏼 any time @Pranav Khurana! happy that you got it solved
p

Pranav Khurana

06/21/2023, 12:30 PM
Could you also please explain just one more functionality to me ?
the
test_catalog_release
function
can't figure out what it's doing
17 Views