https://kedro.org/ logo
Join Slack
Powered by
Hi folks I'm trying to create a custom kedro datas...
# questions
p
Hi folks I'm trying to create a custom kedro dataset (inherited from AbstractVersionedDataSet) I have to write a few tests similar to the ones existing for CsvDataSet However I'm witnessing that a few tests are failing. Need some advice around the same. happy to hop on a call to discuss the details
j
hi @Pranav Khurana! what version of 
kedro
and 
kedro-datasets
are you using, and what tests are failing?
p
kedro version 
0.18.8
test_versioning_existing_dataset
basically, according to this, I should get an error when I try to overwrite an unversioned dataset with a versioned dataset
but for my custom dataset
I'm not getting this error
j
p
Yes I'm inheriting from AbstractVersionedDataset
j
how does your 
_save
method look like?
p
Precisely Juan Both the functions you mentioned, raise error for different reasons, for the path being invalid and for overwriting an already existing dataset with a new one
However, when I try to replicate these tests in my custom dataset , they’re not working as expected
i.e my custom dataset is allowing data files to be overwritten and it is also allowing passing an invalid path
which is why I’m confused because both, the CsvDataSet and my Custom dataset are inheriting the AbstractVersionedDataset class
Maybe there’s something missing in my code
j
Maybe there’s something missing in my code
looks like it. for example, passing an invalid path would fail in the 
get_protocol_and_path
function that CSVDataSet uses: https://github.com/kedro-org/kedro-plugins/blob/031b737337ac2023e29009142a156d1f8969a31e/kedro-datasets/kedro_datasets/pandas/csv_dataset.py#L112 unfortunately creating a dataset currently requires some copy-pasting https://github.com/kedro-org/kedro/issues/1936#issuecomment-1597474696
p
I figured it out!
Thanks a lot juan, this was really helpful
j
🙌🏼 any time @Pranav Khurana! happy that you got it solved
p
Could you also please explain just one more functionality to me ?
the 
test_catalog_release
function
can't figure out what it's doing
23 Views
Open in Slack
PreviousNext
Powered by