https://kedro.org/ logo
Powered by
#questions
Title
# questions
c

Cyril Verluise

09/22/2023, 3:10 PM
Hello, I hope that this finds you well! 'PartitionedDataset' not found I'm using a PartitionedDataset and have been (successfully) testing the config before but running 
kedro run
throws the following error.
Copy code
DataSetError: Class 'PartitionedDataset' not found or one of its dependencies has not been installed.
my dataset config is the following
Copy code
# Raw data
optical_images_raw:
    type: PartitionedDataset
    path: data/01_raw/optical/
    dataset: my_pkg.extras.datasets.image_dataset.ImageDataSet
    filename_suffix: .tif
    overwrite: True
    layer: raw
and yes, 
my_pkg...ImageDataset
is well installed. Help? Anyone has ever faced a similar issue? Google did not help much so far? Any fix idea? kedro version: 0.18.10
K 1
d

datajoely

09/22/2023, 3:10 PM
is there a chance there are any issues with the dependencies of the wrapped dataset?
c

Cyril Verluise

09/22/2023, 3:11 PM
ah! good question! let me check
d

datajoely

09/22/2023, 3:12 PM
my trick with these sorts of issues is to open a notebook and try and do this config with the python api
c

Cyril Verluise

09/22/2023, 3:14 PM
is there a chance there are any issues with the dependencies of the wrapped dataset?
hmm, does not seem to be the case
my trick with these sorts of issues is to open a notebook and try and do this config with the python api
Yes, that's what I meant by "I've (successfully) tested the config before" 😓
d

datajoely

09/22/2023, 3:15 PM
😞
debugger?
any chance the CLI is using the wrong env?
c

Cyril Verluise

09/22/2023, 3:22 PM
hmm, I get the same error If I use 
pandas.CSVDataSet
as dataset
Copy code
optical_images_raw:
    type: PartitionedDataset
    path: data/01_raw/optical/
    dataset: pandas.CSVDataSet
    filename_suffix: .csv
    overwrite: True
n

Nok Lam Chan

09/22/2023, 3:22 PM
DataSetError: Class ‘PartitionedDataset’ not found or one of its dependencies has not been installed.
Uhhh, at this point we should be able to surface the underlying error better. Can you try importing the underlying dataset directly from a terminal?
Any problem with
Copy code
import my_pkg.extras.datasets.image_dataset.ImageDataSet
c

Cyril Verluise

09/22/2023, 3:23 PM
yes, it works
it does not seem to come from the dataset, I get the same error with 
pandas.CSVDataSet
n

Nok Lam Chan

09/22/2023, 3:25 PM
wait, can you try 
PartitionedDataSet
instead of 
PartitionedDataset
?
👍 1
This rename happens in the last few months. I am checking which version did we change this now, probably should update the release notes
Okay, I think the oldest version start changing to 
PartitionedDataset
is 0.18.11
PartitionedDataSet will still work for the entire 0.18.x series, in 0.19 we will have lower case consistently.
Patching the release note now.
c

Cyril Verluise

09/22/2023, 3:28 PM
ok, let me try
n

Nok Lam Chan

09/22/2023, 3:28 PM
If you can, maybe just upgrade to 0.18.11+
c

Cyril Verluise

09/22/2023, 3:35 PM
yep
thanks! working!
d

datajoely

09/22/2023, 3:42 PM
😅
c

Cyril Verluise

09/22/2023, 3:46 PM
guys, that's wonderful to have a responsive and knowledgeable community! This kind of thing can drive you crazy otherwise. Big 🤗 to you
thankyou 4
K 4
❤️ 3
🎉 3
cc @Roberto P. Palomares (fyi)
🥳 1
Powered by