Hello,
I hope that this finds you well!
'Partitio...
# questionsc
Cyril Verluise
09/22/2023, 3:10 PMHello,
I hope that this finds you well!
'PartitionedDataset' not found
I'm using a PartitionedDataset and have been (successfully) testing the config before but running
kedro run Copy code
DataSetError: Class 'PartitionedDataset' not found or one of its dependencies has not been installed. Copy code
# Raw data
optical_images_raw:
type: PartitionedDataset
path: data/01_raw/optical/
dataset: my_pkg.extras.datasets.image_dataset.ImageDataSet
filename_suffix: .tif
overwrite: True
layer: rawmy_pkg...ImageDatasetK 1
d
datajoely
09/22/2023, 3:10 PMis there a chance there are any issues with the dependencies of the wrapped dataset?
c
Cyril Verluise
09/22/2023, 3:11 PMah! good question! let me check
d
datajoely
09/22/2023, 3:12 PMmy trick with these sorts of issues is to open a notebook and try and do this config with the python api
c
Cyril Verluise
09/22/2023, 3:14 PMis there a chance there are any issues with the dependencies of the wrapped dataset?hmm, does not seem to be the case
my trick with these sorts of issues is to open a notebook and try and do this config with the python apiYes, that's what I meant by "I've (successfully) tested the config before" 😓
d
datajoely
09/22/2023, 3:15 PM😞
datajoely
09/22/2023, 3:15 PMdebugger?
datajoely
09/22/2023, 3:15 PMany chance the CLI is using the wrong env?
c
Cyril Verluise
09/22/2023, 3:22 PMhmm, I get the same error If I use
pandas.CSVDataSet Copy code
optical_images_raw:
type: PartitionedDataset
path: data/01_raw/optical/
dataset: pandas.CSVDataSet
filename_suffix: .csv
overwrite: Truen
Nok Lam Chan
09/22/2023, 3:22 PMDataSetError: Class ‘PartitionedDataset’ not found or one of its dependencies has not been installed.Uhhh, at this point we should be able to surface the underlying error better. Can you try importing the underlying dataset directly from a terminal?
Nok Lam Chan
09/22/2023, 3:22 PMAny problem with
Copy code
import my_pkg.extras.datasets.image_dataset.ImageDataSetc
Cyril Verluise
09/22/2023, 3:23 PMyes, it works
Cyril Verluise
09/22/2023, 3:24 PMit does not seem to come from the dataset, I get the same error with
pandas.CSVDataSetn
Nok Lam Chan
09/22/2023, 3:25 PMwait, can you try
PartitionedDataSetPartitionedDataset👍 1
Nok Lam Chan
09/22/2023, 3:26 PMThis rename happens in the last few months. I am checking which version did we change this now, probably should update the release notes
Nok Lam Chan
09/22/2023, 3:27 PMOkay, I think the oldest version start changing to
PartitionedDatasetNok Lam Chan
09/22/2023, 3:27 PMPartitionedDataSet will still work for the entire 0.18.x series, in 0.19 we will have lower case consistently.
Nok Lam Chan
09/22/2023, 3:28 PMPatching the release note now.
c
Cyril Verluise
09/22/2023, 3:28 PMok, let me try
n
Nok Lam Chan
09/22/2023, 3:28 PMNok Lam Chan
09/22/2023, 3:30 PMAh, the release note is here actually. https://github.com/kedro-org/kedro/blob/main/RELEASE.md#upcoming-deprecations-for-kedro-0190-3
Nok Lam Chan
09/22/2023, 3:30 PMIf you can, maybe just upgrade to 0.18.11+
c
Cyril Verluise
09/22/2023, 3:35 PMyep
Cyril Verluise
09/22/2023, 3:42 PMthanks! working!
d
datajoely
09/22/2023, 3:42 PM😅
c
Cyril Verluise
09/22/2023, 3:46 PMguys, that's wonderful to have a responsive and knowledgeable community! This kind of thing can drive you crazy otherwise.
Big 🤗 to you
thankyou 4
K 4
❤️ 3
🎉 3
Cyril Verluise
09/22/2023, 4:05 PMcc @Roberto P. Palomares (fyi)
🥳 1
4 Views