Emilio Gagliardi08/06/2023, 2:52 AM
which I can't find in the documentation under 18.12, but under 15.6. Did something change in the way kedro organizes contrib.io? GPT 4 also said that the built-in kedro JSON dataset doesn't work on azure. Any guidance is appreciated. THanks kindly,
my_partitioned_dataset: type: kedro.io.PartitionedDataSet path: <your_blob_folder_path> credentials: azure_blob_storage dataset: type: kedro.contrib.io.azure.JSONBlobDataSet <- is this valid? container_name: <your_container_name> credentials: azure_blob_storage
Deepyaman Datta08/06/2023, 2:01 PM
was removed in Kedro 0.16, along with a lot of storage-specific datasets. I don't know why the
shouldn't work; not sure I would trust GPT 4. See https://stackoverflow.com/a/69941391/1093967 for example;
should be able to handle Azure blob same way as other storage backends.
Emilio Gagliardi08/07/2023, 2:43 AM
Deepyaman Datta08/07/2023, 4:11 AM
since that was similar to the behavior of old
, which also produces a dataframe.
Emilio Gagliardi08/08/2023, 8:35 AM
there is a json object in the underlying file... any ideas greatly appreciated!
Error loading cleaned-emails-20230806003837.json: Failed while loading data from data set JSONDataSet(filepath=cleaned-emails/cleaned-emails-20230806003837.json, protocol=abfs). Expected object or value
Nok Lam Chan08/09/2023, 8:06 PM
Emilio Gagliardi08/14/2023, 6:57 PM
Nok Lam Chan08/14/2023, 8:20 PM