Mark Einhorn
11/06/2025, 12:30 PMdev env, but when deploying and running in test , we are getting the following error:
DatasetError: Failed while loading data from dataset ParquetDataset(filepath=psi-test-data/***********/data/02_intermediate/formatted_transactions_df.parquet, load_args={}, protocol=s3, save_args={}).
An error occurred (PreconditionFailed) when calling the GetObject operation: At least one of the pre-conditions you specified did not hold
What's weird is that the error is not consistent. Sometimes the node responsible runs through just fine, other times it errors out, without anything (at least what we can see) changing. Any help would be massively appreciated! @Tom McHaleNok Lam Chan
11/06/2025, 1:02 PMNok Lam Chan
11/06/2025, 1:06 PMfsspec / boto version to see if this solve the problem. Could also be some weird race condition with multiple runs, cache on fsspec when listing directory