Hello, team! Does anyone know the best (or maybe most kedroic) way to work with a PartitionedDataset by processing the partitions individually (merging them would consume all memory). I want to aply the same operations to all partitions. Would it be a better idea to use/add namespaces for this (all my files have the format f"sessions_{YYYY-MM-DD}.parquet")? Thank you!

Someone will reply to you shortly. In the meantime, this might help:

You could try using dataset factories for this https://docs.kedro.org/en/stable/data/kedro_dataset_factories.html

I'll check it out. Thanks!