Gauthier Pierard
11/26/2025, 10:03 AMAbstractDataset predefined currently for polars to delta table?
would something like this do the job?
class PolarsDeltaDataset(AbstractDataset):
def __init__(self, filepath: str, mode: str = "append"):
self.filepath = filepath
self.mode = mode
def _load(self) -> pl.DataFrame:
return pl.read_delta(self.filepath)
def _save(self, data: pl.DataFrame) -> None:
write_deltalake(
self.filepath,
data,
mode=self.mode
)
def _describe(self):
return dict(
filepath=self.filepath,
mode=self.mode
)Laura Couto
11/26/2025, 1:50 PMNok Lam Chan
11/26/2025, 3:49 PMNok Lam Chan
11/26/2025, 3:49 PMNok Lam Chan
11/26/2025, 3:49 PMGauthier Pierard
11/26/2025, 4:34 PM