https://kedro.org/ logo
#questions
Title
# questions
d

Declan Jackson

02/29/2024, 12:00 AM
Hi all, do you know if we will have support for using
partition_cols
when saving parquet datasets? At the moment custom save and load methods are required which is quite cumbersome. P.S. in the documentation here (pictured below) it has YAML examples using
partition_on
even though this is not supported.
d

datajoely

02/29/2024, 8:00 AM
It’s just a thin wrapper over
<http://pd.DataFrame.to|pd.DataFrame.to>_parquet
isn’t this available ? https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.to_parquet.html
d

Declan Jackson

03/01/2024, 1:28 AM
but is this possible to use without defining a custom dataset?
d

datajoely

03/01/2024, 11:45 AM
It should’ve possible today do you get an error?
j

Juan Luis

03/01/2024, 12:25 PM
hi @Declan Jackson, as far as I understand, you could specify
Copy code
ds:
  type: pandas.ParquetDataset
  save_args:
    partition_cols: ...
could you try?
in the documentation here (pictured below) it has YAML examples using
partition_on
even though this is not supported.
what do you mean? does it give an error?