Hi! I'm not sure if this is quite the right place ...
# user-research
f
Hi! I'm not sure if this is quite the right place for something like this, but I created a feature request (https://github.com/kedro-org/kedro/issues/2016) to gauge interest in Avro support for the Kedro
dask.ParquetDataSet
. I had a use case where I needed to parse an existing Avro schema and transform it into a pyarrow schema so that the
<http://dask.to|dask.to>_parquet
function behaves nicely. I was wondering whether this is something the community would be interested in and would appreciate feedback.
d
@Deepyaman Datta is our dask dude - I’m sure he has opinions here!
👍 1
d
Would this be in addition to #1746? I honestly don't know off the top of my head (don't actually use Dask now, just did a but of prototyping in the past) but I can remind myself how the schema parsing works tonight.
f
The idea was for it to be in addition to #1746, yes.. but I haven't looked at the code there in detail (just going off the discussions), so it might be possible to integrate it there instead. It probably should not be part of that same PR though, IMO, as it would blow up the scope a bit.
d
For sure
I will try and take a look over the weekend, sorry for the delay
f
No worries, whenever you've got time.