Hi, Is it possible to raise / remove the ceiling for pyarrow, it is currently pinned to <7.0 and we wanted to use kedro with some libraries which needs pyarrow 8. Thank you!

which libraries?

We use optbinning which for some reason gives a segmentation error on pyarrow < 8 in mac but works fine on other OS. We use

pandas.parquertDataset

in our pipeline, I tried with pyarrow == 8 nothing breaks

@Vaibhav feel free to open a PR on https://github.com/kedro-org/kedro-plugins/tree/main/kedro-datasets to bump up the version of pyarrow

I’ve just raised the PR https://github.com/kedro-org/kedro-plugins/pull/100

Is it possible to do this for for 0.17 as well, as we haven’t migrated to 0.18 yet ?

it’s not possible to retroactively change releases like that

yes of course not retroactively changing the release. I was thinking it its a candidate for a patch version, but haven’t seen patch version in kedro yet.

Patching would only really happen for fixing bug issues. Bumping of dependencies doesn’t really warrant a patch unfortunately.

nw - thanks a lot for the help Merel and Joel! We can live with two step install to solve this until we upgrade to 0.18

You’ll also note our move to

kedro-datasets

being a standalone package will make this way way easier in the future as we can bump / change / push releases way faster than corep