https://kedro.org/ logo
#questions
Title
# questions
g

Gary McCormack

11/14/2023, 11:33 AM
Hi guys, My Poetry venvs have been failing to build all morning. Looks like some dependency issues. They did however work fine on Friday. After some trial and error, I found that removing Kedro-viz from my toml file and rebuilding the venv seems to have fixed it. Have you had any other similar issues reported lately?
d

datajoely

11/14/2023, 11:33 AM
I’m not familiar with Poetry, is there a lock file you can post so we can see where the conflicts are?
g

Gary McCormack

11/14/2023, 11:34 AM
I can pipe a verbose installation log file into the chat. Give me a few mins 🙂
Apologies for the delay. Assuming I have a working venv, if I then try to install kedro-viz (
poetry add kedro-viz
) you can see on line 17569 where the issues are occurring when it tries to install
botocore
.
d

datajoely

11/14/2023, 1:33 PM
which version of Kedro and Kedro Viz?
g

Gary McCormack

11/14/2023, 1:36 PM
Here are my versions of kedro and kedro-datasets. I can't install kedro-viz, however
poetry add kedro-viz
looks for the most recent compatible version, which we can see from the txt file I sent over
d

Deepyaman Datta

11/14/2023, 1:52 PM
Kedro-Datasets requires very old versions of s3fs, if you're using a dataset that requires it: https://github.com/kedro-org/kedro-plugins/blob/kedro-datasets-1.8.0/kedro-datasets/setup.py#L9 By chance, are you using something like
SparkDataset
extra? (Not 100% saying that's the root cause of the issue, but I do see it trying a wide range of
s3fs
versions, including those outside the tight Kedro-Datasets bounds.)
d

datajoely

11/14/2023, 1:54 PM
Also Kedro Viz just introduced it
@Tynan sharable URLs may have caused some conflicts
d

Deepyaman Datta

11/14/2023, 2:10 PM
Ah, right, and even the bottom end of that range requires CalVered s3fs: https://github.com/fsspec/s3fs/blob/2021.04.0/requirements.txt#L2
There's already an issue to track this: https://github.com/kedro-org/kedro/issues/1995 As well as a couple attempts from @Matthias Roels and myself, but it hasn't been entirely trivial, and so not yet done. In the meantime, may need to revert to an older Viz.
g

Gary McCormack

11/14/2023, 2:46 PM
Great, I'll keep an eye on this issue for updates. Thanks for the good work guys! thankyou