Hi guys, My Poetry venvs have been failing to buil...
# questions
g
Hi guys, My Poetry venvs have been failing to build all morning. Looks like some dependency issues. They did however work fine on Friday. After some trial and error, I found that removing Kedro-viz from my toml file and rebuilding the venv seems to have fixed it. Have you had any other similar issues reported lately?
d
I’m not familiar with Poetry, is there a lock file you can post so we can see where the conflicts are?
g
I can pipe a verbose installation log file into the chat. Give me a few mins 🙂
Apologies for the delay. Assuming I have a working venv, if I then try to install kedro-viz (
poetry add kedro-viz
) you can see on line 17569 where the issues are occurring when it tries to install
botocore
.
d
which version of Kedro and Kedro Viz?
g
Here are my versions of kedro and kedro-datasets. I can't install kedro-viz, however
poetry add kedro-viz
looks for the most recent compatible version, which we can see from the txt file I sent over
d
Kedro-Datasets requires very old versions of s3fs, if you're using a dataset that requires it: https://github.com/kedro-org/kedro-plugins/blob/kedro-datasets-1.8.0/kedro-datasets/setup.py#L9 By chance, are you using something like
SparkDataset
extra? (Not 100% saying that's the root cause of the issue, but I do see it trying a wide range of
s3fs
versions, including those outside the tight Kedro-Datasets bounds.)
d
Also Kedro Viz just introduced it
@Tynan sharable URLs may have caused some conflicts
d
Ah, right, and even the bottom end of that range requires CalVered s3fs: https://github.com/fsspec/s3fs/blob/2021.04.0/requirements.txt#L2
There's already an issue to track this: https://github.com/kedro-org/kedro/issues/1995 As well as a couple attempts from @Matthias Roels and myself, but it hasn't been entirely trivial, and so not yet done. In the meantime, may need to revert to an older Viz.
g
Great, I'll keep an eye on this issue for updates. Thanks for the good work guys! thankyou