I m curious does anyone uses the conda ecosystem not only to

Question

I m curious does anyone uses the conda ecosystem not only to manage environments but also to manage Kedro project dependencies

datajoely · Answer

I m in category 1 not a huge fan of conda for the other stuff This talk I watched a few years ago talked about the different env manager options and covered why I like it s approach to env management <https www youtube com watch v=YKfAwIItO7M>

datajoely · Answer

In general for the packaging dependency stuff we try and stick to `pip` and `pypa` 1st party standards rather than things like Poetry PDM PyEnv etc

datajoely · Answer

Or 1st party complimentary integrations like `pip compile`

Juan Luis · Answer

I m leaning towards pip for installation these days but been a big fan of conda for tricky stuff like gpu enabled deep learning frameworks geospatial stuff and the like I d definitely use conda rather its younger cousin `mamba` which is blazing fast even for `conda forge` packages

Ian Anderson · Answer

We use `conda` for environment management and `kedro` project dependencies It was in part because Anaconda has some managed services that helped our public sector org from the security perspective So there s always a command that pulls from a managed distribution channel on a server inside the firewall `conda env create n new environment file src environment yml` Not trying to change the subject I just know that Jordan s question was fundamental as we were trying to figure out how to do the source code part of reproducible modular maintainable data science code How do orgs handle security concerns around OSS and tools like `pip`

Juan Luis · Answer

gt How do orgs handle security concerns around OSS and tools like `pip` I think possible answers are a They don t highly risky b They use ` extra index url` not secure at all see c They have a PyPI proxy and block all access to `` the only secure option

Jordan · Answer

gt This talk I watched a few years ago talked about the different env manager options and covered why I like it s approach to env management Nice I ll check this out

Jordan · Answer

For most of my personal projects I have no particular need to package them hence `setup py` is kind of irrelevant The only thing I use it for is to install my package in editable mode during development `conda develop src` which just adds the relevant dirs that include ` init py` to the path is enough to meet my needs most of the time but it s unofficially depreciated these days so not really a long term solution So looks like `pip install no build isolation no deps e src` would be the best way to install in editable mode if one were using conda to manage all dependencies

Juan Luis · Answer

I see you re subscribed to <https github com conda conda build issues 4251> as well smile

Jordan · Answer

gt I see you re subscribed to <https github com conda conda build issues 4251> as well smile That s hilarious I had no idea I was talking to the same person Looks like we move in the same circles wink