Jordan
01/19/2023, 4:15 AMpip
and pypa
1st party standards rather than things like Poetry, PDM, PyEnv etcpip-compile
)Juan Luis
01/19/2023, 10:11 AMmamba
, which is blazing fast even for conda-forge
packages).Ian Anderson
01/19/2023, 12:27 PMconda
for environment management and kedro
project dependencies. It was in part because Anaconda has some managed services that helped our public sector org from the security perspective. So there’s always a command that pulls from a managed distribution channel on a server inside the firewall:
conda env create -n new_environment --file src/environment.yml
Not trying to change the subject, I just know that Jordan’s question was fundamental as we were trying to figure out how to do the source code part of reproducible, modular, maintainable data science code.
How do orgs handle security concerns around OSS and tools like pip
?Juan Luis
01/19/2023, 1:53 PMHow do orgs handle security concerns around OSS and tools likeI think possible answers are a) They don't (highly risky) b) They use?pip
--extra-index-url
(not secure at all, see https://scribe.rip/@alex.birsan/dependency-confusion-4a5d60fec610)
c) They have a PyPI proxy and block all access to <http://pypi.org|pypi.org>
(the only secure option)Jordan
01/19/2023, 3:03 PMThis talk I watched a few years ago talked about the different env manager options and covered why I like it’s approach to env managementNice, I’ll check this out
setup.py
is kind of irrelevant. The only thing I use it for is to install my package in editable mode during development. conda develop src
, which just adds the relevant dirs that include __init__.py
to the path, is enough to meet my needs most of the time, but it’s unofficially depreciated these days, so not really a long term solution. So looks like pip install --no-build-isolation --no-deps -e src
would be the best way to install in editable mode if one were using conda to manage all dependencies.Juan Luis
01/19/2023, 3:50 PMJordan
01/19/2023, 3:58 PMI see you’re subscribed to https://github.com/conda/conda-build/issues/4251 as well 😄That’s hilarious, I had no idea I was talking to the same person. Looks like we move in the same circles 😉