https://kedro.org/ logo
#questions
Title
# questions
s

Siavash Sakhavi

01/22/2024, 1:50 PM
Hi Kedro team, Now that you've moved everything to project.toml, what not propose using a tool like PDM to manage dependencies within the TOML file so we remove the requirements file as well? Just recently I was cleaning my project with PDM and it was a calm for the chaos.
K 1
d

datajoely

01/22/2024, 1:51 PM
Anything that supports PEP621 is good in my eyes! https://peps.python.org/pep-0621/
👍 1
👍🏼 1
so PDM , Poetry not yet…
overall we nominally recommend sticking to Python 1st party tools as much as possible so that would be anything maintained by the PyPA https://www.pypa.io/en/latest/
s

Siavash Sakhavi

01/22/2024, 2:09 PM
Can be anything that supports and respects TOML.
Poetry is also an option
d

datajoely

01/22/2024, 2:11 PM
Poetry still doesn’t support the Python standard https://github.com/python-poetry/roadmap/issues/3
and they’ve been working on it 3/4 years since it’s been signed off
j

Juan Luis

01/22/2024, 2:15 PM
I personally create most of my Kedro projects with PDM already 😄 and I wrote a small tool that can turn any PEP 621-compliant library structure to a Kedro project https://github.com/astrojuanlu/kedro-init @Siavash Sakhavi when you mean "propose using a tool like PDM", do you meen our
kedro new
+ starters?
s

Siavash Sakhavi

01/22/2024, 2:17 PM
@Juan Luis mainly as a best practice for dependency management. I was thinking in the docs.
👍🏼 1
My current project was in chaos because everyone was pip installing and suddenly somebody decided to do a pip freeze to the requirement file. I totally lost control of all the dependencies and the new libraries added were causing my pipeline to not even run.
😱 1
j

Juan Luis

01/22/2024, 2:21 PM
typical horror story from Python packaging... 😅 any opinions on Poetry vs PDM vs Hatch?
💯 1
s

Siavash Sakhavi

01/22/2024, 2:25 PM
I've seen the other two. Didn't use them extensively. I just have a bad memory of using poetry first time I used it. I eventually settled on PDM. Any tool that can help is welcome
I am also looking for a pre-commit hook that checks if the packages in your environment are different from the lock file.
j

Juan Luis

01/22/2024, 2:36 PM
I'm personally very happy with PDM 👍🏼
i

Iñigo Hidalgo

01/22/2024, 2:39 PM
I will add a +1 for PDM, it fits all my needs perfectly, the developer is super responsive, and since it’s PEP621 compliant, switching to any other tool would be mostly a nonissue. @Juan Luis what does your kedro-init tool actually do? I can’t really get it from the readme.
j

Juan Luis

01/22/2024, 2:42 PM
@Iñigo Hidalgo it adds the appropriate
[tool.kedro]
section to
pyproject.toml
, creates
settings.py
, and
pipeline_registry.py
, and creates
conf/{base,local}
🎉 1
d

datajoely

01/22/2024, 7:08 PM
j

Juan Luis

01/23/2024, 8:25 AM
I miss Pedram shitposts so much
d

datajoely

01/23/2024, 9:34 AM
you can follow him on Linkedin if you’d like
@Juan Luis I wonder if we could think about your
kedro-init
as either 1. A recipe workflow 2. A kedro-plugin
j

Juan Luis

01/23/2024, 10:08 AM
maybe 🙂 but it should follow as the conclusion of https://github.com/kedro-org/kedro/issues/2512 (still to be closed)
d

datajoely

01/23/2024, 10:23 AM
the more I think about it - it would be really cool to make it easy for people to define and share add-ons
4 Views