hi team! i'm wondering if its possible to have 2 s...
# questions
n
hi team! i'm wondering if its possible to have 2 separate pipeline directories within 1 kedro project and if so how i can go about adding that? here's an example of what i'm after...
Copy code
my_project
├── src
|   ├── use_case_1
|       ├── __init__.py
|       ├── __main__.py
|       ├── pipelines
|       |   ├── use_case_1_data_engineering
|       |   ├── use_case_1_modeling
|   ├── use_case_2
|       ├── __init__.py
|       ├── __main__.py
|       ├── pipelines
|       |   ├── use_case_2_data_ingestion
|       |   ├── use_case_2_data_science
|
y
Hi Nik! AFAIK, the only requirement that Kedro needs is that
my_project/src/my_project
contains: •
pipeline_registry.py
settings.py
main.py
(With double underscores..) •
init.py
(With double underscores..) Then you can arrange the pipelines / nodes / other directories however you prefer. --- Also, given that you have two
main.py
files in your example... Do you want to have 2 different Kedro projects in a single repo and wondering if that's possible? The answer is yes. If you want a single project, but non-standard way to arrange pipeline directories, that's also possible, see my reply above
thankyou 1
j
from a Python packaging perspective, I think there's nothing wrong with having 1 distribution containing N packages. or at least PDM supports it without sweating:
Copy code
❯ tree
.
├── README.md
├── pyproject.toml
└── src
    ├── pdm_multi_a
    │   └── __init__.py
    └── pdm_multi_b
        └── __init__.py

4 directories, 4 files
❯ cat pyproject.toml 
[project]
name = "pdm-multi"
version = "0.1.0"
readme = "README.md"
requires-python = ">=3.11"
dependencies = []

[build-system]
requires = ["pdm-backend"]
build-backend = "pdm.backend"
now, I'm wondering if Kedro has some hardcoded logic to look into a specific path inside
src/
, and I think it does...
@Nik Linnane if you did
kedro pipeline create
with that layout, where would you expect to happen?