https://kedro.org/ logo
#questions
Title
# questions
m

Marc Gris

08/11/2023, 12:45 PM
Hi again, Speaking of
kedro ipython
: Is there a way to specify /override the
conf_source
? 🙂 Thx
f

Flavien

08/11/2023, 12:59 PM
👋 , if you use
--env
, you can use the environment you wish.
a

Ankita Katiyar

08/11/2023, 1:03 PM
From
CONF_SOURCE
in
settings.py
, I believe
f

Flavien

08/11/2023, 1:04 PM
My bad, I misunderstood the question.
m

Marc Gris

08/11/2023, 1:05 PM
@Flavien: No worries. Thx anyway 🙂
@Ankita Katiyar Thx. That’s how I did it so far. But not really convenient. My problem comes from the fact that we have a single pipeline / repo for all our clients. To better enforce “tenant isolation” I have organized
conf
and
data
as follow:
Copy code
conf
└── client_A
    ├── base
    └── local
└── client_B
    ├── base
    └── local
Copy code
data
├── client_A
│   └── 01_raw
│   └── 02_intermediate
│   └── ...
└── client_B
│   └── 01_raw
│   └── 02_intermediate
│   └── ...conf
I did so because I liked the idea of not being able to run the pipeline without being explicit about which client it should be done for… (the structure above makes it so that
kedro run
without
--conf-source
will raise an error) Another reason I did so (if I recall correctly) was to avoid having “duplicate keys” conflicts in the different yaml files across clients…
n

Nok Lam Chan

08/11/2023, 1:37 PM
I think this should be available in the
reload_kedro
magic, similarly you can do
reload_kedro --env=prod
(which we haven’t added)
@Ankita Katiyar Can you help to draft an issue?
a

Ankita Katiyar

08/11/2023, 1:42 PM
@Nok Lam Chan Yep!
thankyou 1
m

Marc Gris

08/11/2023, 2:31 PM
Thx @Nok Lam Chan & @Ankita Katiyar Another similar issue (which I’ll leave here since related) With the structure above, I can’t use
kedro viz
It doesn’t have a
conf-source
param and
kedro viz --env client_A/base
does not work. Am I just being “a pain” or does this sound like a legit expectation 🙂
n

Nok Lam Chan

08/11/2023, 2:38 PM
I think this is valid - though in this case settings.py would still be a workaround for now.
m

Marc Gris

08/11/2023, 2:46 PM
Thx Nok. Indeed, for now, I’ll specify the conf_source with setting.py 🙂
👍🏼 1
👍 1
n

Nok Lam Chan

08/11/2023, 2:54 PM
feel free to open an issue in viz repo https://github.com/kedro-org/kedro-viz/issues
m

Marc Gris

08/11/2023, 3:01 PM
Thx. Will do.
11 Views