Hello kedroids !
I have a question about the read...
# questionsm
Massinissa Saïdi
02/08/2023, 1:59 PMHello kedroids !
I have a question about the read priority of credentials files. Suppose I have a
conf/baseconf/prodcredentials.ymlconf/localkedro runconf/local/credentials.ymlconf/base/credentials.ymlkedro run --env prodd
datajoely
02/08/2023, 2:00 PMhey so the answer is on a per key basis, if the same key exists in both. Also a regular kedro run can be thought of as
kedro run --env basedatajoely
02/08/2023, 2:00 PMI think local still wins in every case
m
Massinissa Saïdi
02/08/2023, 2:03 PM"Local wins !"
Thank you 🙂
d
datajoely
02/08/2023, 2:04 PMyou can mess around with your own config loader if you want to change this
m
Massinissa Saïdi
02/08/2023, 2:05 PMBut it seems if I have set nothing in
local/credentials.ymlprod/credentials.ymld
datajoely
02/08/2023, 2:11 PMhmm I wouldn’t expect that
datajoely
02/08/2023, 2:11 PMi.e. the key only exists in prod?
m
Massinissa Saïdi
02/08/2023, 2:19 PMyes, the credentials in local was empty
Massinissa Saïdi
02/08/2023, 2:22 PMand I remove
local/credentials.yml Copy code
No files found in ['/Users/.../conf/base', '/Users/.../conf/local'] matching the glob pattern(s): ('credentials*', 'credentials*/**', '**/credentials*')
warn(f"Credentials not found in your Kedro project config.\n{str(exc)}")Massinissa Saïdi
02/08/2023, 2:28 PMOk its maybe because I read credentials from
KedroSession Copy code
KedreSession.create().load_context()._get_config_credentials()d
datajoely
02/08/2023, 2:41 PMah you need the file there
datajoely
02/08/2023, 2:41 PMbut it can be empty
datajoely
02/08/2023, 2:42 PMor more specifically you need a file under the base or local folder to be present, but no content is needed.
a
Ankita Katiyar
02/08/2023, 2:43 PMIf you do
kedro run --env prodconf/prodconf/local👍 1
Ankita Katiyar
02/08/2023, 2:49 PMEssentially config is always read from the
baselocal--env prodprod👍 1
m
Massinissa Saïdi
02/08/2023, 3:26 PMyes I think my mistake was my checking with this
KedreSession.create().load_context()._get_config_credentials()d
datajoely
02/08/2023, 3:38 PMsorry my mistake @Massinissa Saïdi - thanks @Ankita Katiyar
👍 2
m
Massinissa Saïdi
02/08/2023, 3:58 PMdo you think its possible to get credentials infos depending of environment with a function inside code ? Becase this
KedreSession.create().load_context()._get_config_credentials()d
datajoely
02/08/2023, 3:59 PMso all of this doesn’t happen in the context
datajoely
02/08/2023, 3:59 PMthe resolution is in the ConfigLoader class
m
Massinissa Saïdi
02/08/2023, 4:07 PMwith ConfigLoader class when I read the credentials with env variables it doesnt convert them
d
datajoely
02/08/2023, 5:12 PMthis is how you can do it
https://github.com/kedro-org/kedro/issues/403
🙏 1
13 Views