hi, does kedro support out of the box google cloud...
# questionsa
Artur Dobrogowski
11/15/2024, 2:42 PMhi, does kedro support out of the box google cloud logging lib in logging formats? I am not clear on the documentation how advanced adding custom handlers is possible or do I have to do it manually? When would it be better to initialize? After or before kedro loading?
h
Hall
11/15/2024, 2:42 PMSomeone will reply to you shortly. In the meantime, this might help:
a
Artur Dobrogowski
11/15/2024, 2:43 PMThis is what I'm refering to https://cloud.google.com/python/docs/reference/logging/latest/std-lib-integration
Artur Dobrogowski
11/15/2024, 2:44 PMah the logging format in logging.yml seems to be the dictConfig format which can also be used to configure google logger, looks like the answer is yes.
Artur Dobrogowski
11/15/2024, 2:44 PMI wasn't aware that it was the standard https://docs.python.org/3/library/logging.config.html#logging-config-dictschema
d
datajoely
11/15/2024, 2:48 PMYeah I think you can just swap the handler!
datajoely
11/15/2024, 2:48 PMif you get it running it would be great if you can show us what your config looks like?
a
Artur Dobrogowski
11/15/2024, 2:50 PMThere's a catch though:
Copy code
Note that since CloudLoggingHandler requires an already initialized Client, you must initialize a client and include it in the dictConfig entry for a CloudLoggingHandler.Artur Dobrogowski
11/15/2024, 2:51 PMSo it looks like it would be doable with omegaconfigloader hook in that place - does reading logging conf support omegaconf?
d
datajoely
11/15/2024, 2:52 PMHmm I'm not sure
datajoely
11/15/2024, 2:52 PMlet me ask the team
👍 1
a
Artur Dobrogowski
11/15/2024, 3:59 PMdid you get an answer? I'll just run a quick check
d
datajoely
11/15/2024, 4:04 PMnot yet
datajoely
11/15/2024, 4:04 PMmay be a next week thing
a
Artur Dobrogowski
11/15/2024, 4:04 PMlooks like it does not support it
Copy code
raise ValueError('Unable to configure handler '
ValueError: Unable to configure handler 'console2'
ValueError: Unable to set formatter '${foo}'
ValueError: Unable to set formatter '${foo:}'Artur Dobrogowski
11/15/2024, 4:05 PM^ attempted to add foo handler to omegaconf
n
Nok Lam Chan
11/18/2024, 12:50 PMLogging does not support omegaconf out of the box, you can however add additional handler or update it in hooks.
from kedro.framework.project import LOGGING
👍 1
a
Artur Dobrogowski
11/18/2024, 12:55 PMDoes it make sense to update it to enable omegaconf handling of loading that yaml?
n
Nok Lam Chan
11/18/2024, 1:46 PMWe actually went through the opposite path,
logging.ymlparameterscatalog