Hello everyone! I have a kedro project running ke...
# questions
a
Hello everyone! I have a kedro project running kedro 0.18.14 and recently upgraded it to the latest kedro 0.19.6. During testing if everything went ok during the upgrade, I found weird issue with rich logging - warning are not "rich"ified anymore. I tried to remove logging config - warnings became "rich"ified again. There is nothing sophisticated in the logging config, it's pretty simple. This specific warning on screenshots comes from xgboost package, but I get exactly same issue with warnings from kedro-datasets and another packages as well. It was working fine with 0.18.14. Any ideas what could it be? Thanks!
j
hello @Alexey Gravanov! thanks for the detailed writeup, we're looking into this and will get back to you soon
a
@Juan Luis thanks for looking into it! found another example. my logging config is in place, one warning coming from kedro-viz is "richified", another one, from kedro itself - not. if I remove logging.yml, both are "richified".
r
Would u be able to raise this as a bug request on github and we will have one of the team members look into this issue. Thanks
a
sure, will do it!
thankyou 1
n
Hmm
Which version is the "expected behavior"?
just looking at the
logging.yml
, it looks pretty outdated so I think that's the root cause.
and where do you place your
logging.yml
?
The new default is under
conf/logging.yml
, otherwise you need to specify environment variable
KEDRO_LOGGING_CONFIG
to point to the config file (or it used the default)
a
@Nok Lam Chan • it's
conf/logging.yml
• i would expect behavior like in 0.18.14, all warnings should be "richified" • hm, what makes it "pretty outdated"? that I'm not listing every and all packages explicitly? shouldn't it fall back to the root one in this case? why would warning from kedro-viz be richified and the one from kedro and xgboost - not?
n
Hi @Alexey Gravanov, so there is two main things affect the logging settings. 1. Default setting from
kedro
2.
logging.yml
that override the default
loggers:
anyconfig:
level: WARNING
handlers: [rich, info_file_handler, error_file_handler]
propagate: no
I see this in your logging.yml and this settings is <18.1, as you can see from the link below we don't specify
anyconfig
anymore for a while. https://github.com/kedro-org/kedro-starters/blob/0.18.2/pyspark/%7B%7B%20cookiecutter.repo_name%20%7D%7D/conf/base/logging.yml
• i would expect behavior like in 0.18.14, all warnings should be "richified"
This should still be the case, where are those WARNINGS come from?
Copy code
loggers:
  kedro:
    level: INFO

  pet_project:
    level: INFO
If you focus at the
loggers
, this is where the
logging.yml
will have impact on. This is done on purpose because Kedro will not change the logging settings for other package by default. The reason for this is this could have unpredicted effect and produce noisy logging.
^ That's also why you only see
WARNING
from external package, but you see
INFO
from your current project.
a
Check my messages above, they say, where they come from. the one which is getting richified - comes from kedro-viz. two others, which are not getting richified - come from kedro itself and from xgboost. none of this has anything to do with anyconfig.
n
Sorry maybe I didn't make myself clear. I am not suggesting this has to do with
anyconfig
, I am simply saying that I see
anyconfig
is still in your logging.yml, which gives me information that this is a pretty old
logging.yml
inherit from <0.18.2 projects. There are more difference, for example you can see the later version of
logging.yml
doesn't change
root
level anymore.
a
Apologies for dropping the discussion for several days - had holidays. I tried to put loggers as you suggested and it didn't changed anything. I attached the config I used, it looks pretty much as default one except error handler. I ran a node with a logging config placed under
conf/logging.yml
and then I deleted
conf/ogging.yml
and ran same node once again. Results are on the screenshot. Sorry if I didn't make myself clean. I do not have any problems with visibility of warning messages, all messages I want to see are there, regardless if logging config is there or not. What I have problems with is the formatting of specific warning message with rich handler. When my
conf/logging.yml
is there, a warning message is printed out in a clear text. When I have no
conf/logging.yml
, exactly same warning is printed formatted with rich logging handler. See the area on the screenshot in a red rectangles. Exactly same warning message, but formatted different way.
logging.png,logging.yml
This warning message I'm having problem with is printed out by kedro, not somewhere in my code.