https://kedro.org/ logo
Join Slack
Powered by
Hello, I have a problem with Kedro in AWS, lambda ...
# questions
a
Hello, I have a problem with Kedro in AWS, lambda overwrites the docker user with one with less privileges and does not let me run the project. Any solution? LOGS LAMDA: 2024-01-22T162326.994+00:00 /home/sbx_user1051 2024-01-22T162327.372+00:00 kedro_group 2024-01-22T162327.508+00:00 uid=993(sbx_user1051) gid=990(kedro_group) groups=990(kedro_group) OSError: [Errno 30] Read-only file system: '/home/sbx_user1051/info.log'
n
if you don't need the log, go to 
logging.yml
and remove the file logger. chances are you are probably not getting the 
info.log
from lambda anyway, so there is no point writing that log file.
a
Hi, the creation of logs in files has been removed and I have only left logs in the console, but I still have the same error
version: 1 disable_existing_loggers: true formatters: simple: format: "%(asctime)s - %(name)s - %(levelname)s - %(message)s" json_formatter: format: "%(asctime)s - %(name)s - %(levelname)s - %(message)s" class: pythonjsonlogger.jsonlogger.JsonFormatter handlers: console: class: logging.StreamHandler level: INFO formatter: simple stream: ext://sys.stdout loggers: anyconfig: level: WARNING handlers: [console] propagate: no kedro.io: level: INFO handlers: [console] propagate: no kedro.pipeline: level: INFO handlers: [console] propagate: no root: level: INFO handlers: [console]
n
Can you check if this is updated in your docker?
If you get the same error it suggests that the updated configuration isn't read. Which version of kedro are you running?
If you are > 0.19.0, make sure you configure your logging with the env variable https://docs.kedro.org/en/stable/logging/index.html#how-to-customise-kedro-logging
a
kedro, version 0.18.1
in my docker file add this
ENV KEDRO_LOGGING_CONFIG = /home/sbx_user1051/conf/aws/logging.yml
n
You may need to refer to the older version of docs - https://docs.kedro.org/en/0.18.1/ The env variable doesn't exist in that version
a
Copy code
first use: "kedro package"
then I compile the image
then I upload the image to aws
finally I execute the lambda
n
Can you go inside to your image and inspect the configuration are there?
a
Copy code
I'm going to check the link you sent me and validate if I have something different
👍🏼 1
n
If you try to edit some settings , let say the 
format
of the 
logging.yml
, does it change anything at all?
I am guessing the configuration doesn't exist in the image, thus kedro is using the default. In 0.18 the default with create the 
info.log
, in 0.19 we switch it to opt-in, thus by default there shouldn't be any 
info.log
a
i have checked docker and i see that it does make changes when i modify the file
I see in the docker image that the files and paths are correct.
n
But it wouldn’t change when you remove the info logger?
a
I have deleted the log files, however, I still get this error, I have tried with another project and that one works
n
Maybe it helps to diff the two projects to see if anything catch your attention
Open in Slack
PreviousNext
Powered by