https://kedro.org/ logo
#questions
Title
# questions
a

Andres Alvarado

01/22/2024, 10:34 PM
Hello, I have a problem with Kedro in AWS, lambda overwrites the docker user with one with less privileges and does not let me run the project. Any solution? LOGS LAMDA: 2024-01-22T162326.994+00:00 /home/sbx_user1051 2024-01-22T162327.372+00:00 kedro_group 2024-01-22T162327.508+00:00 uid=993(sbx_user1051) gid=990(kedro_group) groups=990(kedro_group) OSError: [Errno 30] Read-only file system: '/home/sbx_user1051/info.log'
n

Nok Lam Chan

01/22/2024, 11:52 PM
if you don't need the log, go to
logging.yml
and remove the file logger. chances are you are probably not getting the
info.log
from lambda anyway, so there is no point writing that log file.
a

Andres Alvarado

01/23/2024, 6:29 PM
Hi, the creation of logs in files has been removed and I have only left logs in the console, but I still have the same error
version: 1 disable_existing_loggers: true formatters: simple: format: "%(asctime)s - %(name)s - %(levelname)s - %(message)s" json_formatter: format: "%(asctime)s - %(name)s - %(levelname)s - %(message)s" class: pythonjsonlogger.jsonlogger.JsonFormatter handlers: console: class: logging.StreamHandler level: INFO formatter: simple stream: ext://sys.stdout loggers: anyconfig: level: WARNING handlers: [console] propagate: no kedro.io: level: INFO handlers: [console] propagate: no kedro.pipeline: level: INFO handlers: [console] propagate: no root: level: INFO handlers: [console]
n

Nok Lam Chan

01/23/2024, 6:32 PM
Can you check if this is updated in your docker?
If you get the same error it suggests that the updated configuration isn't read. Which version of kedro are you running?
If you are > 0.19.0, make sure you configure your logging with the env variable https://docs.kedro.org/en/stable/logging/index.html#how-to-customise-kedro-logging
a

Andres Alvarado

01/23/2024, 6:35 PM
kedro, version 0.18.1
in my docker file add this
ENV KEDRO_LOGGING_CONFIG = /home/sbx_user1051/conf/aws/logging.yml
n

Nok Lam Chan

01/23/2024, 6:37 PM
You may need to refer to the older version of docs - https://docs.kedro.org/en/0.18.1/ The env variable doesn't exist in that version
a

Andres Alvarado

01/23/2024, 6:38 PM
Copy code
first use: "kedro package"
then I compile the image
then I upload the image to aws
finally I execute the lambda
n

Nok Lam Chan

01/23/2024, 6:40 PM
Can you go inside to your image and inspect the configuration are there?
a

Andres Alvarado

01/23/2024, 6:40 PM
Copy code
I'm going to check the link you sent me and validate if I have something different
👍🏼 1
n

Nok Lam Chan

01/23/2024, 6:40 PM
If you try to edit some settings , let say the
format
of the
logging.yml
, does it change anything at all?
I am guessing the configuration doesn't exist in the image, thus kedro is using the default. In 0.18 the default with create the
info.log
, in 0.19 we switch it to opt-in, thus by default there shouldn't be any
info.log
a

Andres Alvarado

01/23/2024, 9:15 PM
i have checked docker and i see that it does make changes when i modify the file
I see in the docker image that the files and paths are correct.
n

Nok Lam Chan

01/23/2024, 10:18 PM
But it wouldn’t change when you remove the info logger?
a

Andres Alvarado

01/29/2024, 9:52 PM
I have deleted the log files, however, I still get this error, I have tried with another project and that one works
n

Nok Lam Chan

01/30/2024, 11:05 AM
Maybe it helps to diff the two projects to see if anything catch your attention