Xinghong Fang02/27/2023, 3:14 AM
issue when launching the pipeline. A quick google search bring me to this issue https://stackoverflow.com/questions/34005930/multiprocessing-semlock-is-not-implemented-when-running-on-aws-lambda Looks like AWS Lambda's python runtime is missing
_multiprocessing.SemLock is not implemented
, which seems to be needed by the
Has anyone successfully ran a kedro pipeline on AWS Lambda? Thanks in advance!
Nok Lam Chan02/27/2023, 7:49 AM
Xinghong Fang02/27/2023, 8:35 AM
, I can only get around the issue if I patch the
function (not sure if it is safe to do)
we are using the default sequential runner, but I still feel skeptical about patching out the lock.
from pathlib import Path from unittest.mock import patch def handle_event(event, context): print("================ TEST PATCH ===============") with patch("multiprocessing.Lock"): from kedro.framework.cli.cli import KedroCLI cli_collection = KedroCLI(project_path=Path.cwd()) cli_collection(args=["run"])
Nok Lam Chan02/27/2023, 8:45 AM
shouldn’t have any dependency on
is specified in
Xinghong Fang02/27/2023, 9:29 AM
This code now runs without error. Thanks!
from pathlib import Path def handle_event(event, context): from kedro.framework.cli.cli import KedroCLI cli_collection = KedroCLI(project_path=Path.cwd()) cli_collection(args=["run"])
Nok Lam Chan02/27/2023, 9:30 AM