https://kedro.org/ logo
#questions
Title
# questions
a

Aayush

09/11/2023, 5:32 AM
I am trying to execute kedro-sagemaker run command but with the help of kedro-airflow. kedro-sagemaker supports kedro-airflow but i cannot find any documentation that can help me understand how to integrate them. I am trying to leverage Amazon managed apache airflow service to orchestrate pipelines created with the help of kedro-sagemaker plugin. Does any know of an optimal way to do this?
m

marrrcin

09/11/2023, 7:42 AM
You probably should just execute single Airflow node that will run kedro sagemaker run in the docker image with your pipeline
a

Aayush

09/11/2023, 8:05 AM
Thanks i am trying to do that only as of now..
@marrrcin when i try to run the container build not from my machine i am getting this.. any idea what can be the issue?
m

marrrcin

09/11/2023, 11:08 AM
You’re trying to execute the
entrypoint
cli command during docker build. That is not intended use of this command and you should not use it outside of Sagemaker context.
Why are you executing it there?
a

Aayush

09/11/2023, 11:45 AM
i had build docker image on my own without using --auto-build but i continued with the same docker file provided to me by sagemaker plugin.. Entrypoint for the container image is kedro sagemaker entrypoint.. there is a comment which says do not change else it will break the integration.. so i didn't change that
the container created by using this image when i am deploying that throws me this error...
m

marrrcin

09/11/2023, 1:16 PM
The entrypoint itself is fine, but when are you executing the container? Is the error from the screen happening in SageMaker or somewhere else?
a

Aayush

09/11/2023, 1:40 PM
its is happening when i am trying to run the container on the docker engine in localhost for testing...
m

marrrcin

09/11/2023, 1:42 PM
kedro sagemaker entrypoint
only works for runs that will happen within SageMaker. If you want to test your pipeline locally with docker, change the entrypoint to
kedro run
instead (you don’t have to modify the Dockerfile, see https://docs.docker.com/engine/reference/run/#entrypoint-default-command-to-execute-at-runtime)
a

Aayush

09/11/2023, 1:44 PM
Thank you for clarifying my issue.. 👍
5 Views