I am trying to execute kedro-sagemaker run command but with the help of kedro-airflow. kedro-sagemaker supports kedro-airflow but i cannot find any documentation that can help me understand how to integrate them. I am trying to leverage Amazon managed apache airflow service to orchestrate pipelines created with the help of kedro-sagemaker plugin. Does any know of an optimal way to do this?

You probably should just execute single Airflow node that will run kedro sagemaker run in the docker image with your pipeline

Thanks i am trying to do that only as of now..

You’re trying to execute the

entrypoint

cli command during docker build. That is not intended use of this command and you should not use it outside of Sagemaker context.

i had build docker image on my own without using --auto-build but i continued with the same docker file provided to me by sagemaker plugin.. Entrypoint for the container image is kedro sagemaker entrypoint.. there is a comment which says do not change else it will break the integration.. so i didn't change that

The entrypoint itself is fine, but when are you executing the container? Is the error from the screen happening in SageMaker or somewhere else?

its is happening when i am trying to run the container on the docker engine in localhost for testing...

kedro sagemaker entrypoint

only works for runs that will happen within SageMaker. If you want to test your pipeline locally with docker, change the entrypoint to

kedro run

instead (you don’t have to modify the Dockerfile, see https://docs.docker.com/engine/reference/run/#entrypoint-default-command-to-execute-at-runtime)

Thank you for clarifying my issue.. 👍