Massinissa Saïdi
01/27/2023, 4:12 PMsagemaker_entry_point.py
script, example:
...
from pipelines.ml_model.model import train_model
...
def main():
....
regressor = train_model(...)
...
if __name__ == "__main__":
# SageMaker will run this script as the main program
main()
Because I have this error: ModuleNotFoundError: No module named 'pipelines'
Thanks for your help 🙂datajoely
01/27/2023, 4:29 PMMassinissa Saïdi
01/29/2023, 3:25 PMmarrrcin
01/30/2023, 9:11 AMMassinissa Saïdi
01/30/2023, 9:16 AMmarrrcin
01/30/2023, 9:17 AMMassinissa Saïdi
01/30/2023, 11:54 AMClientError: Failed to invoke sagemaker:CreateProcessingJob. Error Details: Invalid image URI massisaidi/testimage. Please provide a valid Amazon Elastic Container Registry path of the Docker image to run.
This is my sagemaker.yml
docker:
image: "massisaidi/testimage"
working_directory: /home/kedro
The image was push correctly on dockerhub
but sagemaker seems to need ECR registry no ?marrrcin
01/30/2023, 11:57 AMMassinissa Saïdi
01/30/2023, 12:03 PMimage_uri (str or PipelineVariable): If specified, the estimator will use this image for training and hosting, instead of the appropriate SageMaker official image based on framework_version and py_version. It can be an ECR url or dockerhub image and tag.
Examples:
<http://123.dkr.ecr.us-west-2.amazonaws.com/my-custom-image:1.0custom-image:latest|123.dkr.ecr.us-west-2.amazonaws.com/my-custom-image:1.0custom-image:latest>.
It seems we can use dockerhub image no ?marrrcin
01/30/2023, 3:27 PMMassinissa Saïdi
01/30/2023, 3:39 PMkedro sagemaker run --tag
or --env
?marrrcin
01/30/2023, 3:40 PMkedro sagemaker -e <name of the env> run
Massinissa Saïdi
01/30/2023, 3:41 PM--tag
?marrrcin
01/30/2023, 3:42 PMMassinissa Saïdi
01/30/2023, 3:42 PM--tag
on my side?marrrcin
01/31/2023, 8:41 AMMassinissa Saïdi
01/31/2023, 10:16 AMpipeline([ node(...,name="split_data_node"),
node(...,name="train_model_node"),
])
When I run in local kedro run --node=split_data_node
everything works. But when I run `kedro sagemaker run --pipeline=MyPipeline`I have this error on sagemaker logs ValueError: Pipeline does not contain nodes named ['split_data_node']
. Does someone know why ? 🙏marrrcin
01/31/2023, 3:09 PMMyPipeline
or the default one?kedro sagemaker run --pipeline=MyPipeline --auto-build -y