Hello <@U045L91RV9D> &amp; Kedro community I'm c...
# plugins-integrations
Hello @marrrcin & Kedro community I'm currently using the Kedro-SageMaker plugin to run a Kedro pipeline on SageMaker. My Kedro project has multiple parameters (using several parameters YAML files). While executing
kedro sagemaker run -i <image name>
, I encountered this error:
ClientError: ValidationError for UpdatePipeline operation: Unable to parse pipeline definition. Model validation failed - container ProcessingEnvironmentMap length (132) exceeds maximum limit (100).
In my scenario, the number of parameters surpasses the 100 limits. How can I best manage this situation? Is there an option in SageMaker to increase the parameter limit? Looking forward to your suggestions.
Hi @Vijay Jawaharlal, I’m happy that you’re using our plugin! Please add an issue on our GitHub and explain it in more details - preferably with some example project so that we could replicate it. Thanks!
👍 1
I have added an issue in your repo. Unfortunately, I can't share my code. but it is easy to reproduce with spaceflight example. Here are the steps to reproduce the issue. - Add a dummy_parameter.yml file under conf/base/parameters/ in the kedro space flight example code. (check dummy_pamarameter.yml file content below) - Simply run
kedro sagemaker run --auto-build
dummy_parameter.yml file content:
Copy code
  parameter1: 1
  parameter2: 2
  parameter3: 3
  parameter4: 4
  parameter5: 5
 parameter100: 100
Great, thank you! We will update you on GH