Ed Henry
10/25/2023, 9:38 PMkedro kubeflow
with a repo that has multiple pipelines and lots and lots of parameters in aggregate across pipelines, and I'm receiving an error when trying to upload a compiled manifest : The pipeline spec is invalid.: Invalid input error: The input parameter length exceed maximum size of 10000.
This looks to be a limitation of K8s, specifically : https://github.com/kubeflow/pipelines/issues/2286
I've tried modifying some of the kedro-kubeflow plugin code to account for passing in specific pipelines via kedro kubeflow compile --pipeline <blah> -o <pipeline_blah>.yml
and the manifest looks clean but I'm getting another error now : kedro: error: argument --configure_logging: invalid <lambda> value: 'config.yaml'
and I was just curious if folks, including @marrrcin, had seen this previously, or not. 🙂--config
flag on kedro version 0.18.12-c
instead fixes it