Ed Henry
10/25/2023, 9:38 PMkedro kubeflow with a repo that has multiple pipelines and lots and lots of parameters in aggregate across pipelines, and I'm receiving an error when trying to upload a compiled manifest : The pipeline spec is invalid.: Invalid input error: The input parameter length exceed maximum size of 10000.
This looks to be a limitation of K8s, specifically : https://github.com/kubeflow/pipelines/issues/2286
I've tried modifying some of the kedro-kubeflow plugin code to account for passing in specific pipelines via kedro kubeflow compile --pipeline <blah> -o <pipeline_blah>.yml and the manifest looks clean but I'm getting another error now : kedro: error: argument --configure_logging: invalid <lambda> value: 'config.yaml' and I was just curious if folks, including @marrrcin, had seen this previously, or not. 🙂Ed Henry
10/25/2023, 9:39 PMEd Henry
10/25/2023, 11:12 PM--config flag on kedro version 0.18.12Ed Henry
10/25/2023, 11:12 PM-c instead fixes it