Tomás Rojas
06/02/2023, 6:15 AMModularPipelineError: Failed to map datasets and/or parameters:
and a list of datasets that do exist on the catalog.
This is the code:
from kedro.pipeline import Pipeline, node
from kedro.pipeline.modular_pipeline import pipeline
from .nodes import select_columns
def create_pipeline(**kwargs) -> Pipeline:
nominal_template = pipeline(
[
node(
func=select_columns,
inputs=["nominal_raw_data_normalized", "params:columns"],
outputs="nominal_raw_data_features",
name="extracting_columns_nominal_data",
namespace="data_preprocessing"
)
]
)
faulty_template = pipeline(
[
node(
func=select_columns,
inputs=[f"fault_{i}_raw_data_normalized", "params:columns"],
outputs=f"fault_{i}_raw_data_features",
name=f"extracting_columns_fault_{i}",
namespace="data_preprocessing"
) for i in range(1, 29)
]
)
reactor_nominal = pipeline(
pipe=nominal_template,
inputs={f"fault_{i}_raw_data_normalized" for i in range(1, 29)},
parameters={"params:columns": "params:reactor_columns"},
namespace="reactor"
)
reactor_faulty = pipeline(
pipe=faulty_template,
inputs={f"fault_{i}_raw_data_normalized" for i in range(1, 29)},
parameters={"params:columns": "params:reactor_columns"},
namespace="reactors"
)
reactor = reactor_nominal + reactor_faulty
return reactor
Any ideas on what is the error? Maybe I am not using the module correctly thanks in advance 🙂Juan Luis
06/02/2023, 1:08 PMJuan Luis
06/02/2023, 1:32 PMkedro run
might include names in that error message of datasets that do exist in the catalog if there's a misnamed dataset in any of the subpipelines.
for example, taking the tutorial: https://docs.kedro.org/en/stable/tutorial/add_another_pipeline.html#optional-extend-the-project-with-namespacing-and-a-modular-pipeline
and changing in the first node the inputs to inputs=["model_input_table_NOT_FOUND", "params:model_options"],
, will raise this error:
ModularPipelineError: Failed to map datasets and/or parameters: model_input_table
so, have a look at your nodes inputs @Tomás Rojas, the error might be there.Juan Luis
06/02/2023, 1:42 PMTomás Rojas
06/03/2023, 4:58 AMTomás Rojas
06/03/2023, 4:59 AMTomás Rojas
06/03/2023, 5:01 AMJuan Luis
06/03/2023, 10:11 PM