Hello, I upgraded to kedro 1.0 accepting the sugge...
# questions
s
Hello, I upgraded to kedro 1.0 accepting the suggestion from Kedro CLI output. A lot of things broke down, which I could put to work thanks to the documentation (0.19 > 1.0), including changing the internal kedro library function from catalog.filter() to catalog.list(). However, now I cannot follow through with other functionalities such as 'namespaces'. Is there a support for namespaces in 1.0? If yes, how?
r
Hi @Shah, namespaces are still supported. What support are you looking for ? You can follow our docs for any info - https://docs.kedro.org/en/1.0.0/build/namespaces/ We also have migration docs for reference - https://docs.kedro.org/en/1.0.0/about/migration/#using-kedro-as-a-framework For the namespace part, you can find some info in the migration doc.
Copy code
--namespace argument in kedro run command was removed in favour of --namespaces which accepts multiple namespaces. If you used the --namespace argument, you need to change it to --namespaces and pass a comma-separated list of namespaces.
Let us know if you need further information. Thank you
s
Hi Ravi, Thanks for the reply with link. I was following the doc https://docs.kedro.org/en/1.0.0/tutorials/add_another_pipeline/#how-it-works-the-modular-arguments-in-pipeline Here the method looked pretty straightforward, and I wrote the following code to apply namespaces with individual inputs.
Copy code
ds_pipeline_1 = pipeline(
        pipe=pipeline_instance,
        inputs="presence_motion_co2_combined_cleaned_neg_removed",
        namespace="active_modelling_pipeline",
    )
    ds_pipeline_2 = pipeline(
        pipe=pipeline_instance,
        inputs="presence_motion_co2_combined_cleaned_neg_zeroed",
        namespace="disabled_modelling_pipeline",
    )

    return ds_pipeline_1 + ds_pipeline_2
However, it didn't work. Now, following the documentation that you've pointed. In this case, the namespaces are defined in a different way, and seems its overlapping with the functionality of 'extending a base pipeline'. I mean, IMHO it seems as if namespaces do not have much utility if we are to extend base pipeline with new mapped parameters. I feel like, now, the better solution for me would be to define a base pipeline and extend it with two different pipelines. Or may I still use namespace just to define one pipeline and execute it differently using different sets of parameters? Trying to figure this out. Rearranging my code. Will write if I have any specific error.
Just to clarify my objective(s). 1. Define two different data_science pipelines / or pipeline execution modes: demo_data_science with standardized inputs, training and evaluation options; extended_data_science with extra/different inputs, extra/different training algorithms, and extra/different evaluation metrics. 2. These pipelines could/should be executed separately (in which case, I guess, just extending two versions from the same base should be enough), or together (in which case, I think, the namespaces must be defined). Please correct me if I am wrong.
r
Hi Shah, we’ll need the traceback to understand why it didn’t run. Please share the error so we can assist. thanks
s
Hi Rashida, It was the error saying it didn't recognise the "pipe" parameter. Now, I have changed it as per the documentation shared by Ravi. Also, I realised I didn't need namespaces for now. Because, in order to use namespace, I had to create two separate input items, each with the
namespace_name.
prefix, pointing to the same file. This broke the visualization flow, because the corresponding output item (the same file) was without that prefix. So, three items referring to the same file, but all detached.
Hence, for now, I have removed the namespaces, and just extending base pipeline with a different set of parameters (using mapping). The original flow is back.
👍 1