hey guys,
I have a code which has kind of submode...
# questionsf
Fazil B. Topal
09/07/2023, 1:05 PMhey guys,
I have a code which has kind of submodels in it. (3 of them) They follow the same naming pattern for execution nodes. The problem is i can't put them all under
defaultd
datajoely
09/07/2023, 1:17 PMNamespaces are your friend here!
datajoely
09/07/2023, 1:17 PMfor this exact problem
f
Fazil B. Topal
09/07/2023, 3:07 PMCool, will try to use namespaces then 👍 thankyou
Fazil B. Topal
09/07/2023, 3:38 PMI have the following problem now. Without passing namespaces, i used the following convention in data catalog already:
Copy code
modelA.orders: ...
modelB.orders: ...
modelC.orders: ...
modelA.pipeline: ...
modelB.pipeline: ...
modelC.pipeline: ...pipelinemodelA.modelA.orders Copy code
node(
func: ...
inputs={
modelA=modelA.pipeline,
modelB=modelB.pipeline,
modelC=modelC.pipeline,
}
)d
datajoely
09/07/2023, 3:39 PMare you using a dynamic pipeline here?
f
Fazil B. Topal
09/07/2023, 3:41 PMNope, only 3 models and so far i didn't pass
namespaced
datajoely
09/07/2023, 3:42 PMokay gotcha
f
Fazil B. Topal
09/07/2023, 3:42 PMI so far invoked individual runs on my machine, for instance just running -p modelA or something
Fazil B. Topal
09/07/2023, 3:42 PMI wonder if tags would just solve it 😄
d
datajoely
09/07/2023, 3:43 PMwe are talking about explicitly adding the namespace to the pipeline piece right?
Copy code
ds_pipeline_1 = pipeline(
pipe=pipeline_instance,
inputs="model_input_table",
namespace="active_modelling_pipeline",
)f
Fazil B. Topal
09/07/2023, 3:44 PMyes right now I have 3 submodels(a,b,c) and final model takes those submodels so I was referring them explicitly in the node. All catalog/params looks namespaced but python code doesn't refer to namespace param as i did everything manually (wiring)
Fazil B. Topal
09/07/2023, 3:45 PMAdding code piece here for clear reference. That's it is defined now (namespace is not passed anywhere in the code
d
datajoely
09/07/2023, 3:46 PMokay namespace is applied at a pipeline level
datajoely
09/07/2023, 3:47 PMf
Fazil B. Topal
09/07/2023, 3:49 PMIn the pipeline function, im also not passing namespace 😄 My catalog I have defs like
drivingTime.modelPipeline#s3Fazil B. Topal
09/07/2023, 3:49 PMI use kedro.pipeline, don't import it from modular pipeline as well
d
datajoely
09/07/2023, 3:58 PMso that method should still have a
namespace=f
Fazil B. Topal
09/07/2023, 4:00 PMpipeline has param called namespace yes
Fazil B. Topal
09/07/2023, 4:00 PMI first passed but then kedro tries to load
modelA.modelA.ordersd
datajoely
09/07/2023, 4:01 PMyou may have to change your
.datajoely
09/07/2023, 4:02 PMnamespaces are a little tricky to get the hang of, maybe worth running through the tutorial before migrating your existing stuff?
f
Fazil B. Topal
09/07/2023, 4:02 PMI see but you're saying using tags wouldn't solve the issue, it has to be namespaced?
3 Views