https://kedro.org/ logo
#plugins-integrations
Title
# plugins-integrations
g

George Drakoulas

01/26/2024, 10:47 AM
Hey team, after multiple trials, thanks to your support, I managed to overcome my previous issues and deploy both the data processing and data science pipelines in Azure ML 😀. Now, you can see that the overall Kedro procedure has been executed successfully in Azure ML. The only paradox is that by checking the std_log.txt file, there are a few permission error messages (see the attached). Should I consider those or ignore them since they might be related to the previous errors I was facing and maintained in the memory?
❤️ 2
K 1
azure 1
m

marrrcin

01/26/2024, 11:28 AM
Are you trying to write to “local” paths in the container during pipeline execution?
Seems like this error is caught somewhere and ignored by your code / some library’s code that you’re using
g

George Drakoulas

01/26/2024, 12:30 PM
the fact is that when I don't include the code directory in the pipeline (~), the model is not able to be deployed, generating the following error
However, when including the code directory (.), the error is overcomed, the model is trained and the obtained "skipped" errors are generated in the log file
However, I'm not completely sure if the log error is due to this fact
m

marrrcin

01/26/2024, 1:43 PM
So there are two options in the plugin: a) with code upload b) with docker For a) you have to have pre-build docker image (registered as environment). In this option, every time you do
kedro azureml run
from your local machine, the code will be uploaded to Azure and executed there. For b) every time you want to run the pipeline in Azure ML you have to re-build the docker image and push it to ACR. As you can clearly see, the error is thrown from
kedro-viz
package - it seems Kedro Viz that tries to save data into local paths (local paths in Azure ML machine that runs your pipeline), which is not supported. Maybe @Nero Okwa can chip in - do you know what’s happening there? Maybe it’s some experiment tracking thing 🤔
g

George Drakoulas

01/29/2024, 10:24 AM
Thanks a lot!
Do you have any updates on the messages/warning/errors I received? Should I ignore them since the .pck file is extracted? Is there any way to tackle those? Please let me know if you need any additional information. thanks a lot in advance!