[SOLVED] Hi Folks Any ideas how to log metrics during neural networks (NN) training following

kedro-mlflow

concept? In my case, I use

pytorch lightning

, and they provided automatic logging using

mlflow

. Using this logger, the metrics are automatically saved into a new folder. I'm wondering how to save this training metrics as artifact of experiment that can also be tracked in

Kedro

catalog. Please help :)

It's possible. In lightning you have

from lightning.pytorch import Trainer
from lightning.pytorch.loggers import MLFlowLogger

mlf_logger = MLFlowLogger(experiment_name="lightning_logs", tracking_uri="file:./ml-runs")
trainer = Trainer(logger=mlf_logger)

So you need to bind the lightning logger to the current mlflow configuration. It's just a matter of getting the

run_id

from the current kedro-mlflow run:

def get_mlflow_run_id():
    run_id = None
    if (ar := mlflow.active_run()) and ar.info.run_id:
        run_id = ar.info.run_id
    return run_id

And similarly for other parameters. You can also use Kedro hooks to extract

experiment_name

/

tracking_uri

from `kedro-mlflow`'s config, as it attaches itself to Kedro Context like

context.mlflow

Hi @marrrcin thanks for your help. I will try it :)

Honestly it is weird that it does not work out of the box. Kedro mlflow set most of the config globally (with

mlflow.set_tracking_uri

and

mlflow.set_experiment

so the

MlflowLogger

of pytorch lightning should default to this. If it doesn't, it means that there is some configuration override happening under the hood, so we should check the source code

The behaviour is simple there - if run_id is not set, it creates new run. https://lightning.ai/docs/pytorch/stable/_modules/lightning/pytorch/loggers/mlflow.html#MLFlowLogger

Hi @marrrcin need clarification from your suggestion. Extracting

experiment_name

and

tracking_uri

using Kedro hooks means should use I

after_context_created

? Then, how to bring those info into the training node? What I think is to add them into catalog, which I can access those two as inputs in a training node. Is it a good approach or there is a better way? thanks

I would rather go with a single hook class that implements both

after_context_created

and

before_node_run

- first one captures reference to the Kedro Context and the second one injects `experiment_name`/`tracking_uri` etc into the node. Example (of this idea - not exactly your case): https://linen-slack.kedro.org/t/10362890/hi-all-another-question-slightly-smiling-face-i-am-creating-#ce99c8ab-3307-4932-9ff8-1e416e7f48ac