I’ve encountered an error when trying to load a custom Transformers module that I’ve saved to MLflow using Kedro-MLflow. It failed while loading data from dataset

MlflowModelLoggerDataSet

. The error indicates that the module

transformers

does not have the specified attribute

LSGDistilBertForSequenceClassification

. Below is the traceback. Any insights on how to resolve this issue would be greatly appreciated.

/home/ec2-user/PROJECT/.venv/lib/python3.9/site-packages/kedro/io/core.py:193 in load
return self._load()
...
/home/ec2-user/PROJECT/.venv/lib/python3.9/site-packages/kedro_mlflow/io/models/mlflow_model_logger_dataset.py:91 in _load
return self._mlflow_model_module.load_model(model_uri=self.model_uri, **self._load_args)
...
/home/ec2-user/PROJECT/.venv/lib/python3.9/site-packages/mlflow/transformers/init.py:875 in load_model
return _load_model(local_model_path, flavor_config, return_type, device, **kwargs)
...
/home/ec2-user/PROJECT/.venv/lib/python3.9/site-packages/mlflow/transformers/init.py:935 in _load_model
model_instance = getattr(transformers, flavor_config[_PIPELINE_MODEL_TYPE_KEY])
...
/home/ec2-user/PROJECT/.venv/lib/python3.9/site-packages/transformers/utils/import_utils.py:1375 in getattr
raise AttributeError(f"module {self.name} has no attribute {name}")

**Error**: `AttributeError: module transformers has no attribute LSGDistilBertForSequenceClassification`

This error occurred while executing a Kedro pipeline that loads a model from MLflow. Any guidance on troubleshooting or fixing this issue would be greatly appreciated. Thank you!

Just to clarify, how was the model saved? Through

pipeline_ml_factory

or through

MlflowModelLoggerDataSet.save

method?

it was saved through MlflowModelLoggerDataSet.save, and it is a custom class that is constructed when you run AutoModelForSequenceClassification.from_pretrained(“lsg-distilbert-base-uncased-4096”)

Ok so this is not a custom class you created? I guess

AutoModelForSequenceClassification.from_pretrained

do some dark magic with the PYTHONPATH to import the code. You need to import it manually in your project to make the class available.