https://kedro.org/ logo
#plugins-integrations
Title
# plugins-integrations
h

Hugo Evers

01/19/2024, 1:36 PM
I’ve encountered an error when trying to load a custom Transformers module that I’ve saved to MLflow using Kedro-MLflow. It failed while loading data from dataset
MlflowModelLoggerDataSet
. The error indicates that the module
transformers
does not have the specified attribute
LSGDistilBertForSequenceClassification
. Below is the traceback. Any insights on how to resolve this issue would be greatly appreciated.
Copy code
/home/ec2-user/PROJECT/.venv/lib/python3.9/site-packages/kedro/io/core.py:193 in load
return self._load()
...
/home/ec2-user/PROJECT/.venv/lib/python3.9/site-packages/kedro_mlflow/io/models/mlflow_model_logger_dataset.py:91 in _load
return self._mlflow_model_module.load_model(model_uri=self.model_uri, **self._load_args)
...
/home/ec2-user/PROJECT/.venv/lib/python3.9/site-packages/mlflow/transformers/init.py:875 in load_model
return _load_model(local_model_path, flavor_config, return_type, device, **kwargs)
...
/home/ec2-user/PROJECT/.venv/lib/python3.9/site-packages/mlflow/transformers/init.py:935 in _load_model
model_instance = getattr(transformers, flavor_config[_PIPELINE_MODEL_TYPE_KEY])
...
/home/ec2-user/PROJECT/.venv/lib/python3.9/site-packages/transformers/utils/import_utils.py:1375 in getattr
raise AttributeError(f"module {self.name} has no attribute {name}")

**Error**: `AttributeError: module transformers has no attribute LSGDistilBertForSequenceClassification`
This error occurred while executing a Kedro pipeline that loads a model from MLflow. Any guidance on troubleshooting or fixing this issue would be greatly appreciated. Thank you!
y

Yolan Honoré-Rougé

01/19/2024, 7:02 PM
Just to clarify, how was the model saved? Through
pipeline_ml_factory
or through
MlflowModelLoggerDataSet.save
method?
Is
LSGDistilBertForSequenceClassification
a custom class? Where is the code stored ?
h

Hugo Evers

01/19/2024, 8:26 PM
it was saved through MlflowModelLoggerDataSet.save, and it is a custom class that is constructed when you run AutoModelForSequenceClassification.from_pretrained(“lsg-distilbert-base-uncased-4096”)
y

Yolan Honoré-Rougé

01/20/2024, 2:01 PM
Ok so this is not a custom class you created? I guess
AutoModelForSequenceClassification.from_pretrained
do some dark magic with the PYTHONPATH to import the code. You need to import it manually in your project to make the class available.