Hello team! I am trying to work with a Kedro project (stored in an external repo) on a Databricks environment. I am having issue setting up the catalog, in particular, I get the following error when trying to load a dataset from the hive DB in the Catalog:

DatasetError: Failed while loading data from data set SparkHiveDataset(database=hive_metadata.<DB_NAME>, format=hive, table=<TABLE_NAME>, table_pk=[], write_mode=errorifexists).
[UC_NOT_ENABLED] Unity Catalog is not enabled on this cluster.

Is Unity Catalog enablement for the cluster strictly necessary? Or can this be circumvented somehow? Thank-you 🙂