anyone with issue with

databricks.ManagedTableDataSet

I have a pipeline with 2 outputs, since I am using ThreadRunner, both dataset need to upsert data at the same time. Since the implementation of

databricks.ManagedTableDataSet

use a fixed “`update`” view to do the upsert there is collision, the last one whichs is set survives.

This is interesting- are they both pointing to the same table?

Noo, different ones. but since this line runs twice: https://github.com/kedro-org/kedro-plugins/blob/1831245dad0879b4513925440c58311b97[…]dro-datasets/kedro_datasets/databricks/managed_table_dataset.py the view is overwrite. my really quick solution was to change from

update_data.createOrReplaceTempView("update")

to this: using the view name as a parameters

update_table = f"update_{self._table.table}"

in this way I can ensure that the same view is not created twice

Okay - nice solution, glad you’re unblocked