Erwin
08/18/2023, 2:57 PMdatabricks.ManagedTableDataSet
I have a pipeline with 2 outputs, since I am using ThreadRunner, both dataset need to upsert data at the same time. Since the implementation of databricks.ManagedTableDataSet
use a fixed “`update`” view to do the upsert there is collision, the last one whichs is set survives.datajoely
08/18/2023, 3:44 PMErwin
08/18/2023, 3:49 PMupdate_data.createOrReplaceTempView("update")
to this: using the view name as a parameters
update_table = f"update_{self._table.table}"
in this way I can ensure that the same view is not created twicedatajoely
08/18/2023, 3:50 PM