Hey Team, I have a kedro pipeline which saves its last 5 nodes data directly to database using kedroSQLTableDataSet .
Is there a way to implement rollback of all data that was saved in case any one of the data fails during insertion/appending to the database .
I saw that we can create a custom dataset to do a bulk insertion operation within a transaction. Please let me know what would be the best approach to do this. Thanks
@Priyanka Patil fyi
Nok Lam Chan
11/03/2023, 3:10 AM
11/03/2023, 9:19 AM
if you are using the default pandas dataset available in Kedro remember the following
are available to help manage this sort of things
If you really need to create your own dataset, you can copy our implementation and add this sort of pattern to the