tableA:
  type: pandas.SQLTableDataset
  table_name: tableA
  load_args:
    schema: my.dev
  save_args:
    schema: my.dev
    if_exists: "append"
    index: False

~I have a table, TableA, which is the final output of my pipeline. The table already contains primary keys. If the primary key values are not yet in the table, I want to append the new rows. However, for rows where the primary key values already exist, I want to update those rows with the new results. What would be the right save_args to use in this case? I tried 'replace' for if_exists, but this keeps deleting the whole table and only the current results are stored. If I use 'append', despite the primary keys, duplicated results will still be inserted into the table https://docs.kedro.org/en/0.18.14/kedro_datasets.pandas.SQLTableDataset.html#kedro_datasets.pandas.SQLTableDataset~

Someone will reply to you shortly. In the meantime, this might help:

Solved. Basically, I have to ensure primary keys are created after the first insertion to the table. When I experimented with replace, the whole table seems to have lost the primary key definition. And it's fixed when I redefine the primary keys