Its possible to create custom delta table dataset ...
# questions
j
Its possible to create custom delta table dataset with change data capture option? and how i can create a table form my custom schame before writstreaming?
m
Yes it's possible to create a custom dataset. Instructions for that are here: https://docs.kedro.org/en/stable/data/how_to_create_a_custom_dataset.html
On your second question on how to create a table from a schema before streaming, I'm not really sure. Can you explain a bit more what your use case is?
j
creating a Delta table if the table doesn't already exist, its nessary for change data capture and for keepinghistorical data. my case i read form kafka and i wont to wirte to a delta table. But befor wirint i need to create a table with schma defintion and enable change data capture (i get extra columns). I want a kind of schema enforcement befor wiriting data from kefka
m
Okay, I don't have a lot of experience with this myself. I'm also not sure how much of this is a Kedro thing vs. just how you'd do it normally. I'll ask the team if anyone has some suggestions.
👍 1