#questions
Title
# questions
e
Elias
11/25/2022, 10:13 AM Copy code
catalog.yml:
inspection_output:
type: pandas.SQLQueryDataSet
credentials: postgresql_credentials
table_name: shuttles
layer: model_output
save_args:
index: trued
datajoely
11/25/2022, 10:16 AMYou’re using SQLQueryDataSet not SQLTableDataSet
e
Elias
11/25/2022, 10:17 AMah, thx!
d
datajoely
11/25/2022, 10:17 AMno worries that’s a subtle one!
e
Elias
11/25/2022, 10:20 AMcan you also version these datasets? or is it best practice to do so by a custom created column?
d
datajoely
11/25/2022, 10:24 AMWe don’t support this on the SQL side unfortunately. You could use append only mode and add a timestamp column although this may get very tall over time.
You may also consider a simple subclass that does this automatically
e
Elias
11/25/2022, 2:11 PMokay nice, is there any way to attain the timestamp kedro uses to version datasets during runtime? I want to give the same timestamp to multiple tables, which are written in different nodess.
d
datajoely
11/28/2022, 9:56 AMit’s created in the session object, if you really want to you could create a hook that sets this as an environment variable and then you can reuse that. In the future this will be easier to configure.
4 Views