Richard Purvis
10/27/2023, 2:40 AMSQLTableDataset
while specifying data types? From what I’m seeing, the pandas API requires a dictionary of SQL alchemy type objects. Would this require a custom resolver to pass the types through the catalog?
FYI I’m attempting to save tables to a SQL Server/Azure DWH.Juan Luis
10/27/2023, 8:36 AMWould this require a custom resolver to pass the types through the catalog?yes! here's an example for Polars:
Richard Purvis
10/27/2023, 4:27 PMsqlalchemy.types.String(32)
Juan Luis
10/27/2023, 4:29 PM"${sqlalchemy:String:32}"
:
?Richard Purvis
10/27/2023, 4:30 PMCONFIG_LOADER_ARGS = {
"custom_resolvers": {
"sql_types": lambda x: getattr(sqlalchemy.types, x),
}
}
CONFIG_LOADER_ARGS = {
"custom_resolvers": {
"sql_string": lambda x: sqlalchemy.types.String(x),
}
}
Juan Luis
10/30/2023, 9:17 AMCONFIG_LOADER_ARGS = {
"custom_resolvers": {
"sql_types": lambda value: getattr(sqlalchemy.types, value.split(":")[0])(value.split(":")[1]),
}
}
or rather
def get_sql_type(value):
type_cls_name, arg = value.split(":")
type_cls = getattr(sqlalchemy.types, type_cls_name)
return type_cls_name(arg)
does it make sense @Richard Purvis?Richard Purvis
10/30/2023, 1:44 PM