Balazs Konig
12/09/2022, 12:19 PMschema
for a SparkDataSet
in the catalog entry itself? What’s the best practice to represent the StructType()
object in yaml?
EDIT: or is the best practice to always save the schema to a separate params file and add just the file_path
to the catalog entry?Deepyaman Datta
12/09/2022, 6:18 PMdriverId INT,
driverRef STRING,
number STRING,
code STRING,
name STRUCT<forename:STRING,surname:STRING>,
dob DATE,
nationality STRING,
url STRING