Hi Team! :party-parrot: Quick question hopefully: ...
# questions
b
Hi Team! 🦜 Quick question hopefully: How can I specify
schema
for a
SparkDataSet
in the catalog entry itself? What’s the best practice to represent the
StructType()
object in yaml? EDIT: or is the best practice to always save the schema to a separate params file and add just the
file_path
to the catalog entry?
d
Use the string representation
For example:
Copy code
driverId INT,
            driverRef STRING,
            number STRING,
            code STRING,
            name STRUCT<forename:STRING,surname:STRING>,
            dob DATE,
            nationality STRING,
            url STRING