Define column names when reading a spark dataset in kedro
With kedro, how can I define the column names when reading a spark.SparkDataSet? below my catalog.yaml.
user-playlists:
type: spark.SparkDataSet
file_format: csv
filepath: data/01_raw/lastfm-dataset-1K/userid-timestamp-artid-artname-traid-traname.tsv
load_args:
sep: "\t"
header: False
# schema:
# filepath: conf/base/playlists-schema.json
save_args:
index: False
I have been trying to use the following schema, but it doesn't seem to be accepted (schema...