Rob
01/23/2023, 3:43 PMDeepyaman Datta
01/23/2023, 3:45 PMitem_features
?Rob
01/23/2023, 3:47 PM_pandas_parquet: &_pandas_parquet
type: pandas.ParquetDataSet
_spark_parquet: &_spark_parquet
type: spark.SparkDataSet
file_format: parquet
load_args:
header: true
save_args:
header: true
mode: overwrite
delisting.basic_item_features:
<<: *_spark_parquet
filepath: ${base_path_spark}/feature_generation/basic_item_features.parquet
delisting.basket_level_item_features:
<<: *_spark_parquet
filepath: ${base_path_spark}/feature_generation/basket_level_item_features.parquet
delisting.customer_level_item_features:
<<: *_spark_parquet
filepath: ${base_path_spark}/feature_generation/customer_level_item_features.parquet
delisting.item_features@spark:
<<: *_spark_parquet
filepath: ${base_path_spark}/feature_generation/item_features.parquet
delisting.item_features@pandas:
<<: *_pandas_parquet
filepath: ${base_path}/feature_generation/item_features.parquet
This is my catalog, but IDK if the error is referring to the path or the name of the datasetDeepyaman Datta
01/23/2023, 4:06 PMnode(my_node_func, "delisting.item_features@pandas", "whatever")
instead of node(my_node_func, "delisting.item_features", "whatever")
Rob
01/23/2023, 4:44 PM