Hi team, beginner question, is it possible to crea...
# questions
Hi team, beginner question, is it possible to create a partitioned dataset where you store many pickles with models, i.e:
Copy code
                 ├── 202201.pkl
                 ├── 202202.pkl
                 ├── 202203.pkl
                 ├── 202204.pkl


so on.
Is there a way of saving pickles this way? or maybe what would be a better way of doing this? any thoughts?
So enabling
versioning: True
will do this on a normal dataset, but we don’t currently give you control over the timestamp format so you’ll get one that it’s a lot more precise rather than month level
The only way I found similar of how it was being saved in a looped function for each period with pickle:
Copy code
   type: PartitionedDataSet
   path: "s3_path/folder_name/"
      type: pickle.PickleDataSet
   filename_suffix: ".pkl"
Since it was already looping for each period, every model was being saved in a dictionary and then with this way of defining the catalog it allowed me to save it the way it was before using the datacatalog. It might not be the most optimal thing though !