https://kedro.org/ logo
Join Slack
Powered by
I am trying to create a "catch all" patterns using...
# questions
n
I am trying to create a "catch all" patterns using the Dataset Factory in DataCatalog. I experienced something unexpected. Steps to reproduce: 
1. kedro new
(Select no tools and default pipeline) 2. Add a pattern, depending on how I name my pattern, the pattern is being used sometimes. Details in 🧵
Copy code
"{catch_all}":
  type: pickle.PickleDataset
  filepath: data/06_models/{catch_all}.pickle
This would works fine and generate warning that I am overwriting the default dataset
Copy code
"{nok}":
  type: pickle.PickleDataset
  filepath: data/06_models/{nok}.pickle
If I use this instead, there is no warning generated and it ignores my pattern completely. Cc @Ankita Katiyar
👀 1
a
That’s strange, let me try it out 👀
Can reproduce, in some cases it’s using the 
"{default}"
pattern which comes from the runner instead
From the docs -
Copy code
Under the hood Kedro uses the pattern name "{default}" to generate the default datasets set in the runners. If you want to overwrite this pattern you should make sure you choose a name that comes before "default" in the alphabet for it to
https://docs.kedro.org/en/stable/data/kedro_dataset_factories.html#how-to-override-the-default-dataset-creation-with-dataset-factories
I believe this is from the changes in https://github.com/kedro-org/kedro/pull/3332 ^ only patterns that start with a/b/c will then be used as catch-all patterns which might be strange behaviour, I think we should fix this
n
@Ankita Katiyar Can you help to open this issue? The case here should be included as a regression test case. I tested it works fine in 0.18.14, it was introduced in 0.19.0
a
Yeah will do!
thankyou 1
i
only patterns that start with a/b/c
that seems quite arcane to me as a user. is there a motivation for the third ranking step being alphabetical instead of just raising an error saying that the resolution is impossible?
👍🏼 1
3 Views
Open in Slack
PreviousNext
Powered by