# questions

Felipe Monroy

02/02/2024, 1:31 PM
Hi, I know is still not possible to use something similar to the dataset factory patterns with the parameters. Do you know if there's any reason for this? Maybe I am missing something (like is already possible with OmegaConf) but I believe it would be a nice feature. At the moment I am using a custom DataCatalog to achieve it.


02/02/2024, 1:35 PM
ooh I’d love to learn more about how this works? Is it a bit like Hydra to generate different permutation?

Felipe Monroy

02/02/2024, 1:40 PM
Not really, it is a bit limited. I am only reusing the catalog resolver with the parameters.
Copy code
class DataCatalogParamsFactory(DataCatalog):
    """DataCatalog that also resolves factories in the parameters."""

    def _get_dataset(
        dataset_name: str,
        version: Version | None = None,
        suggest: bool = True,
    ) -> AbstractDataset:
        params_pattern = {
            k: v
            for k, v in self._datasets.items()
            if k.startswith("params:") and self._is_pattern(k)

        matched_pattern = self._match_pattern(params_pattern, dataset_name)
        if dataset_name not in self._datasets and matched_pattern:
            # If the dataset is a patterned dataset, materialise it and add it to
            # the catalog
            config_copy = copy.deepcopy(params_pattern[matched_pattern])
            dataset_config = self._resolve_config(
                dataset_name, matched_pattern, config_copy

            # This only works with MemoryDataset, all the params corresponds to this.
            if isinstance(dataset_config, MemoryDataset):
                self.add(dataset_name, dataset_config)
                return dataset_config

        return super()._get_dataset(dataset_name, version, suggest)

Ankita Katiyar

02/02/2024, 1:40 PM
There’s an issue open for something like this - would you mind adding your use case as a comment?
👍🏼 1
ty 1

Julian Nowak

03/06/2024, 9:17 AM
I added mine as well
❤️ 2


03/06/2024, 9:18 AM
Thanks for contributing!
😎 1