Hi ! I want to modify a small thing of the load me...
# questionsa
Ana Paula Rojas
12/10/2023, 2:56 AMHi ! I want to modify a small thing of the load method of a specific Kedro Dataset. Is there a way to achieve this without needing to create a new Custom Data Set?
d
Deepyaman Datta
12/10/2023, 3:03 PMThe easiest thing I would recommend is creating a custom dataset that subclasses the dataset you want to modify. You can just override load.
If you really don't want to create a custom dataset, you can monkeypatch the dataset (i.e. using
unittest.mock.patch👍 1
n
Nok Lam Chan
12/10/2023, 5:05 PMOr you can just monkeypatch it, simple as dataset._load = your__load_func
d
Deepyaman Datta
12/11/2023, 1:40 AMOr you can just monkeypatch it, simple as dataset._load = your__load_func
(If you're using the dataset instance programmatically; won't work e.g. if using the YAML configuration) I guess maybe you could do the above in a before load hook...? But that's hacky AF. I'd do proper monkeypatching if really don't want to subclass. 🙈
a
Ana Paula Rojas
12/11/2023, 2:09 AMYes I want to use the YAML configuration, so I agree the best way is to subclass it.
Ana Paula Rojas
12/11/2023, 2:24 AMI also have a few questions if maybe you can guide me 😄
1. When I subclass it, I only need to define load again right? The rest of the methods will stil work the same?
2. Is this code supposed to go in src/extras/datasets ? (As in the docs of a new dataset?)
3. If I want the new load to use a new input that will come from the catalog.yml file, how can I achieve this?
m
marrrcin
12/11/2023, 8:04 AM1. Yes
2. It doesn’t matter, you just have to use full reference to the dataset class in the catalog
3. Just specify your dataset in the
type:👍 2
a
Ana Paula Rojas
12/11/2023, 4:32 PMHi super clear, thanks!
For point N3, I was actually referring to something different. In the calog.yml file:
my_catalog:
type: my_new_dataset
... (other inputs that already existed in the original dataset)
some_new_input: "blabla"
My question is how can I use some new input that didnt exist in the Dataset that I was subclassing it.
d
Deepyaman Datta
12/11/2023, 5:22 PMMy question is how can I use some new input that didnt exist in the Dataset that I was subclassing it.You need to overwrite the
__init__filepath👍 1
a
Ana Paula Rojas
12/11/2023, 5:50 PMYes it is a new dataset argument like filepath.
Thanks for the answers, will try it 🙂