https://kedro.org/ logo
#questions
Title
# questions
j

Julien Witty

03/24/2023, 4:00 PM
Hey, I hope you are doing well. Our team is evaluating to use kedro as a framework. I was wondering if some of you have some experience with kedro integration with huggingface library. I was wondering how you are handleling models after training. My first reflex was to implement a custom dataset for this usecase not sure how much effort it is though Then maybe use a string as the model path (not very clean) for next node.
d

datajoely

03/24/2023, 4:10 PM
We’re very much overdue a native HuggingFace dataset, people have built custom datasets like you suggest
👍🏼 1
Happy to coach you through building it as it should be relatively simple and we’d love a contribution 🙂
n

Nok Lam Chan

03/24/2023, 9:29 PM
CustomDataSet should work, shout if you have need more help for implementation
3 Views