Adding to hugging face recently?
I have a PoC here...
# questionsp
Pascal Brokmeier
10/20/2025, 2:08 PMAdding to hugging face recently?
I have a PoC here
https://github.com/everycure-org/matrix/tree/feat/kedro-hf-dataset/pipelines/huggingface-dataset-demo
Curious to get feedback on that and happy to bring to the kedro-plugins repo ofc. @Merel just fyi (and sorry for sleeping on our google sheets dataset, I nudged laurens to pick that back up)
m
Merel
10/20/2025, 2:12 PMHello @Pascal Brokmeier, good to see you here again 👋 We've got some hugging face datasets in
kedro-datasetsp
Pascal Brokmeier
10/20/2025, 2:12 PMyeah that one reads only and it doesn't use the new file format
Pascal Brokmeier
10/20/2025, 2:12 PMthe above is meant to read/write + support spark/polars/pandas
👍 1
Pascal Brokmeier
10/20/2025, 2:13 PM(we're building this because we're getting read to release our knowledge graph to the public and want to have this well built in our release pipeline)
m
Merel
10/20/2025, 2:18 PMOkay cool in that case it can maybe replace the older ones
p
Pascal Brokmeier
10/20/2025, 3:25 PMOK I got it a bit more cleaned up and since we need it for now we're storing it badly named under our GCP datasets
https://github.com/everycure-org/matrix/blob/feat/kedro-hf-dataset/libs/matrix-gcp-datasets/src/matrix_gcp_datasets/huggingface.py
Pascal Brokmeier
10/20/2025, 3:25 PMI'll see if our academic devs can take this up and contribute it to the general datasets plugins
💛 1
m
Merel
10/20/2025, 3:28 PMThat would be great 😄