https://kedro.org/ logo
#random
Title
# random
j

Juan Luis

12/19/2023, 4:51 PM
stumbled upon this blog post again https://incident.io/blog/data-stack (I remember reading it a couple of times last year) what does your (modern?) data stack look like? for local projects I usually go with: Meltano, Delta tables on MinIO, Kedro, Metabase
👀 1
i

Iñigo Hidalgo

01/15/2024, 11:34 AM
Took inspiration from you and have started to test delta tables on MinIO 🙂 atm just using my local dev machine, but it should transfer to smth like digitalocean quite seamlessly https://github.com/inigohidalgo/delta-python/blob/main/Delta-on-MinIO.ipynb
(dont worry the API key is fake 😅 )
j

Juan Luis

01/15/2024, 11:35 AM
nice!! how did you find the experience so far @Iñigo Hidalgo?? 🙂
i

Iñigo Hidalgo

01/15/2024, 11:38 AM
I wish it wasn't an experimental API as I wish I could start using it in prod already! I think last time I looked at the delta-rs implementation they didn't have support for loading as a certain version other than just passing the version number. Using the timestamp is much better UX. I still have to figure out all the table merging stuff, as the true power of delta tables is in there, but it seems very very promising.
🥳 1
2 Views