new blog post: Building scalable data pipelines with Kedro and Ibis Kibis https://kedro.org/blog/building-scalable-data-pipelines-with-kedro-and-ibis If you have ever • ...slurped up large amounts of data into memory, instead of pushing execution down to the source database/engine • ...prototyped a node in pandas, and then rewritten it in PySpark/Snowpark/some other native dataframe API • ...implemented a proof-of-concept solution in 3-4 months on data extracts, and then struggled massively when you needed to move to running against the production databases and scale out • ...insisted on using Kedro across the full data engineering/data science workflow for consistency (fair enough), although dbt would have been the much better fit for non-ML pipelines, because you essentially needed a SQL workflow then this blog post is for you! thanks @Deepyaman Datta for writing it, and to @Cody Peterson and the rest of the Voltron team for reviewing the draft! 👏🏼

is there a twitter post we could promote? else can just write one for @IbisData

we have a linkedin post but not a twitter/x one

what's this "twitter" thing you talk about? 🧌 as @datajoely says we're only on LinkedIn and Mastodon https://social.lfx.dev/@kedro

we don't have an Ibis linkedin (just use personal) but here's twitter! https://x.com/IbisData/status/1752681491341991939?s=20

this is real professional milestone for me 🤯

ibis 🎉 Nice one guys 🙂 This will be so useful to push for further Ibis adoption in my teams ❤️ I fully agree that the union of kedro and ibis seems like a match made in heaven

@Iñigo Hidalgo thanks for leading the way on integrating Kedro and Ibis! On a related note, somebody from the Ibis team actually brought up your name yesterday, when talking about people in the OSS community excited about Ibis. :)

They were certainly right 🙂 I wonder how they pronounced my name though 😅