#resources
- n
Nok Lam Chan
06/12/2023, 12:54 PMFind this blog post from #questions today, maybe we can add it to the
list? https://itstherealdyl.com/2022/03/29/deploying-a-recommendation-system-the-kedro-way/awesome-kedro✅ 1jdm- 4
- 8
- m
marrrcin
06/22/2023, 10:46 AMWe’re pushing MLOps platform on Snowflake with the help of K Kedro and MLflow https://www.linkedin.com/posts/marrrcin_from-0-to-mlops-with-part-2-architecting-acti[…]241131597824-nD7v?utm_source=share&utm_medium=member_desktop Likes on LinkedIn highly appreciated ☺️👍 2🥳 3❄️ 2 - m
marrrcin
06/29/2023, 11:27 AMOur real-world use case of using Kedro at a core of a project - delivered in 2 months from idea to prod https://getindata.com/blog/from-concept-production-2-months-sales-forecasting-machine-learning-model-dema-ai/🥳 6🚀 6😃 1yi- 3
- 3
- j
Juan Luis
07/05/2023, 8:40 AMnice and short writeup on Cloud development environments https://contributor-experience.org/docs/guide/tools/cdes.html I'm very fond of Gitpod for trainings and live demos but I still use my local development environment for almost anything else. what do folks use on their day to day?dd- 3
- 2
- j
Jo Stichbury
07/10/2023, 8:50 AMI spotted this article this morning, just in case anyone else finds it useful: https://www.prestonblackburn.com/projects/KedroSnowflake and https://github.com/PrestonBlackburn/kedro-snowflake-extensions - j
Juan Luis
07/11/2023, 12:53 PMsuper cool: query Snowflake directly from Polars using the ADBC driver https://pola-rs.github.io/polars/py-polars/html/reference/api/polars.read_database.html (more reading: https://arrow.apache.org/adbc/0.5.1/driver/snowflake.html) - j
Juan Luis
07/12/2023, 7:00 AMnice churn prediction Kedro project! K https://github.com/laizaparizotto/churn-prediction-kedro👍 1 - j
Juan Luis
08/14/2023, 9:13 AMthe new Prefect website looks rad! https://www.prefect.io/👍🏼 1dc- 3
- 4
- j
Juan Luis
08/23/2023, 5:49 PM"LLM Apps Are Mostly Data Pipelines" 👀 https://meltano.com/blog/llm-apps-are-mostly-data-pipelines/👍🏼 1cai- 4
- 7
- j
Juan Luis
08/30/2023, 1:58 PM
, DataFrame APIs for BigQuery 🔥 https://cloud.google.com/python/docs/reference/bigframes/latest, https://pypi.org/project/bigframes/bigframesibis 3🎉 3dcd- 4
- 17
- j
Juan Luis
08/30/2023, 5:38 PMKedro + Vineyard https://medium.com/cncf-vineyard/efficient-data-sharing-in-data-science-pipelines-on-kubernetes-bb42d36c739 via @datajoely and authored by @何涛🚀 3du- 3
- 4
- n
Nok Lam Chan
09/02/2023, 7:15 PMRandomly find this project today👀 https://pypi.org/project/kedro-graphql/#data👀 4 - j
Juan Luis
09/06/2023, 4:17 AMhttps://github.com/chdb-io/chdb a competitor to DuckDB powered by ClickHouse!👀 4dc- 3
- 3
- j
Juan Luis
09/06/2023, 10:12 AM - m
marrrcin
09/12/2023, 3:57 PM - j
Juan Luis
09/29/2023, 4:41 PMone of my favourite newsletters about data https://www.blef.fr/modern-data-stack-upgrade/ published a handful of very good links: • cost structures of different data stacks ("Modern", bundled, self-hosted) https://win.hyperquery.ai/p/does-the-modern-data-stack-work-at?ref=blef.fr • data modelling https://towardsdev.com/data-modeling-in-the-modern-data-stack-d29be964b3a7 • data layers https://medium.com/@sivailango.s/principles-of-data-layers-in-data-platform-a336a0ff9e1e an alternative to the Kedro convention cc @Andrew Stewart • an aplication of the Diátaxis documentation framework to data platforms https://deezer.io/rethinking-your-data-platform-documentation-so-that-people-actually-read-it-84baff70b9a4👍 3🔖 1 - d
datajoely
09/29/2023, 4:47 PMthe sub-layers map pretty identically to the Kedro convention - super interesting - m
marrrcin
10/12/2023, 2:32 PMOur approach to dynamic pipelines in Kedro K https://www.linkedin.com/posts/marrrcin_kedro-mlops-machinelarning-activity-7118241663164067840-ueuT?utm_source=share&utm_medium=member_desktop🚀 7K 4👍 3 - m
marrrcin
10/18/2023, 3:24 PMHot off the GetInData’s press: https://www.linkedin.com/posts/marrrcin_deploying-efficient-kedro-pipelines-on-gcp-ac[…]513563742208-9XzA?utm_source=share&utm_medium=member_desktop + KUDOS to @Artur Dobrogowski🚀 7getindata 2K 2 - n
Nok Lam Chan
10/24/2023, 3:32 PMFor those who read https://kedro.org/blog/build-a-custom-kedro-runner (Kedro SoftFailRunner) before, there is a <https://github.com/noklam/kedro-softfail-runner
|pip-installable package> now. (Yes, you can extend Kedro Runner too)
Copy codepip install kedro_softfail_runner kedro run --runner=kedro_softfail_runner.SoftFailRunnerK 4🥳 4❤️ 3d- 2
- 1
- a
Afiq Johari
10/26/2023, 12:14 PMBesides
andiris
, has anyone found other online examples that could be interesting to explore? It is helpful to learn from different designs and how people are implementing Kedro into their workflows. Or maybe there's already a list somewhere that I'm not aware of. Thanksspaceflightsdn- 3
- 5
- j
Juan Luis
11/08/2023, 10:02 AMthis one is new to me: polyglot project templates https://github.com/scientific-python/cookie works with copier, cookiecutter, and cruft 🤯- 1
- 1
- d
datajoely
11/09/2023, 5:54 PMNice post from Google Cloud 🚀 https://medium.com/google-cloud/deploying-kedro-pipelines-on-vertex-ai-the-mlops-journey-of-a-life-company-8bec98f26649K 4jm+2- 5
- 6
- j
Juan Luis
11/11/2023, 1:25 PMyesterday I discovered
, a tool that allows you to query PyPI download stats from the CLI:pypinfoCopy code
it can filter by date, Python version, package version... read on! https://github.com/ofek/pypinfo#usage ~/Projects/Personal/pypi-stats-playground ························· 14:23:29 ❯ echo $GOOGLE_APPLICATION_CREDENTIALS /home/juanlu/Projects/Personal/pypi-stats-playground/core-guard-404813-c03393ad4050.json ~/Projects/Personal/pypi-stats-playground ························· 14:23:40 ❯ pypinfo kedro Served from cache: False Data processed: 657.83 MiB Data billed: 658.00 MiB Estimated cost: $0.01 | download_count | | -------------- | | 349,627 |❤️ 2 - j
Juan Luis
11/22/2023, 4:22 AMthe schedule of PyData Global is up https://pydata.org/global2023/schedule/ I'll be giving a talk "Who needs ChatGPT? Rock solid AI pipelines with Hugging Face and Kedro" K there are dozens of super interesting talks, check it out!🎉 9👍🏼 1 - l
Laiza Milena Scheid Parizotto
11/24/2023, 3:01 PMHiii folks, Do you guys know any project using
andpillow.ImageDataSet
just for reference?PartitionedDatasetjm- 3
- 2
- j
Juan Luis
11/27/2023, 11:40 AMsome hints on how the data ecosystem is going to consolidate (because the question is not if, it's how) https://www.dataengineeringweekly.com/p/data-engineering-in-retrospect-key👀 1 - j
Juan Luis
11/29/2023, 11:21 AMSimon Brugman (contributor to kedro-airflow) and team have released PyCodeHash, a library to hash Python functions https://github.com/pycodehash/pycodehash🤔 2👀 3d- 2
- 3
- j
Juan Luis
12/01/2023, 8:00 AMif you follow Chad Sanderson on LinkedIn you might have heard of data contracts. if so, check this out https://github.com/bitol-io/open-data-contract-standard❤️ 1👀 1d- 2
- 2
- c
Chris Schopp
12/05/2023, 2:52 PMA tip I wanted to share: You can name a node using the dunder method
instead of typing out a__name__
.strCopy code
If you're setting the node name to be the function name, this makes it so renaming the function and the node is just one step.node( func=explode_missing_values, ... name=explode_missing_values.__name__, ) # is equivalent to node( func=explode_missing_values, ... name="explode_missing_values", )K 3🙏🏼 1🙏 1djm- 4
- 13