Juan Luis
04/24/2023, 11:01 AMNero Okwa
04/25/2023, 2:08 PMJo Stichbury
04/28/2023, 10:58 AMNero Okwa
04/28/2023, 4:41 PMpip install kedro-viz==6.1.0
• React: npm install @quantumblack/kedro-viz@latest
What's next?
• Collaboration features within Kedro-Viz.
• Create your own reports.
Follow our roadmap for more updates, and suggest future features to the community.
Until next time,
The Kedro Team 💜Merel
05/02/2023, 4:33 PM0.18.8
is out! K
TLDR: In this release, we added lots of documentation changes, introduced theKedro 0.18.8 is a non-breaking release in the 0.18.x series, which means you can upgrade with:environment variable to configure logging and added the fullKEDRO_LOGGING_CONFIG
CLI command to session store to improve run reproducibility using Kedro-Viz experiment tracking.kedro run
pip install kedro==0.18.8
or
conda install -c conda-forge kedro==0.18.8
🚀 Major Features/Changes
• Added KEDRO_LOGGING_CONFIG
environment variable, which can be used to configure logging from the beginning of the kedro process.
• Removed logs folder from the Kedro new project template. File-based logging will remain but just be level INFO
and above and go to project root instead.
🪲 Bug fixes and other changes
• Improvements to Jupyter E2E tests.
• Added full kedro run
CLI command to session store to improve run reproducibility using Kedro-Viz experiment tracking.
✍️ Documentation changes
• Improvements to Sphinx toolchain including incrementing to use a newer version.
• Improvements to documentation on visualising Kedro projects on Databricks, and additional documentation about the development workflow for Kedro projects on Databricks 🧱
• Improvements to documentation about configuration.
• Updated table of contents for documentation to reduce scrolling.
• Added deprecation warnings about the removal of kedro.extras.datasets
.
• And more!
🗞️ In other news: we’ve welcomed @marrrcin to the Kedro Technical Steering Committee! You can read more details about this fantastic news on our blog.
Have you tried Kedro-Viz?
Kedro-Viz is an interactive development tool for building and visualising data science pipelines with Kedro. It enables you to monitor the status of your ML project, present it to stakeholders, and smoothly bring new team members onboard. It also offers experiment tracking, and the ability to preview code and datasets.
How do I get Kedro-Viz?
• Python: pip install kedro-viz==6.0.0
• React: npm install @quantumblack/kedro-viz@latest
Visit demo.kedro.org to try out Kedro-Viz
We welcome every community contribution, large or small. See what we're working on now and report bugs or suggest future features.
Until next time,
The Kedro Team 💛Juan Luis
05/04/2023, 1:30 PMJuan Luis
05/05/2023, 7:00 AMJuan Luis
05/17/2023, 11:08 AMJuan Luis
05/19/2023, 9:22 AMJuan Luis
05/23/2023, 7:10 AMkedro.extras.datasets
is deprecated and will be removed in Kedro 0.19, so these improvements won't be backported. to benefit from them, run
pip install "kedro-datasets==1.3.0"
and check out the complete list of datasets in our documentation https://docs.kedro.org/en/stable/kedro_datasets.html
🚀 Major Features/Changes
• Added pandas 2.0 support.
• Added SQLAlchemy 2.0 support (and dropped support for versions below 1.4).
• Added a save method to the APIDataSet
• Reduced constructor arguments for APIDataSet
by replacing most arguments with a single constructor argument load_args
. This makes it more consistent with other Kedro DataSets and the underlying requests
API, and automatically enables the full configuration domain: stream, certificates, proxies, and more.
• Relaxed Kedro version pin to >=0.16
• Added metadata
attribute to all existing datasets. This is ignored by Kedro, but may be consumed by users or external plugins.
• Added ManagedTableDataSet
for managed delta tables on Databricks.
🐛 Bug fixes and other changes
• Relaxed delta-spark
upper bound to allow compatibility with Spark 3.1.x and 3.2.x.
• Upgraded required polars
version to 0.17.
• Renamed TensorFlowModelDataset
to TensorFlowModelDataSet
to be consistent with all other plugins in kedro-datasets.
kedroid Community contributions
Thank you to Brian Cechmanek, Joseph McDonnell, and Danny Farah!
Until next time,
The Kedro Team 💛Nero Okwa
05/26/2023, 4:22 PMpip install kedro-viz==6.2.0
• React: npm install @quantumblack/kedro-viz@latest
What's next?
• Collaborative experiment tracking version 2.
• Rebrand of Kedro.
• Shareable URLs for Kedro-Viz pipeline visualisation.
Follow our roadmap for more updates, and suggest future features to the community.
Until next time,
The Kedro Team 💜Nok Lam Chan
05/31/2023, 12:36 PMJuan Luis
06/01/2023, 8:39 AM0.18.9
is out! K
TLDR: In this release, we added support for aKedro 0.18.9 is a non-breaking release in the 0.18.x series, which means you can upgrade with:attribute in datasets, introduced a newmetadata
that is more flexible and configurable, and fixed some bugs withkedro.logging.RichHandler
. We also added substantial improvements to our deployment docs, and we keep making progress towards following modern Python packaging standards.OmegaConfigLoader
pip install kedro==0.18.9
or
conda/mamba/micromamba install -c conda-forge kedro=0.18.9
🚀 Major Features and improvements
• kedro run --params
now updates interpolated parameters correctly when using OmegaConfigLoader
.
• Added metadata
attribute to <http://kedro.io|kedro.io>
datasets. This is ignored by Kedro, but may be consumed by users or external plugins.
• Added kedro.logging.RichHandler
. This replaces the default rich.logging.RichHandler
and is more flexible, user can turn off the rich
traceback if needed.
🪲 Bug fixes and other changes
• OmegaConfigLoader
will return a dict
instead of DictConfig
.
• OmegaConfigLoader
does not show a MissingConfigError
when the config files exist but are empty.
✍️ Documentation changes
• Added documentation for collaborative experiment tracking within Kedro-Viz.
• Revised section on deployment to better organise content and reflect how recently docs have been updated.
• Minor improvements to fix typos and revise docs to align with engineering changes.
⚠️ Breaking changes to the API
• kedro package
does not produce .egg
files anymore, and now relies exclusively on .whl
files. While this is technically a breaking change, we included it anyway because .egg
files have long been deprecated and we hope not many users are relying on them.
💌 Community contributions
• Many thanks to tomasvanpottelbergh and debugger24 for contributing PRs to this release!
We welcome every community contribution, large or small. See what we're working on now and report bugs or suggest future features.
Until next time,
The Kedro Team 💛Nero Okwa
06/07/2023, 1:20 PMYetunde
06/09/2023, 4:24 PMkedro
, kedro-viz
, awesome-kedro
, our docs, and our website.
In our releases, we also urge you to look for new features like variable interpolation with our OmegaConfigLoader
in Kedro 0.18.10 and multi-user experiment tracking from Kedro-Viz 6.2.0.
How do you learn more?
• Have a look at our rebranding blogpost
• Check our branding guidelinesJuan Luis
06/22/2023, 9:38 AMJuan Luis
06/26/2023, 8:57 AMJuan Luis
07/03/2023, 2:54 PM0.18.11
is out! K
TLDR: In this release, we added added a newKedro 0.18.11 is a non-breaking release in the 0.18.x series, which means you can upgrade with:official starter and significantly improved the documentation around Databricks deployments. We also fixed some bugs around micropackaging and remote datasets, updated the documentation for Prefect 2.0, and deprecated some class names.databricks-iris
pip install kedro==0.18.11
or
conda/mamba/micromamba install -c conda-forge kedro=0.18.11
🚀 Major Features and improvements
• Added databricks-iris
as an official starter.
🪲 Bug fixes and other changes
• Reworked micropackaging workflow to use standard Python packaging practices.
• Make kedro micropkg package
accept --verbose
.
• Compare for protocol and delimiter in PartitionedDataSet
to be able to pass the protocol to partitions which paths starts with the same characters as the protocol (e.g. <s3://s3-my-bucket>
).
✍️ Documentation changes
• Significant improvements to the documentation that covers working with Databricks and Kedro, including a new page for workspace-only development, and a guide to choosing the best workflow for your use case.
• Updated documentation for deploying with Prefect for version 2.0.
⚠️ Upcoming deprecations
• Renamed dataset and error classes, in accordance with the Kedro lexicon. Dataset classes ending with "DataSet" and error classes starting with "DataSet" are deprecated and will be removed in 0.19.0.
💌 Community contributions
• Many thanks to jmalovera10, @Rahul Kumar, @Julius Hetzel, jacobweiss2305, eduardoconto and tomasvanpottelbergh for contributing PRs to this release!
We welcome every community contribution, large or small. See what we're working on now and report bugs or suggest future features.
Until next time,
The Kedro Team 💛Nero Okwa
07/07/2023, 4:43 PM6.3.2
is out! KKKKK
@here we're excited to announce that Kedro-Viz 6.3.2 is out! 🥳
Kedro-Viz is the interactive development tool for building and visualising data science pipelines with Kedro. It enables you to monitor the status of your ML project, present it to stakeholders, and smoothly bring new team members onboard.
Kedro-Viz also comes with an experiment tracking feature enabling you to compare different runs of a Kedro project, and you can also preview code and datasets.
We've added a new feature and several bug fixes in the 6.3 versions. Check out the demo and get full details in the release notes.
🚀 What can you expect in this release?
• Added validation for layers in transcoding datasets.
• Added support for dataset preview as specified in the Kedro catalog under metadata.
• Updated the UI with the new Kedro rebrand look and feel.
• Enabled original node input and output names in metadata panel.
🪲 Bug fixes and other changes
• Fixed large pipeline layout warning message.
• Fixed flowchart not updating on back/forward clicks after selecting a registered pipeline.
• Fixed broken URL link between experiment tracking and flowchart in a non-default pipeline.
How do I get the latest release?
• Python: pip install kedro-viz==6.3.2
• React: npm install @quantumblack/kedro-viz@latest
What's next?
• Shareable URLs for Kedro-Viz pipeline visualisation.
Follow our roadmap for more updates, and suggest future features to the community.
Until next time,
The Kedro TeamLinenBot
07/31/2023, 9:04 AMtynan.debold
joined #announcements.Merel
08/01/2023, 4:13 PM0.18.12
is now available! K and so are kedro-datasets 1.5.1
& kedro-airflow 0.6.0
📣
TLDR: In the new release of Kedro, we introduce the dataset factories feature, activated all built-in resolvers by default for OmegaConfigLoader (except for oc.env), and added a new CLI command to rank dataset factories. We made some important bug fixes and updated the documentation, notably recommending ruff as the linter.See the 🧵 for details of each release. We welcome every community contribution, large or small. See what we're working on now and report bugs or suggest future features. Until next time, The Kedro Team 💛
Nero Okwa
08/18/2023, 8:34 AMImportError
as kedro-datasets is now lazily loaded.
• Fixed Sidebar search result based on Pretty name setting.
• Fixed issue of encountering a black page in Safari when interacting with modular pipelines.
How do I get the latest release?
• Python: pip install kedro-viz==6.4.0
• React: npm install @quantumblack/kedro-viz@latest
What's next?
• Shareable URLs for Kedro-Viz pipeline visualisation.
• Investigating the debugging use case of Kedro-Viz.
Try these features and provide feedback.
Follow our roadmap for more updates, comment on the tickets, and suggest future features to the community.
Until next time,
The Kedro Team
KKJuan Luis
08/18/2023, 2:40 PMkedro-datasets
1.5.3 is out!
if you've been having problems installing the optional dependencies ("extras") lately, hopefully this release addresses them all. thank you for your patience!
on the other hand, @FlorianGD contributed a fix to api.APIDataSet
so that now it doesn't perform an extra encoding of the responses, and @Flavien contributed an improvement to databricks.ManagedTableDataSet
so that now it's read-only by default. thanks a lot! kedroid
read the complete release notes online: https://github.com/kedro-org/kedro-plugins/releases/tag/kedro-datasets-1.5.3
until next time,
The Kedro Team 💛LinenBot
08/24/2023, 12:08 PMemil
joined #announcements.Juan Luis
08/31/2023, 2:34 PM0.18.13
and kedro-datasets 1.6.0
are available! K
TLDR: In this new release of Kedro we finally introduce official support for Python 3.11, as well as some newKedro 0.18.13 is a non-breaking release in the 0.18.x series, which means you can upgrade with:features, a newOmegaConfigLoader
command, and a leaner project structure. We also made significant documentation improvements around the Data Catalog and the datasets. Preparations for 0.19.0 are underway!kedro catalog resolve
pip install kedro==0.18.13
or
conda/mamba/micromamba install -c conda-forge kedro=0.18.13
🚀 Major Features and improvements
• Added support for Python 3.11.
• Added new OmegaConfigLoader
features: registering of custom resolvers through CONFIG_LOADER_ARGS
and support for global variables.
• Added kedro catalog resolve
CLI command that resolves dataset factories in the catalog with any explicit entries in the project pipeline.
• Implemented a flat conf/
structure for modular pipelines and updated kedro pipeline create
and kedro catalog create
accordingly.
• Made various updates to the Kedro project template and Kedro starters: use of OmegaConfigLoader
, transition from setup.py
to pyproject.toml
, and flat conf/
structure.
🪲 Bug fixes and other changes
• Updated OmegaConfigLoader
to ignore config from hidden directories like .ipynb_checkpoints
.
✍️ Documentation changes
• Restructure beginner and advanced pages about the Data Catalog and datasets.
• Moved contributor documentation to the GitHub wiki.
• Updated example of using generator functions in nodes.
• Added migration guide from the ConfigLoader
and the TemplatedConfigLoader
to the OmegaConfigLoader
.
⚠️ Upcoming deprecations
• The ConfigLoader
and the TemplatedConfigLoader
are deprecated and will be removed in the 0.19.0
release.
• AbstractDataSet
becomes AbstractDataset
and AbstractVersionedDataSet
becomes AbstractVersionedDataset
, old names are deprecated and will be removed in 0.19.0.
• Using the layer
attribute at the top level is deprecated and it will be removed in Kedro 0.19.0
. Please move layer
inside the metadata -> kedro-viz
attributes.
🐍 Migration Tips for Python 3.11
• Users on Windows with Python >=3.8 should note we've pinned pytables
to 3.8.0
due to compatibility issues.
💌 Community contributions
Many thanks to @Laiza Milena Scheid Parizotto and @Jon Cohen for contributing PRs to this release!
🚨 This will be among the last releases in the 0.18.x line, very soon we will start working on 0.19.0. Stay tuned!
We welcome every community contribution, large or small. See what we're working on now and report bugs or suggest future features.
Until next time,
The Kedro Team 💛Nero Okwa
08/31/2023, 5:23 PMpip install kedro-viz==6.5.0
• React: npm install @quantumblack/kedro-viz@latest
What's next?
• Shareable URLs for Kedro-Viz pipeline visualisation.
• Investigating the debugging use case of Kedro-Viz.
Try these features and provide feedback.
Follow our roadmap for more updates, comment on the tickets, and suggest future features to the community.
Until next time,
The Kedro Team
KKLinenBot
09/11/2023, 1:19 PMjonathan_daniel
joined #announcements. Also, nok.lam.chan
joined.Yetunde
09/26/2023, 4:48 PMkedro-mlflow
, kedro-pandera
, and he is thoughtful about progressing Kedro's roadmap and development. You'll experience him as a helpful community member in channels like #plugins-integrations. You can read more about our TSC.
We're also celebrating the open sourcing of a new project that already integrates with Kedro. This is a QuantumBlack project that we've been excited about for a while, and it is called Vizro - I swear we did not name it. 🤡
Vizro is a toolkit for creating modular data visualisation applications, and it is built on Plotly and Dash. It integrates with Kedro and allows you to trigger pipeline re-runs from the dashboard views. 🚀
You can:
• 📊 See a live demo
• ⭐ Visit and star the GitHub repo (if you want)
• 📚 Check out the documentation
• 🦠 Read a Medium blog post about VizroJo Stichbury
09/27/2023, 9:11 AMThe use of direct language with consideration given to different levels of expertise is very effective. This combined with the clear and user friendly navigation makes the documentation very attractive. The layout was really clear.Thanks to all in the team for believing in great communication! 🤩 🏆 And to you the users for giving us feedback, and contributions, to guide us, inspire us, and make it a worthwhile and satisfying task.