https://kedro.org/ logo
#questions
Title
# questions
d

Dhaval Thakkar

12/20/2022, 3:02 PM
Any help for great expectations would really be appreciated
d

Deepyaman Datta

12/20/2022, 3:05 PM
The Kedro team does not maintain a Great Expectations integration. For general help on Great Expectations, use their support channels. For a sample Kedro integration, look in hook examples--but you will need to extend it and learn GE as necessary.
d

Dhaval Thakkar

12/20/2022, 3:11 PM
I've looked into the hooks integration for GE but it's causing a lot of issues. If there's one end-to-end example as to the steps required to use it, it'd really be great. This is causing a lot of blockers for me to use Kedro
d

Deepyaman Datta

12/20/2022, 7:47 PM
What issues is it causing? Are they Kedro-related? What have you tried to resolve them?
m

Maurits

12/20/2022, 11:21 PM
Let's connect @Dhaval Thakkar. After my question of Monday, I'm planning to test Kedro & Great Expectations also this week
d

Dhaval Thakkar

12/21/2022, 5:20 AM
@Deepyaman Datta I wouldn't have posted a question here if I tried everything to solve them, right? @Maurits Let me know when can we connect so that I can even get it done. Really appreciate your helpšŸ˜Š
@Deepyaman Datta As far as I can understand, there is a version mismatch v2/v3 for great-expectations. It's listed in the issue here
šŸ‘ 1
Also this was written on the official site of great-expectations while 0.13.8 version was launched back in Feb 2021
Copy code
V3 (Batch Request) API vs The V2 (Batch Kwargs) API

The 0.13 major release of Great Expectations introduced a group of new features based on "new style" Datasources and Modular Expectations that we call the V3 (Batch Request) API. The V2 (Batch Kwargs) API will be deprecated in the future.
šŸ‘ 1
e

Erwin

12/21/2022, 2:32 PM
here a e2e example with the last version of both libraries. Notice that now in GE is necessary to use checkpoints (which are linked to a suite). https://github.com/erwinpaillacan/kedro-great-expectations-example
šŸ‘ 2
d

Deepyaman Datta

12/21/2022, 4:42 PM
@Dhaval Thakkar Thanks! It's very helpful to get this context (e.g. the changes across versions). Maybe we can look into updating the docs to work with V3 (these aren't automatically tested), or--if you all have looked into it--any contributions to update this part of the docs is probably also welcome!
šŸ‘ 1
d

Dhaval Thakkar

12/23/2022, 7:19 AM
Got this response from the great-expectations team
šŸ‘ 1
d

Deepyaman Datta

12/23/2022, 3:17 PM
Let me bring this to the team's notice! We were talking about if it may be possible to put something together in conjunction with the GE team, but no promises (yet). With regards to fast tracking, I doubt it's something we would be able to do; most of the team is on break right now, and then we would need to prioritize this alongside core Kedro development. We support a pretty limited set of integrations at this time. FYI for anybody else visiting this thread, @Dhaval Thakkarā€™s issue on GE is https://github.com/great-expectations/great_expectations/issues/6620.
K 1