Any help for great expectations would really be ap...
# questionsd
Dhaval Thakkar
12/20/2022, 3:02 PMAny help for great expectations would really be appreciated
d
Deepyaman Datta
12/20/2022, 3:05 PMThe Kedro team does not maintain a Great Expectations integration. For general help on Great Expectations, use their support channels. For a sample Kedro integration, look in hook examples--but you will need to extend it and learn GE as necessary.
d
Dhaval Thakkar
12/20/2022, 3:11 PMI've looked into the hooks integration for GE but it's causing a lot of issues. If there's one end-to-end example as to the steps required to use it, it'd really be great. This is causing a lot of blockers for me to use Kedro
d
Deepyaman Datta
12/20/2022, 7:47 PMWhat issues is it causing? Are they Kedro-related? What have you tried to resolve them?
m
Maurits
12/20/2022, 11:21 PMLet's connect @Dhaval Thakkar. After my question of Monday, I'm planning to test Kedro & Great Expectations also this week
d
Dhaval Thakkar
12/21/2022, 5:20 AM@Deepyaman Datta I wouldn't have posted a question here if I tried everything to solve them, right?
@Maurits Let me know when can we connect so that I can even get it done. Really appreciate your helpš
Dhaval Thakkar
12/21/2022, 10:46 AM@Deepyaman Datta As far as I can understand, there is a version mismatch v2/v3 for great-expectations. It's listed in the issue here
š 1
Dhaval Thakkar
12/21/2022, 11:08 AMAlso this was written on the official site of great-expectations while 0.13.8 version was launched back in Feb 2021
Copy code
V3 (Batch Request) API vs The V2 (Batch Kwargs) API
The 0.13 major release of Great Expectations introduced a group of new features based on "new style" Datasources and Modular Expectations that we call the V3 (Batch Request) API. The V2 (Batch Kwargs) API will be deprecated in the future.š 1
e
Erwin
12/21/2022, 2:32 PMhere a e2e example with the last version of both libraries.
Notice that now in GE is necessary to use checkpoints (which are linked to a suite).
https://github.com/erwinpaillacan/kedro-great-expectations-example
š 2
d
Deepyaman Datta
12/21/2022, 4:42 PM@Dhaval Thakkar Thanks! It's very helpful to get this context (e.g. the changes across versions). Maybe we can look into updating the docs to work with V3 (these aren't automatically tested), or--if you all have looked into it--any contributions to update this part of the docs is probably also welcome!
š 1
d
Dhaval Thakkar
12/23/2022, 7:19 AMGot this response from the great-expectations team
š 1
d
Deepyaman Datta
12/23/2022, 3:17 PMLet me bring this to the team's notice! We were talking about if it may be possible to put something together in conjunction with the GE team, but no promises (yet).
With regards to fast tracking, I doubt it's something we would be able to do; most of the team is on break right now, and then we would need to prioritize this alongside core Kedro development. We support a pretty limited set of integrations at this time.
FYI for anybody else visiting this thread, @Dhaval Thakkarās issue on GE is https://github.com/great-expectations/great_expectations/issues/6620.
K 1