Hello everyone!
I am exploring ways to implement data validation on data pipeline output using Kedro and Databricks. Do you have any suggestions about the modules I can use for this?
d
datajoely
04/03/2024, 6:08 PM
So if you want to do runtime checks Iām a big fan of pandera
š 1
datajoely
04/03/2024, 6:09 PM
if you want to do checks at rest Great Expectations is often a good shout
š 1
o
Ojal Sethi
04/03/2024, 7:23 PM
Thanks @datajoely, my use case is more aligned with runtime check. I will check out pandera.