Has anyone used a feature store with kedro? My use-case is that we have many (kedro) projects sharing the same features (but potentially joined with different spines). We mainly run batch pipelines and we want to have a way to re-use feature engineering code, but also have an easy way to track data lineage, metadata/documention,… We looked at Feast but that seems way overkill for our needs. Are there simpler alternatives? Or maybe ways to build it in Kedro directly?

I thought Feast is the "simple" option; at least, when I worked in the feature store space, it was the one that was just a feature store as opposed to a feature platform that managed the compute. The most bare-bones, pure-Kedro route would be to write a dataset for each feature set, joinable to the spine using entity key columns. This approach was standard across QuantumBlack projects 5+ years ago (maybe still is, wouldn't know). Are you specifically looking to store more of that other info with this?

Feast requires a dedicated deployment (e.g. you have to self-host several components). It’s also used to store pre-computed features in an online store. You don’t have to use it but you certainly have to set it up to have a properly working system. I think a bare-bones kedro route is simpler in this case.

If you view a minimal feature store as a single key-value store with the latest feature values, you could easily do that in Kedro. (Our feature store defaulted to Redis, you could use Valkey or some hosted solution.) Just requires another dataset write. Whenever you batch compute a feature, you want to write to both file storage (full historical record) and the KV store (fast lookup by feature key). https://huyenchip.com/2023/01/08/self-serve-feature-platforms.html is a pretty good writeup based on our experience researching and building a feature store, if you want additional context--and then you can back into the minimum viable solution for yourself from there.