Is there a good way on using Kedro with structured...
# questions
r
Is there a good way on using Kedro with structured streaming? Customizing runner or doing other similar things? I just started studying streaming and would be nice to have some good scaffolding on this. Also making the overall project structure similar between batch and streaming would be interesting in a learning curve as people would only see one single structure.
d
You can see an old example a friend and I put together here during a hackathon: https://github.com/deepyaman/kedro-streaming Definitely not production ready, but it worked and could serve as inspiration. I know that there are some people who work at QuantumBlack who have done something more recently, but I haven't seen it/no idea how it differs.
Also making the overall project structure similar between batch and streaming would be interesting in a learning curve as people would only see one single structure.
I have some thoughts I can hopefully share here in the next month or two (once things are public). 😉
❤️ 2