[Kedro Pipeline, deployment with Prefect] Hey ever...
# questions
d
[Kedro Pipeline, deployment with Prefect] Hey everyone, has anyone been able to successfully create a Prefect 2 deployment from a Kedro project? If so, could you share some insights or tips on how to go about it? I'm currently working on a project and would love to integrate Prefect into my Kedro pipeline, but I'm not sure where to start. Any help would be greatly appreciated! I am trying to convert the
register_prefect_flow.py
script given by Kedro documentation. Unfortunately, I don't seem to succeed because Prefect 1.0 API is different from Prefect 2.0. Thanks in advance !
d
Where are you struggling in trying to convert the flow? I just took a quick look, but some things to consider: •
Flow
still exists in Prefect 2, but I don't think it's a context manager (at least, looking at https://github.com/PrefectHQ/prefect/blob/main/src/prefect/flows.py, I didn't see any
__enter__
or
__exit__
). So either you construct a
Flow
object yourself or use the
@flow
decorator programmatically. • Task API is different. Also, maybe you can just use Prefect 2 native orchestration of Python functions? Would recommend starting with a super simple pipeline (e.g. Iris and working out what needs to be changed). I'm also not 100% sure, but I think it may make sense to look at Prefect Blocks for storage.