Hi all, Has anyone used Kedro for building pipelines in a repo which also houses non-pipeline code, like lambda functions? I am bringing in Kedro, but also need some way of porting over our existing lambda functions to live in the same repo as our pipelines. Splitting them out into a separate repo is not really feasible due to the common code used by each and the extra work/dependency management that would introduce. We use aws sam as a framework of sorts within each of our lambda functions, so could just put them in

src/my_repo/lambdas/

next to

src/my_repo/pipelines/

and have some 3rd directory with shared code

src/my_repo/shared/

, but thought there may be a different way to go about this! Thanks!

Hi @Baden Ashford, interesting question! Is this more about the organisation of code or you need to use Kedro with Lambda?

Hi @Nok Lam Chan, probably the former. But can Kedro be used with Lambda?

It can be run with lambda, but we would recommend Step Function over Lambda if serverless is a requirement. See https://docs.kedro.org/en/stable/deployment/aws_step_functions.html

The challenge is just organising the code so we are not working against the kedro framework whilst being able to have the repo house both pipelines and lambdas. From the sounds of it there are no obvious footguns and it will just take some thought and discipline from us to keep things in order! Thanks for your help @Nok Lam Chan

Awesome, I think you have a good approach already : )

also please play back your learnings, it would be great to document this for future kedroid s