Hello I have created an endpoint using FastAPI. Every request creates KeddroSession, run specific pipeline and return results:

with KedroSession.create(...) as session:

context = session.load_context()

cat = context.catalog

return SequentialRunner().run(catalog=cat, pipeline=pipeline)[...]

I have just discovered, that my API is single-threaded and new requests have to wait untill previeous requests finish. Does anybody solution for this problem and knows how to make API multithreaded?

Fastapi support async.

I didn't mention, that I have already used async. I am wondering now, if creating Kedro session can block async in some way.

How do you start the app?

I start it with uvicorn, just

uvicorn endpoint.api:app --host 0.0.0.0 --port 80

Let me have a look

kedro_pipeline_run

is not async, so there’s nothing to “await” there, check: https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.loop.run_in_executor

I will check it, thanks 🙂

Hi, one more question, how do you run the kedro pipeline in any endpoint(cloud, fastapi, flask, django, etc) to make predictions from a model with all data transformation? In a similar way as above or do you do it entirely differently (e.g. you don't use kedro but just the code from the node? or scikit-learn pipeline? )

@Dawid Bugajny Are you able to resolve the issue with

async def

?

@Nok Lam Chan We fix the problem based on the below example

@app.post("/async-endpoint")
async def test_endpoint():
    loop = asyncio.get_event_loop()
    with concurrent.futures.ProcessPoolExecutor(max_workers=1) as pool:
        result = await loop.run_in_executor(pool, kedro_pipeline_run, *args)

https://stackoverflow.com/questions/63169865/how-to-do-multiprocessing-in-fastapi It was also possible to remove

async

from the function, but then the threads were interrupted only when kedro loaded the data, which made it faster, but not as fast as creating a new process (we have a dozen or so queries per minute, so there is no problem with creating a new process).