https://kedro.org/ logo
#questions
Title
# questions
d

Dawid Bugajny

07/17/2023, 9:14 AM
Hello I have created an endpoint using FastAPI. Every request creates KeddroSession, run specific pipeline and return results:
with KedroSession.create(...) as session:
context = session.load_context()
cat = context.catalog
return SequentialRunner().run(catalog=cat, pipeline=pipeline)[...]
I have just discovered, that my API is single-threaded and new requests have to wait untill previeous requests finish. Does anybody solution for this problem and knows how to make API multithreaded?
n

Nok Lam Chan

07/17/2023, 9:15 AM
Fastapi support async.
d

Dawid Bugajny

07/17/2023, 9:18 AM
I didn't mention, that I have already used async. I am wondering now, if creating Kedro session can block async in some way.
m

marrrcin

07/17/2023, 9:18 AM
How do you start the app?
First, see if the blocking happens when you run your app with uvicorn: https://fastapi.tiangolo.com/deployment/manually/
d

Dawid Bugajny

07/17/2023, 9:23 AM
I start it with uvicorn, just
Copy code
uvicorn endpoint.api:app --host 0.0.0.0 --port 80
And I have this fragment in my api code
project_path = Path.cwd()
metadata = bootstrap_project(project_path)
There is code from my API
Untitled.py
n

Nok Lam Chan

07/17/2023, 10:21 AM
Let me have a look
m

marrrcin

07/17/2023, 10:58 AM
kedro_pipeline_run
is not async, so there’s nothing to “await” there, check: https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.loop.run_in_executor
👍 1
There’s also a nice lib from the FastAPIs author: https://asyncer.tiangolo.com/tutorial/asyncify/
❤️ 1
d

Dawid Bugajny

07/17/2023, 11:10 AM
I will check it, thanks 🙂
a

Alan Tetich

07/17/2023, 11:39 AM
Hi, one more question, how do you run the kedro pipeline in any endpoint(cloud, fastapi, flask, django, etc) to make predictions from a model with all data transformation? In a similar way as above or do you do it entirely differently (e.g. you don't use kedro but just the code from the node? or scikit-learn pipeline? )
n

Nok Lam Chan

07/17/2023, 5:18 PM
@Dawid Bugajny Are you able to resolve the issue with
async def
?
a

Alan Tetich

07/17/2023, 6:13 PM
@Nok Lam Chan We fix the problem based on the below example
Copy code
@app.post("/async-endpoint")
async def test_endpoint():
    loop = asyncio.get_event_loop()
    with concurrent.futures.ProcessPoolExecutor(max_workers=1) as pool:
        result = await loop.run_in_executor(pool, kedro_pipeline_run, *args)
https://stackoverflow.com/questions/63169865/how-to-do-multiprocessing-in-fastapi It was also possible to remove
async
from the function, but then the threads were interrupted only when kedro loaded the data, which made it faster, but not as fast as creating a new process (we have a dozen or so queries per minute, so there is no problem with creating a new process).
👍 1