https://kedro.org/ logo
#questions
Title
# questions
m

Muhammed Afnas

01/10/2024, 7:28 AM
Hi Everyone, I have a multi-user dash application which is backed by a kedro pipeline, I have local files (required as per the code structure) which is utilized in kedro through catalog (includes a lot of file details). There would be around 4 file at 4 respective folders which would flow through the pipeline sequentially through nodes. But here, i am facing the issue of file overlapping when multiple user works at the same/similar timeframe as the same file names being served throughout the kedro pipeline via catalog. I have an approach: 1. using a uuid to separate between the users such that local file would be created with uuid in the filename for each node processes, but in this case, what would be the possible approach to make sure kedro gets the intended file without editing the kedro catalog. Could anyone highlight if this is the best way and do we have any other kedro functionality that can be utilised to overcome this issue? Please reply if anyone already handled this situation before. Thank you for the help!
t

Takieddine Kadiri

01/10/2024, 9:12 AM
Hi Muhammed, If you're using kedro<0.19, you can look at kedro-boot. It offer a set of abstractions/interfaces that can help you integrate your Dash App with your Kedro project. You can refer to a project example to get started. For your use cases, maybe you need to use the template_params, they are used in the fastapi example, it allow you to inject a param into the catalog definition at each run iteration.
m

Muhammed Afnas

01/10/2024, 3:09 PM
Hi Takieddine, Thanks for you input! If I am not wrong, using this we could only add any extra parameters for the kedro process right and not data catalog changes. Please correct me if I wrong.