https://kedro.org/ logo
#questions
Title
# questions
f

Fabian

11/29/2022, 9:59 AM
Hi Team,
another beginner's question: I have created a pipeline that nicely analyzes my DataFrame. Now, I add a new level of complexity to my DataFrame and want to execute the pipeline on each level, similiar to a function in groupby.apply.
Can I do this without modifiying the pipeline itself? E.g., splitting the DataFrame ahead of the pipeline and remerging it afterwards while leaving the existing pipeline as it is?
d

datajoely

11/29/2022, 10:03 AM
I guess if you use a Dask runner this happens explicitly, but I think you’d have to manage the splitting and reconstitution process yourself + ParallelRunner if you’re using pure pandas.