https://kedro.org/ logo
#questions
Title
# questions
c

Cyril Verluise

09/25/2023, 2:27 PM
Hey, hope that this finds you well. Leaked Semaphore object interrupts pipeline while node seem healthy I'm running a pipeline, everything goes well and then I get:
Copy code
segmentation fault  kedro run
/Users/cyril_verluise/opt/anaconda3/envs/my-project/lib/python3.9/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
and the pipeline is interrupted. What is strange is that, if I run the "current" node when the pipeline stops in isolat, everything goes well. Note also that I'm not implementing multiprocessing myself. Any idea? I'll keep googling on my side, just in case anyone has already seen that before. Set up kedro 0.18.13
n

Nok Lam Chan

09/25/2023, 2:29 PM
Are you using parallelrunner?
or is this the libraries that you are using
c

Cyril Verluise

09/25/2023, 2:34 PM
no, not using parallelrunner
is this the libraries that you are using
might be the case
you mean that? https://docs.kedro.org/en/stable/kedro.runner.ParallelRunner.html if yes, then no, I'm not using ParallelRunner
👍🏼 1
n

Nok Lam Chan

09/25/2023, 2:54 PM
Then it is one of the libraries that you are using. Kedro only use multi-processing when you use
ParallelRunner
j

Juan Luis

09/25/2023, 2:55 PM
n

Nok Lam Chan

09/25/2023, 3:20 PM
Can you show your
settings.py?
The issue that @Juan Luis show may related, but that should be fixed 1 year ago. https://github.com/kedro-org/kedro/pull/1614
Any chance that you are importing the
ShelveStore
?
c

Cyril Verluise

09/25/2023, 3:22 PM
Thanks. No, not using ShelveStore
So, I guess that I have identified the node creating the issue. It contained a call to tqdm, which is said to be responsible for such errors in some cases. I removed that call: • I still have an error • but the error does not refer to a specific multiprocessing explicitly, it's only
segmentation fault  kedro run
not sure it helps much
n

Nok Lam Chan

09/25/2023, 3:26 PM
Ah - do you run into the error if you just run the python function (without kedro)?
c

Cyril Verluise

09/25/2023, 3:27 PM
nope
n

Nok Lam Chan

09/25/2023, 3:27 PM
https://github.com/tqdm/tqdm/issues/1197 - Not sure can you remove the tqdm from your code, it’s the library that used to render your progress bar.
c

Cyril Verluise

09/25/2023, 3:28 PM
All nodes work fine in isolation.
can you remove the tqdm from your code
yes, I did. It did not fix the issue - only changed the error message
n

Nok Lam Chan

09/25/2023, 3:32 PM
Hmm… I am still unsure whether it is a Kedro issue or some 3rd party libraries problem.
All nodes work fine in isolation.
With or without Kedro?
c

Cyril Verluise

09/25/2023, 3:34 PM
both
n

Nok Lam Chan

09/25/2023, 3:34 PM
I see, then I am more inclined to some libraries issue rather than Kedro’s
Are you using some more exotic computation libraries?
c

Cyril Verluise

09/25/2023, 3:37 PM
I'm using pdal https://pdal.io/en/latest/ to ingest and process lidar data. That seems to be the node causing the issue (if I run the following nodes, it works well, if I run them with the lidar ingestion/processing node, i get the errror)
I suspect there is some multiprocessing or so happening in pdal.
n

Nok Lam Chan

09/25/2023, 3:41 PM
Lidar data, sounds fun😁. So it could be the libraries problem as I do see people reporting seg fault issues (21 issues ) https://github.com/PDAL/PDAL/issues/3848
💥 2
c

Cyril Verluise

09/25/2023, 3:44 PM
ohoho!
looks we have our top candidate here
thanks a lot for the help. Sorry to disturb you for non kedro issues.
n

Nok Lam Chan

09/25/2023, 4:06 PM
Did you manage to find the issue?
c

Cyril Verluise

09/25/2023, 4:10 PM
hmm, pdal is not the most straightforward tool to be fair. In short, I'm not using the filters mentioned in the issue, but still find it most likely that it's the cause. Now, fixing the issue is another story
👍🏼 1