https://kedro.org/ logo
Join Slack
Powered by
Hey, hope that this finds you well. Leaked Semaph...
# questions
c
Hey, hope that this finds you well. Leaked Semaphore object interrupts pipeline while node seem healthy I'm running a pipeline, everything goes well and then I get:
Copy code
segmentation fault  kedro run
/Users/cyril_verluise/opt/anaconda3/envs/my-project/lib/python3.9/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
and the pipeline is interrupted. What is strange is that, if I run the "current" node when the pipeline stops in isolat, everything goes well. Note also that I'm not implementing multiprocessing myself. Any idea? I'll keep googling on my side, just in case anyone has already seen that before. Set up kedro 0.18.13
n
Are you using parallelrunner?
or is this the libraries that you are using
c
no, not using parallelrunner
is this the libraries that you are using
might be the case
you mean that? https://docs.kedro.org/en/stable/kedro.runner.ParallelRunner.html if yes, then no, I'm not using ParallelRunner
👍🏼 1
n
Then it is one of the libraries that you are using. Kedro only use multi-processing when you use 
ParallelRunner
j
n
Can you show your 
settings.py?
The issue that @Juan Luis show may related, but that should be fixed 1 year ago. https://github.com/kedro-org/kedro/pull/1614
Any chance that you are importing the 
ShelveStore
?
c
Thanks. No, not using ShelveStore
So, I guess that I have identified the node creating the issue. It contained a call to tqdm, which is said to be responsible for such errors in some cases. I removed that call: • I still have an error • but the error does not refer to a specific multiprocessing explicitly, it's only 
segmentation fault  kedro run
not sure it helps much
n
Ah - do you run into the error if you just run the python function (without kedro)?
c
nope
n
https://github.com/tqdm/tqdm/issues/1197 - Not sure can you remove the tqdm from your code, it’s the library that used to render your progress bar.
c
All nodes work fine in isolation.
can you remove the tqdm from your code
yes, I did. It did not fix the issue - only changed the error message
n
Hmm… I am still unsure whether it is a Kedro issue or some 3rd party libraries problem.
All nodes work fine in isolation.
With or without Kedro?
c
both
n
I see, then I am more inclined to some libraries issue rather than Kedro’s
Are you using some more exotic computation libraries?
c
I'm using pdal https://pdal.io/en/latest/ to ingest and process lidar data. That seems to be the node causing the issue (if I run the following nodes, it works well, if I run them with the lidar ingestion/processing node, i get the errror)
I suspect there is some multiprocessing or so happening in pdal.
n
Lidar data, sounds fun😁. So it could be the libraries problem as I do see people reporting seg fault issues (21 issues ) https://github.com/PDAL/PDAL/issues/3848
💥 2
c
ohoho!
looks we have our top candidate here
thanks a lot for the help. Sorry to disturb you for non kedro issues.
n
Did you manage to find the issue?
c
hmm, pdal is not the most straightforward tool to be fair. In short, I'm not using the filters mentioned in the issue, but still find it most likely that it's the cause. Now, fixing the issue is another story
👍🏼 1
2 Views
Open in Slack
PreviousNext
Powered by