Skip to content Skip to sidebar Skip to footer

Memory Use If Multiprocessing Queue Is Not Used By Two Separate Processes

I have a thread in my python program that acquires images from a webcam and puts them in a multiprocessing queue. A separate process then takes these images from the queue and does

Solution 1:

I figured out what was happening, so I'll post it here for the benefit of anyone that stumbles across question.

My memory problem resulted from a numpy bug in numpy version 1.16.0. Reverting to numpy version 1.13.3 resolved the problem.

To answer the basic question: No, there is no need to worry which thread/process is doing the consuming (get) and which thread/process is doing the producing (put) for multiprocessing queues. There is nothing special about multiprocessing queues with respect to garbage collection. As kindall explains in response to a similar question:

When there are no longer any references to an object, the memory it occupies is freed immediately and can be reused by other Python objects

I hope that helps someone. In any case, the numpy bug should be resolved in the 1.16.1 release.


Post a Comment for "Memory Use If Multiprocessing Queue Is Not Used By Two Separate Processes"