Posts

Post not yet marked as solved
1 Replies
1.9k Views
I am trying to get a queue-based application to work.Basically, the multiprocessing queue is limited to a 32-bit number(32768)Ubuntu Linux 18.04 does not have this limitation. I am guessing this is a sysctl setting for the OS.ctx.BoundedSemaphore(maxsize). Does anyone know where the limitation is coming from? I have tried a number of settings in sysctl and rebooted the machie. (etc etc). The limit remains.colin_bitterfield$ python3 Python 3.7.4 (default, Sep 7 2019, 18:27:02) [Clang 10.0.1 (clang-1001.0.46.4)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> from multiprocessing import Queue >>> myQ = Queue(64000) Traceback (most recent call last): File "", line 1, in File "/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/context.py", line 102, in Queue return Queue(maxsize, ctx=self.get_context()) File "/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/queues.py", line 48, in __init__ self._sem = ctx.BoundedSemaphore(maxsize) File "/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/context.py", line 87, in BoundedSemaphore return BoundedSemaphore(value, ctx=self.get_context()) File "/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/synchronize.py", line 145, in __init__ SemLock.__init__(self, SEMAPHORE, value, value, ctx=ctx) File "/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/synchronize.py", line 59, in __init__ unlink_now) OSError: [Errno 22] Invalid argument >>> myQ = Queue(32767)On Ubuntu:Welcome to Ubuntu 18.04.3 LTS (GNU/Linux 4.15.0-1051-aws x86_64)python3 Python 3.6.8 (default, Oct 7 2019, 12:59:55) [GCC 8.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from multiprocessing import Queue >>> myQ = Queue(64000)
Posted Last updated
.