1 Reply
      Latest reply on Nov 5, 2019 1:36 AM by eskimo
      cbitterfield Level 1 Level 1 (0 points)

        I am trying to get a queue-based application to work.

         

        Basically, the multiprocessing queue is limited to a 32-bit number(32768)

         

        Ubuntu Linux 18.04 does not have this limitation. I am guessing this is a sysctl setting for the OS.

         

        ctx.BoundedSemaphore(maxsize). Does anyone know where the limitation is coming from? I have tried a number of settings in sysctl and rebooted the machie. (etc etc). The limit remains.

         

         

         

        colin_bitterfield$ python3
        Python 3.7.4 (default, Sep  7 2019, 18:27:02)
        [Clang 10.0.1 (clang-1001.0.46.4)] on darwin
        Type "help", "copyright", "credits" or "license" for more information.
        >>> from multiprocessing import Queue
        >>> myQ = Queue(64000)
        Traceback (most recent call last):
          File "", line 1, in 
          File "/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/context.py", line 102, in Queue
            return Queue(maxsize, ctx=self.get_context())
          File "/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/queues.py", line 48, in __init__
            self._sem = ctx.BoundedSemaphore(maxsize)
          File "/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/context.py", line 87, in BoundedSemaphore
            return BoundedSemaphore(value, ctx=self.get_context())
          File "/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/synchronize.py", line 145, in __init__
            SemLock.__init__(self, SEMAPHORE, value, value, ctx=ctx)
          File "/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/synchronize.py", line 59, in __init__
            unlink_now)
        OSError: [Errno 22] Invalid argument
        >>> myQ = Queue(32767)

         

        On Ubuntu:

        Welcome to Ubuntu 18.04.3 LTS (GNU/Linux 4.15.0-1051-aws x86_64)
        python3
        Python 3.6.8 (default, Oct  7 2019, 12:59:55)
        [GCC 8.3.0] on linux
        Type "help", "copyright", "credits" or "license" for more information.
        >>> from multiprocessing import Queue
        >>> myQ = Queue(64000)