[zeromq-dev] max PUB/SUB topics

[James Harvey]

And regarding the c example where you see the issue if you start publishing
before subscribing.
*" However, if instead of alternating pub/sub, I do the pubs first and then
the subs, I occasionally hit the 1000 limit. " *

That is never going to be reliable as the publisher will throw away the
data if there are no current subscriptions

Search up "late joiner" for possible solutions if that is a feature you
need to support.

On Wed, May 15, 2019 at 10:39 AM James Harvey <jamesdillonharvey at gmail.com>
wrote:

> Hi Benjy,
>
> From your original example you are mixing binary for subscription subjects
> and strings for publish subjects. Use the same for both.
>
> The issue is not with the number of subscriptions, its the fact the
> subjects you send do not match the subjects you subscribed to.
>
> You std::string insert() is expecting a signed char, not unsigned
>
> incoming_string.insert(0,1,i%256);
>
> after 127 its going to turn negative and the std string is going to fail
> to print a unicode character, spitting out unprintable data
>
> James
>
>
>
> On Wed, May 15, 2019 at 9:53 AM Bekritsky, Benjamin via zeromq-dev <
> zeromq-dev at lists.zeromq.org> wrote:
>
>> Justin,
>>
>>
>>
>> I rewrote your code in c and get the same results as you showed for
>> Python. I’m on Ubuntu 16.04 and using the “latest_release” branch (4.3.1).
>>
>>
>>
>> https://gist.github.com/bekritsk/c73af9c3080fb976328b053e0c454166
>>
>>
>>
>> However, if instead of alternating pub/sub, I do the pubs first and then
>> the subs, I occasionally hit the 1000 limit.
>>
>>
>>
>> I tried doing the same thing in Python, but Python seems to work whether
>> the pub/subs are alternating or all the pubs are first followed by all of
>> the subs.
>>
>>
>>
>> Thanks,
>>
>> Benjy
>>
>>
>>
>> *From:* zeromq-dev <zeromq-dev-bounces at lists.zeromq.org> *On Behalf Of *Justin
>> Karneges
>> *Sent:* Tuesday, May 14, 2019 6:01 PM
>> *To:* Jamie Stewart via zeromq-dev <zeromq-dev at lists.zeromq.org>
>> *Subject:* Re: [zeromq-dev] max PUB/SUB topics
>>
>>
>>
>> [External Email]
>>
>> Works for me with 4.2.5 on Ubuntu 18.04.
>>
>>
>>
>> https://gist.github.com/jkarneges/0b8f6e6d70c23807de0ad02c8ed72fcb
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__gist.github.com_jkarneges_0b8f6e6d70c23807de0ad02c8ed72fcb&d=DwMFaQ&c=Qwsh1H-X9ypOoLLEcAIltQ&r=wDgxT19weImS5vjsvnN8A59V8HMU1jmK8Tt64xQOZwA&m=if5ZcgfMqCPKYZ553AP14LU19id8HIg6pMiaE2ez4Yw&s=9owZiXJwjq6awqaVthdwfEAbzrKv2cMkcA2YEHTGbZA&e=>
>>
>>
>>
>> On Tue, May 14, 2019, at 12:59 AM, Bekritsky, Benjamin via zeromq-dev
>> wrote:
>>
>> Justin,
>>
>>
>>
>> Thanks for the guidance, but I set the HWMs to 0 and neither application
>> crashed. I also played with changing the HWMs to 2048 and am still
>> encountering the same issue.
>>
>>
>>
>> I then subscribed to “all” and received the messages from all the topics
>> (0-1024);
>>
>>
>>
>> It seems as though it is limiting the number of subscriptions to 1000. If
>> I subscribed from 20 -1024, I get topics 20 – 1019.
>>
>>
>>
>> I am using lib 4.2.3 and the publisher and subscriber on the same Ubuntu
>> box.
>>
>>
>>
>> Thanks,
>>
>> Benjy
>>
>> *From:* zeromq-dev <zeromq-dev-bounces at lists.zeromq.org> *On Behalf Of *Justin
>> Karneges
>>
>> *Sent:* Tuesday, May 14, 2019 7:10 AM
>>
>> *To:* Jamie Stewart via zeromq-dev <zeromq-dev at lists.zeromq.org>
>>
>> *Subject:* Re: [zeromq-dev] max PUB/SUB topics
>>
>>
>>
>> [External Email]
>>
>> Hi,
>>
>>
>>
>> Subscribing sends a message to the publisher. I believe you're hitting
>> the default high water mark, and subscription messages are getting dropped.
>>
>>
>>
>> If I'm expecting lots of subscriptions, my rule of thumb is to set
>> SNDHWM=0 on the SUB socket and RCVHWM=0 on the PUB socket. This way the
>> application will sooner die than lose a subscription message.
>>
>>
>>
>> Justin
>>
>>
>>
>> On Mon, May 13, 2019, at 10:06 AM, Bekritsky, Benjamin via zeromq-dev
>> wrote:
>>
>> I am using libzmq and setting up topics using binary data. In other
>> words, my topic is the first 2 bytes of my message.
>>
>>
>>
>> I subscribe by setting a and b to the values I’m interested in:
>>
>>         for (uint16_t i = a; i <= b; i++)
>>
>>         {
>>
>>                 d = zmq_setsockopt(sub_socket, ZMQ_SUBSCRIBE, &i, 2);
>>
>>                 if (d)
>>
>>                 {
>>
>>                         printf ("E: subscription failed: %s\n", strerror
>> (errno));
>>
>>                         return -1;
>>
>>                 }
>>
>>         }
>>
>>
>>
>> My publisher publishes like this:
>>
>>                 string incoming_string = " this message";
>>
>>                 incoming_string.insert(0,1,i>>8);
>>
>>                 incoming_string.insert(0,1,i%256);
>>
>>
>>
>>                 int     rc = zmq_send(pub_socket,
>> incoming_string.c_str(), incoming_string.size(), 0);
>>
>>
>>
>> I can’t seem to subscribe to more than 1000 topics (values 0x0000 –
>> 0x03e7). From what I understand, thousands of topics should be available.
>>
>>
>>
>> Any guidance would be helpful.
>>
>>
>>
>> Thanks,
>>
>> Benjy
>>
>>
>>
>> Benjamin Bekritsky
>>
>> c/o Zebra Technologies
>>
>> 2 Negev St. (Motorola Building)
>>
>> 4th Floor
>>
>> Airport City 7019900
>>
>> Israel
>>
>>
>>
>>
>>
>>
>> ------------------------------
>>
>>
>>
>> - CONFIDENTIAL-
>>
>>
>>
>> This email and any files transmitted with it are confidential, and may
>> also be legally privileged. If you are not the intended recipient, you may
>> not review, use, copy, or distribute this message. If you receive this
>> email in error, please notify the sender immediately by reply email and
>> then delete this email.
>>
>>
>> ------------------------------
>>
>>
>>
>> - CONFIDENTIAL-
>>
>> This email and any files transmitted with it are confidential, and may
>> also be legally privileged. If you are not the intended recipient, you may
>> not review, use, copy, or distribute this message. If you receive this
>> email in error, please notify the sender immediately by reply email and
>> then delete this email.
>>
>> _______________________________________________
>>
>> zeromq-dev mailing list
>>
>> zeromq-dev at lists.zeromq.org
>>
>> https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.zeromq.org_mailman_listinfo_zeromq-2Ddev&d=DwMFaQ&c=Qwsh1H-X9ypOoLLEcAIltQ&r=wDgxT19weImS5vjsvnN8A59V8HMU1jmK8Tt64xQOZwA&m=if5ZcgfMqCPKYZ553AP14LU19id8HIg6pMiaE2ez4Yw&s=MFBykfTCfPngIL34eSeJ2V9NMDX85SYfQxo28ITv7RM&e=>
>>
>>
>>
>>
>>
>> _______________________________________________
>>
>> zeromq-dev mailing list
>>
>> zeromq-dev at lists.zeromq.org
>>
>> https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>>
>>
>>
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20190515/1f9454ef/attachment.htm>