[zeromq-dev] PUB/SUB unreliabiliity

Thomas Rodgers rodgert at twrodgers.com
Mon Jun 16 20:00:30 CEST 2014


set_sockopt accepts arguments by void* and casts them, later to int* iif
sizeof(int) == optvalen.  There's no "proper" narrowing conversion, and no
compiler warning.


On Mon, Jun 16, 2014 at 12:36 PM, Gerry Steele <gerry.steele at gmail.com>
wrote:

> Hi Pieter, you have struck on something there.
>
> Converting it to int seems to yield the correct behaviour.
>
> I guess the way setsockopt works type coercion doesn't happen.
>
> Embarrassing! But at least we got to the bottom of it.
>
> I was able to send billions of events without incurring loss. Apologies
> for taking everyones time.
>
> Thanks all.
>
> g
>
>
>
> On 16 June 2014 18:22, Pieter Hintjens <ph at imatix.com> wrote:
>
>> OK, just to double check, you're using ZeroMQ 4.0.x? In your test case
>> (which I'm belatedly looking at), you use a uint64_t for the hwm
>> values; it should be int. Probably not significant.
>>
>> On Mon, Jun 16, 2014 at 6:20 PM, Gerry Steele <gerry.steele at gmail.com>
>> wrote:
>> > In the patent email I have links to the minimal examples on
>> gist.github.com
>> >
>> > Happy to open an issue and commit them later on if that's what you need.
>> >
>> > Thanks
>> >
>> > On 16 Jun 2014 14:43, "Pieter Hintjens" <ph at imatix.com> wrote:
>> >>
>> >> Gerry, can you provide a minimal test case that shows the behavior?
>> >> Thanks.
>> >>
>> >> On Mon, Jun 16, 2014 at 12:49 PM, Gerry Steele <gerry.steele at gmail.com
>> >
>> >> wrote:
>> >> > Thanks Peter. I can't try this out till I get home but it is looking
>> >> > like
>> >> > hwm overflows.
>> >> >
>> >> > If you run the utilities you notice the drops start happening after
>> >> > precisely 1000 events in the first instance (which Is the default
>> hwm).
>> >> >
>> >> > There was another largely ignored thread about this recently
>> mentioning
>> >> > the
>> >> > same problem.
>> >> >
>> >> > I also tried setting the hwm values to a number greater than the
>> number
>> >> > of
>> >> > events and it seemed to have no effect either.
>> >> >
>> >> > g
>> >> >
>> >> > On 16 Jun 2014 09:32, "Pieter Hintjens" <ph at imatix.com> wrote:
>> >> >>
>> >> >> On Mon, Jun 16, 2014 at 9:10 AM, Gerry Steele <
>> gerry.steele at gmail.com>
>> >> >> wrote:
>> >> >>
>> >> >> > Big chunks of messages go missing mid flow and then pick up again.
>> >> >> > There
>> >> >> > is
>> >> >> > no literature that indicates that is expected behaviour.
>> >> >>
>> >> >> Right. The two plausible causes for this are (a) HWM overflows, and
>> >> >> (b) temporary network disconnects. You have excluded (a), though to
>> be
>> >> >> paranoid I'd probably add some temporary logging to libzmq's pub
>> >> >> socket to shout out if/when it does hit the HWM. To detect (b) you
>> >> >> could use the socket monitoring.  The third possibility is that
>> you're
>> >> >> doing something wrong with subscriptions... though that seems
>> >> >> unlikely.
>> >> >>
>> >> >> -Pieter
>> >> >> _______________________________________________
>> >> >> zeromq-dev mailing list
>> >> >> zeromq-dev at lists.zeromq.org
>> >> >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>> >> >
>> >> >
>> >> > _______________________________________________
>> >> > zeromq-dev mailing list
>> >> > zeromq-dev at lists.zeromq.org
>> >> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>> >> >
>> >> _______________________________________________
>> >> zeromq-dev mailing list
>> >> zeromq-dev at lists.zeromq.org
>> >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>> >
>> >
>> > _______________________________________________
>> > zeromq-dev mailing list
>> > zeromq-dev at lists.zeromq.org
>> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>> >
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>
>
>
> --
> Gerry Steele
>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20140616/58ef500f/attachment.htm>


More information about the zeromq-dev mailing list