[zeromq-dev] Problem with fileserver / fileclient

Stathis Gkotsis stathis.gkotsis at commsquare.com
Mon Nov 19 12:09:40 CET 2012


Hi Pieter,

Thanks a lot for your help. I have done some further tests and it seems that setting the HWM on the server to some higher value generally helps, but setting it to 20 is not always enough. Setting it to 30 seems to work, but I am not sure if that is deterministic.

What would you say a value that always guarantees proper functioning would be, until the issue is fixed?
Would this value only depend on the size of the PIPELINE, rather than on other random factors as well?

Thanks,
Stathis

-----Original Message-----
From: zeromq-dev-bounces at lists.zeromq.org [mailto:zeromq-dev-bounces at lists.zeromq.org] On Behalf Of Pieter Hintjens
Sent: Monday, November 19, 2012 9:26 AM
To: ZeroMQ development list
Subject: Re: [zeromq-dev] Problem with fileserver / fileclient

It's definitely timing dependent but the limit isn't two times, it still failed with a HWM 3x higher than the calculated amount.

Sending more or fewer frames had no impact, as you say.

So let's treat this as a documentation issue for now, and we can think about how to process LWM commands faster in the future.

-Pieter

On Mon, Nov 19, 2012 at 2:36 PM, Min <miniway at gmail.com> wrote:
> At the pipe.cpp
>
> the variables, msgs_written and peers_msgs_read, which are used at the
> HWM full calculation are updated only at the last frame.
>
> My guess is that the peers_msg_reads can be late updated because it is
> updated asynchronously.
>
> For example, the dealer got chunks and sent subsequent requests, but
> the router get the new requests before it get 'activate_write'.
> The outgoing pipe can grow to twice of PIPELINE at maximum in an
> extreme case.
>
>
> Thanks
> Min
>
>
> On Mon, Nov 19, 2012 at 10:23 AM, Pieter Hintjens <ph at imatix.com> wrote:
>>
>> I've updated the doc to make this clear, and will change the examples
>> in the Guide accordingly.
>>
>> Thanks for spotting this problem!
>>
>>
>> https://github.com/hintjens/libzmq/commit/7533ebb33d1cc6f23061e0228bf
>> 7eb22713aaa17
>>
>> -Pieter
>>
>> On Mon, Nov 19, 2012 at 10:01 AM, Pieter Hintjens <ph at imatix.com> wrote:
>> > Stathis,
>> >
>> > I've tested this in detail.
>> >
>> > Tthere seems to be a problem with the HWM in the ROUTER, which is
>> > dropping outgoing messages unexpectedly. That is, the HWM setting
>> > of
>> > 10 isn't actually enough for 10 outgoing messages.
>> >
>> > What it looks like is that the HWM is in fact per message frame,
>> > not per message. So in this case you'd have to use:
>> >
>> >     zsocket_set_hwm (router, PIPELINE * 2);
>> >
>> > In the router since the identity frame is counted in the HWM
>> > calculation.
>> >
>> > This is a bug IMO, or at least it's rather surprising.
>> >
>> > -Pieter
>> >
>> >
>> >
>> >
>> > On Mon, Nov 19, 2012 at 3:13 AM, Stathis Gkotsis
>> > <stathis.gkotsis at commsquare.com> wrote:
>> >> Indeed, sorry about this, here are the two links to the code:
>> >>
>> >> http://pastebin.com/1DLa4e2J
>> >> http://pastebin.com/cbJUxJB4
>> >>
>> >> Thanks,
>> >> Stathis
>> >> ________________________________________
>> >> From: zeromq-dev-bounces at lists.zeromq.org
>> >> [zeromq-dev-bounces at lists.zeromq.org] On Behalf Of Min
>> >> [miniway at gmail.com]
>> >> Sent: Sunday, November 18, 2012 7:36 PM
>> >> To: ZeroMQ development list
>> >> Subject: Re: [zeromq-dev] Problem with fileserver / fileclient
>> >>
>> >> The two links seem to be identical.
>> >>
>> >> Could you upload the client code also ?
>> >>
>> >> Thanks
>> >> Min
>> >>
>> >>
>> >> On Mon, Nov 19, 2012 at 1:22 AM, Stathis Gkotsis
>> >> <stathis.gkotsis at commsquare.com<mailto:stathis.gkotsis at commsquare.
>> >> com>>
>> >> wrote:
>> >> Hello,
>> >>
>> >>    I am using zeromq v3.2.1-RC2 on Ubuntu 10.04 / Ubuntu 12.04
>> >> 64bit. I created two simple apps: one fileserver and one
>> >> fileclient, based on the "File Transfer" example code in the
>> >> documentation
>> >> (http://zguide.zeromq.org/page:all#header-151 , model 3). The code
>> >> for these two example apps can be found here: http://pastebin.com/AuGU9EDC and here:
>> >> http://pastebin.com/LusbjGCa . As you can see, I use a file of 100KB.
>> >>
>> >> The problem is that when I start the server and then run the
>> >> client multiple times, sometimes it receives less than 102400
>> >> bytes, like: 98304 or
>> >> 94208 or 90112 bytes. For example, when I run the client 1000
>> >> times with an interval of 100ms between the runs, for 970 runs it
>> >> receives 12400 bytes, while for the rest (30 runs) it receives
>> >> less bytes. I am bit puzzled by this, could you help?
>> >>
>> >> Thanks,
>> >> Stathis
>> >> _______________________________________________
>> >> zeromq-dev mailing list
>> >> zeromq-dev at lists.zeromq.org<mailto:zeromq-dev at lists.zeromq.org>
>> >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>> >>
>> >> _______________________________________________
>> >> zeromq-dev mailing list
>> >> zeromq-dev at lists.zeromq.org
>> >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
_______________________________________________
zeromq-dev mailing list
zeromq-dev at lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev



More information about the zeromq-dev mailing list