[zeromq-dev] Does ZMQ "Over Send" Using OpenPGM (ZMQ 2.0.10)

[Bob Beaty]

Guys,
  I've got a stumper and I just can't seem to figure this guy out. Any help you can give me would be very much appreciated.
  I'm using a very simple ZeroMQ implementation. I have a ZeroMQ transmitter that's setting up a socket of the ZMQ_PUB type... setting the ZMQ_RATE to 200000 (that's 200Mbps), and then connecting it to a "epgm://" URL. As far as I can tell, this is all working perfectly.
  It's on the receiver that I'm having issues.
  On the receiver, I make a new zmq::socket_t of type ZMQ_SUB and call setsocketopt(ZMQ_SUBSCRIBE, "", 0) - to get everything on the URL I'll connect to.
  I then connect the socket to the same "epgm://" URL and it's receiving fine.
  Well... *more* than fine.
  I'm monitoring the transmitter and it's sending about 4,000 msgs/sec.
  I'm monitoring the receiver and it's receiving about 800,000 to 1,000,000 msgs/sec. When I remove duplicates (I can tell which messages are dupes) I get the 4,000 msgs/sec I'm sending, but I'm getting a TON of retries.
  It's as if the OpenPGM is hammering the receivers with repeat sends just to make sure they get them.
  In addition to this, my receiver runs for a few minutes and then logs the assertion failure:

Assertion failed: *tmpbuf > 0 (zmq_decoder.cpp:60)

and when I get into the code, It looks like that guy is saying there's nothing in the message to decode, but a TODO note indicates that something needs to be done about handling oversized messages.
  This seems to mesh with the 800,000 to 1,000,000 msgs/sec that I'm getting.
  If I'm getting "over sends" maybe that could become "oversized", and trigger the problem. If I try a different data source (all these are exchange ticker plants) that's sending about 4 msgs/sec we still get "too many" messages (about 450,000 msgs/sec) and after a few minutes, we get the same assertion issue.
  So what do I need to provide to help with this issue?
  The code is wrapped up in a ticker plant with several classes so it's not trivial to unroll this into a minimal failure case, but I'll work on it.
  My first real question is about the numbers of incoming messages... is this expected?

    Thanks,
        Bob (drbob at TheManFromSPUD.com)
    The Man from S.P.U.D.
    We will write no code before it's designed.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20101019/5201038e/attachment.htm>