[zeromq-dev] cannot send big chunk via PUSH/PULL by pyzmq

[KIU Shueng Chuan]

Each element of the list is sent as a zeromq message frame. Each (short)
frame has a header overhead of 2 bytes. track=True also causes more
overhead per frame.

['0' for x in range(N)] is sent as N message frames.
['0'*N] is sent as 1 message frame. (Did you want to do this instead?)


On Tue, May 12, 2015 at 12:40 PM, Li Ma <skywalker.nick at gmail.com> wrote:

> I just created a list filled by 2 million zero and sended it out. Any
> problems?
>
> On Sun, Apr 26, 2015 at 8:27 PM, KIU Shueng Chuan <nixchuan at gmail.com>
> wrote:
> > I read your code as sending a multipart message composed of 2 million
> parts
> > each of size 1 byte. Is that right?
> >
> > On Apr 26, 2015 16:50, "Li Ma" <skywalker.nick at gmail.com> wrote:
> >>
> >> On Thu, Apr 23, 2015 at 7:50 PM, Pieter Hintjens <ph at imatix.com> wrote:
> >> > - please provide your test case as a minimal program
> >>
> >> Sure.
> >>
> >> Server part:
> >> context = zmq.Context(1)
> >> socket = context.socket(zmq.PULL)
> >> socket.bind("tcp://*:5555")
> >> while True:
> >>     #  Wait for next request from client
> >>     message = socket.recv_multipart()
> >>     print "hello: %s, time: %s\n" % (len(message), datetime.utcnow())
> >>
> >> Client part:
> >> context = zmq.Context(1)
> >>
> >> def send(size):
> >>     buffer = []
> >>     for i in range(size):
> >>         buffer.append('0')
> >>
> >>     socket = context.socket(zmq.PUSH)
> >>     socket.connect("tcp://localhost:5555")
> >>     print("Sending request %s ..." % datetime.utcnow())
> >>     tracker = socket.send_multipart(buffer, copy=False, track=True)
> >>     while(True):
> >>         if tracker.done:
> >>             socket.close(-1)
> >>             return '\nDONE, %s\n' % datetime.utcnow()
> >>         else:
> >>             time.sleep(.01)
> >>
> >> for i in range(3):
> >>     print send(2000000)
> >>
> >> > - don't do any TCP tuning unless you know why you're doing it
> >> >
> >>
> >> I use the distro-default configuration, but it doesn't work.
> >>
> >> > On Thu, Apr 23, 2015 at 8:35 AM, Li Ma <skywalker.nick at gmail.com>
> wrote:
> >> >> Hi all,
> >> >>
> >> >> I wanna test a simple application using PUSH/PULL to send a big chunk
> >> >> of data.
> >> >> The application is straightforward.
> >> >>
> >> >> Server part is to open a PULL socket while client part is sending
> >1MB
> >> >> message per time via PUSH.
> >> >>
> >> >> The socket is listened on the localhost only.
> >> >>
> >> >> Any messages less than 200KB are working well, but messages greater
> >> >> than 500KB is not working. The socket is stuck after handshake
> without
> >> >> any exceptions.
> >> >>
> >> >> By the way, I set a large size (>1MB) for SND/RCV BUF and tcp window
> >> >> scaling is set to 1.
> >> >>
> >> >> I'd appreciate any hints to make it work.
> >> >>
> >> >> --
> >> >>
> >> >> Li Ma (Nick)
> >> >> Email: skywalker.nick at gmail.com
> >> >> _______________________________________________
> >> >> zeromq-dev mailing list
> >> >> zeromq-dev at lists.zeromq.org
> >> >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >> > _______________________________________________
> >> > zeromq-dev mailing list
> >> > zeromq-dev at lists.zeromq.org
> >> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >>
> >>
> >>
> >> --
> >>
> >> Li Ma (Nick)
> >> Email: skywalker.nick at gmail.com
> >> _______________________________________________
> >> zeromq-dev mailing list
> >> zeromq-dev at lists.zeromq.org
> >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >
> >
> > _______________________________________________
> > zeromq-dev mailing list
> > zeromq-dev at lists.zeromq.org
> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >
>
>
>
> --
>
> Li Ma (Nick)
> Email: skywalker.nick at gmail.com
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20150512/40142454/attachment.htm>