[zeromq-dev] How to properly close a socket and HWMRECV

Antonio Teixeira eagle.antonio at gmail.com
Mon Jul 11 13:21:17 CEST 2011

OOpsy Partial E-mail

Hello Martin & List.

In my point of view , ZMQ newbie here , is that with REQ/REP

Broker is REP

Worker is REQ

It would be something like :

while True.
 worker = socket.send('NEED WORK')

and on the boker
message = socket.recv()

if message = "NEED WORK":
 // check if i have work to give
if not work :
 socket.send('NO WORK')

The problem with this :
You are constantly  ( i could do sleep(1) but this still applies ) touching
the Broker for more work , this takes impact in used RAM + CPU and of course
reduces Total Broker capacity.

You are constantly exchanging traffic , this impacts the network Level ,
Machine link will be occupied with both "actual work" but also with load  in
order to get Work.
I know this is extremey small but 19 bytes + ZMQ + Network overhead * 32
Cores / second for only one server.

This was reason i picked PUSH/PULL

Sorry to brother you Martin but just trying to squeeze as much power as

António Teixeira

2011/7/8 Antonio Teixeira <eagle.antonio at gmail.com>

> Hello List.
> Since i have been able to bypass my little ( Big ) Bug i have come for a
> more interesting subject.
> My APP uses the pipeline topology , due to all the load-balance  and other
> features gained from this topology but i have found a problem.
> My APP requires alot of CPU so i have bypassed that with MultiProcessing +
> IPC and it works great , but i have all my processes based on the
> muliprocesing.cpu_count() value.
> (JFYI Messaging processing only takes in my case 0.007 seconds :) With SQL
> Query Included. against 0.12 using rabbitmq :P )
> Example
> Core 2 DUo > Cpu Count = 2
> If you do ps aux you should see 2 workers all goes fine
> I send 2 tasks and both cpus get used , great now the problem.
> If i send a third task to the client the socket will accept it and will
> probably "buffer" it for latter use .
> But i would like to set HWM on my DOWNSTREAM ( At the Client side not the
> Broker ) socket ,  looking at the source i have noticed 2 values :
> zmq.HWM and zmq.RCVHWM
> My objective is the following :
> On a 2 core machine , broker sends one Task
> it will occupy Process 1 and it takes 30 minutes to end this task.
> So we have one more working slot , now broker sends another task , the HWM
> should be reached now and ZMQ ignores further Work from Broker so it can
> overflow to other standby servers,
> This new task will occupy Process 2.
> Now Process 1 ends  and the Task Worker resets into network_socket.recv()
> The HWM should now allow one more task to flow trough.
> Is this the correct way or i'm messing something here ? Should i use HWM or
> ----
> Another question is any nice way to say hello socket please stop accepting
> data , clear any message on the buffer and terminate nicely without ugly
> Exception erros , specially if this socket is working with zmq.Device()
> Thank you all again
> Antonio Teixeira
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20110711/610d24da/attachment.htm>

More information about the zeromq-dev mailing list