[zeromq-dev] Various Majordomo protocol versions

[Brett Viren]

Hi Robert,

Bob Eby <ebyrob at gmail.com> writes:

> Brett Viren <brett.viren at gmail.com> wrote:
>> And, what I *really* need is the Majordomo pattern but with
>> CLIENT/SERVER for thread-safety.  
>
> Pardon if I'm not quite following here, but isn't every ZeroMQ "message" an
> atomic operation already?  In terms of a protocol library what more are you
> expecting in providing thread safety?  Can't the rest be handled with a
> minor wrapper if even that much is necessary?  (multi-thread apps probably
> already have some mechanisms built in but I digress...)

Below is my understanding which led me to this.  If any of it is wrong,
please let me know as I'd rather stick with non-draft sockets in my
application.

1. yes, through the chain: socket->wire->socket, ZeroMQ claims that
   messages are atomic (and I have no reason to doubt it).

2. between app->socket and socket->app, multipart messages are NOT
   handled in an atomic manner.  Ie, one may call zmq_send() with the
   ZMQ_SENDMORE flag a number of times before a final call without the
   "send more" completes construction of the message inside the socket.

3. if the app uses the "non-thread-safe" sockets (notably DEALER) from
   multiple threads, there will be problems.  Problems may be expected
   even if the app promises to not interleave send/recv calls between
   threads or promises to finish a series of multipart send()/recv()
   calls from a common socket.  (This is the key point to my current
   understanding and I'd be grateful to be proven wrong).

4. my app runs from TBB flow graph where a node holding a socket may
   execute on different threads over its lifetime as determined by TBB's
   thread pool rules.  Each individual execution of a node is (or can
   be) guaranteed to be atomic on a given thread while subsequent
   execution may be from a different thread.  No multipart send()/recv()
   calls would span executions but my understanding is that's not enough
   of a guarantee.


As far as a "wrapper", indeed one approach I considered is to write a
"device" with a SERVER on one end and a DEALER on the other and then
execute it on a guaranteed dedicated thread.  It's job would be to
translate between an "almost-Majordomo" client sub-protocol on its
SERVER to one of the MDP client sub-protocols on its DEALER.

However, given the variants of Majordomo protocol and the state of their
implementations, I feel I might be better off dispensing with
ROUTER/DEALER and writing a MDP-like CLIENT/SERVER protocol (and in fact
have started that).


Do please disabuse me of any statement that's not correct here!

Thanks,
-Brett.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20200228/0917c0bb/attachment.sig>