[zeromq-dev] Java/Windows issues

[Robin Weisberg]

On Thu, Aug 20, 2009 at 4:16 AM, Martin Sustrik <sustrik at fastmq.com> wrote:

> Robin,
>
>     It's doable and the use case is a valid one so only question is
>>    whether to implement it in 0MQ/1.0 or rather postpone it for 0MQ/2.0.
>>
>>
>> Sounds like you need to replace PGM (e.g. w/ UDP), right? Thats a pretty
>> big job, I'd just implement in 0MQ2.
>>
>
> Actually, no. Both can be implemented on top of PGM. However, user has to
> choose which mechanism to use (1-to-many or many-to-many) as the semantics
> are slightly different.


>
> As for UDP I would avoid our own implementation of reliable multicast.
> Doing it from scratch would mean simply throwing away hundreds of manyears
> and experience invested into PGM.

Understood. If PGM supports the scenario its a good option to have and easy
place to start.


>
>
> The background is that 0MQ is trying to leverage existing technology rather
> than reinvent the wheel. That's the way to keep it thin and ultra-efficient.


>
> The examples are:
>
> 1. Use processor's synchronisation mechanism (locking the memory bus)
> rather than software-based locking (mutexes) for inter-thread
> synchronisation.
>
> 2. Use IP multicast instead of middleware-level TCP-based message
> distribution. The work is done by he hardware of the IP switch rather than
> by software-based message broker. In other words custom pub-sub mechanism is
> (at least partially) replaced by standard IGMP-based subscriptions.
>
> 3. Use IP multiplexing implemented in HW rather than middleware-level
> SW-based one (see AMQP channel concept for instance).
>
> 4. Use DNS name resolving mechanism rather than custom name resolving
> (applies only to 0MQ/2.0).
>
> etc.


>
>  Not a bad thing really as it allows you to eventually implement a lot of
>> fancy stuff: your own session ids/seq numbers for recovery which can then be
>> used further up the stack e.g. for persistent solutions, checks for
>> duplicate publishers of the same session, awareness of other
>> publishers/listeners and knowing when they arrive/disappear and what seq
>> numbers they are up to, throttling listeners who request recovery too much,
>> etc.
>>
>
> I would rather try to think about how to build these features on the top of
> the existing stack.
>
>     That's the way the multicast group works. Consumers are going to get
>>    everything. Filtering on the topic is done on the consuming side.
>>
>>
>> Understood, but some middleware implementations have the concept of topics
>> built in and filter before queueing the message to the app. Also they may
>> have out of process software message switches which receive multicast and
>> filter by topic before delivering to your process over TCP. Sometimes useful
>> if you have UI's that you don't want directly connected to a high volume
>> multicast group. I think this can just be implemented as a wrapper around a
>> lower level layer like below. This way the overhead isn't there when you
>> don't need it.
>>
>
> There are two distinct ideas here AFAIU:
>
> 1. Consumers would publish special subscribe/unsubscribe messages on the
> bus which would allow producers not to physically send any messages that
> have no consumers. The obvious problem are late-joining producers - they
> would have to get a snapshot of current subscriptions either from a
> centralised repository or from each consumer. Also, consumers would have to
> be monitored and subscriptions should be deleted when a consumer dies
> unexpectedly. The whole thing is more or less doable, but pretty complex.

I think there are actually 2 distinct ideas in here :)
  1) The concept of a topic which can be used for filtering messages on a
multicast group. Wildcards can be built on top of that which adds another
layer of complexity.
  2) The concept of publishers being aware of individual subscribers (using
join/leave/heartbeat messages). This is necessary for doing persistent
messaging and has advantages when it comes to debugging (e.g. in some
middlewares you can see who is "connected" to who from a web browser
embedded in the API which is embedded in your app). Its also useful so
publishers can realize that a particular listener is doing too many resend
requests and is hurting the performance for everyone else so you can
cutoff/throttle responses to it so you don't end up w/ a multicast storms. I
agree its VERY complex though as it requires lots of coordinating. My point
is IF you ever decide to tackle these problems you will find you have
implemented most if not all of the PGM functionality. You may still want to
support PGM for interoperability reasons.




>
> 2. Intermediate message switch. Yes, that's something that can be done in
> the future.


>         ZSock sock = new ZSock("pgm://eth0;239.151.1.1:100
>>        <http://239.151.1.1:100> <http://239.151.1.1:100>");
>>
>>        sock.addZSockListener( new ZSockListener() {
>>         void disconnected() {}
>>         void connected() {}
>>        });
>>
>>        ZRecoverableSock recovSock = new ZRecoverableSock ("Instance1",
>>        sock);//first param is the persistent name which remains the
>>        same across restarts so listeners can identify the sender even
>>        it is moved to a different host
>>
>>
>>    Yes. But would work only for unicast based transports as discussed
>>    above.
>>
>>
>> Why? The ZRecoverableSock will write every message from a file and can
>> replay the messages when any listener requests them. Any listener can
>> request a resend. You may want to add features so listeners request replays
>> over TCP or at least receive the replays over unicast UDP, so you don't bog
>> down other listeners. You could also add options to make it aware if
>> subscribers and tracks which messages have been received by which subscriber
>> and then removes the messages that no longer need to be stored.
>>
>
> The slow consumer problem. With TCP you can simply block the producer when
> consumer is not consuming fast enough and there's no more memory/disk space
> to store the messages to send. With multicast the same strategy would result
> in whole system blocking because of a single misbahaved/slow consumer.
>

I think the assumption that you want the publisher to stop sending if one
listener is slow is wrong. There are uses case for that but I don't think
its the most common one in a multicast world. Obviously you need to handle
the boundry condition of running out of disk space/memory and you can
provide the option of blocking or discarding oldest message/newest message
etc. You may be able to resolve the slow listener issue before than though
or you may have enough disk space to go the whole week w/ a listener down.


>
>
>
>         You could also have a ZReqReplySock that wraps any of the above
>>        sockets.
>>
>>
>>    Will be done in 0MQ/2.0
>>
>> OK, I'm curious to see how thats implemented in multicast. Does everyone
>> see the responses and have to filter them?
>>
>
> No point in implementing req/rep on top of multicast. It's always 1-to-1
> dialogue.


Strongly disagree w/ this. Using multicast to send requests to a server is
very useful (this is my preferred method) because you don't have to know
which machine a server is running on and you can have multiple servers load
balancing the requests. Also useful if you failover or just don't want to
always update your clients when a server moves. Sending replies on mutlicast
is less useful and requires other requestors to receive and ignore responses
not meant for them. (although still useful if you want to log
request/replies from an external process for debugging/stat collection
purposes w/o impacting performance of the critical path), other middleware
implementations implement this feature by having the the request message
contain information for sending the response, e.g. you could include an
ip/port which will receive unicast messages.

Going back to the topics above, if publishers were aware of subscribers and
whether they were behind or not you could allow the app to choose if it
wanted to keep publishing.



>
>     Additional sequence numbers on top of TCP/PGM are unaviodable for
>>    guaranteed delivery (the one that survives component restart). As
>>    you say, UDP is an option, however, it would mean reimplementing all
>>    the nice TCP functionality (thing of congestion control mechanisms)
>>    so I believe few additional bytes per message are worth of it.
>>
>> Congestion control is nice for point to point. Doesn't really apply much
>> in multicast of course, just because one guy goes down doesn't mean anybody
>> else is throttled.
>>
>
> There've been some research done on the topic. I've personally liked the
> PGMCC by Luigi Rizzo. The paper should be available on the web in case you
> are interested.

I'm sure there is a use case, I just don't think its the most common one.
The major middleware vendors don't implement this as far as I know.


>
>
> Martin
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20090820/328b4441/attachment.htm>