[zeromq-dev] Inter thread communication for scalability

[Charles Remes]

Let me restate that final point. The producer socket will not close until all messages have been SENT from that socket which means they are enqueued elsewhere. For inproc, there is no OS buffer or other structure that may disappear and cause a message drop from the source socket closing, so your consumer will receive those messages.

Perhaps someone more familiar with the internals of inproc transport can explain the strength of that guarantee.

On Jan 15, 2014, at 10:01 AM, Charles Remes <lists at chuckremes.com> wrote:

> You need to provide some kind of synchronization mechanism between the producer and the consumer. An explicit ACK for each message or an ACK for a bucket of messages is one way. Another is to enforce a particular shutdown sequence so that the producer always shuts down first. With an infinite LINGER (-1), the socket will not close until the consumer has read all messages on queue.
> 
> 
> On Jan 15, 2014, at 9:49 AM, Kenneth Adam Miller <kennethadammiller at gmail.com> wrote:
> 
>> 
>> On Jan 15, 2014 9:43 AM, "Lindley French" <lindleyf at gmail.com> wrote:
>> >
>> > The thing is, it should be possible to make inproc sockets give you normal C++ object semantics in certain conditions using placement new and placement delete. However, any such effort is hindered by the fact that libzmq's interface is in C. Even if though you're working in C++, and libzmq is implemented in C++, that interface makes it challenging to figure out how such an upgrade could be implemented.
>> >
>> >
>> > On Wed, Jan 15, 2014 at 10:33 AM, KIU Shueng Chuan <nixchuan at gmail.com> wrote:
>> >>
>> >> During socket shutdown, with linger set to 0, messages in-flight would be dropped?
>> >>
>> >> I use synchronized queues to hold the buffer pointers like the OP but use zeromq to send a signal to the consumer thread to pop a buffer from the queue.
>> >>
>> >> On 15 Jan 2014 22:06, "Goswin von Brederlow" <goswin-v-b at web.de> wrote:
>> >>>
>> >>> On Tue, Jan 14, 2014 at 03:40:17PM -0500, Lindley French wrote:
>> >>> > I'm going to caution you about passing pointers through inproc. It may be
>> >>> > possible to do safely, but I haven't yet figured out how to manage
>> >>> > ownership semantics in an environment where messages (pointers) can be
>> >>> > silently dropped.
>> >>>
>> >>> Does a PUSH/PULL inproc socket ever drop messages?
>> >>>
>> 
>> Well I absolutely do not want any messages between threads getting dropped. How can I configure zmq to not drop any in this context?
>> 
> 
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20140115/6b0f9ca7/attachment.htm>