[zeromq-dev] When unreliability is desired
lindleyf at gmail.com
lindleyf at gmail.com
Sat Dec 7 06:34:15 CET 2013
I'll take a look at tipc. I've been using the pgm classes as a reference so far. They appear to use v1_encoder while tcp uses v2_encoder.
Sent from my iPhone
> On Dec 6, 2013, at 6:37 PM, Pieter Hintjens <ph at imatix.com> wrote:
>
> This is the best place for discussions. The encoder classes are for
> TCP only. If you do try to port the UDP transport, my advice is to do
> it in many little steps, so we can all get involved. Start with an
> empty class if needed. The tipc classes in libzmq may be a help.
>
>> On Sat, Dec 7, 2013 at 12:15 AM, Lindley French <lindleyf at gmail.com> wrote:
>> Is there a better place for discussion of this effort? I do have a lot of
>> questions.
>>
>> The latest one is about encoder_t etc. This appears to have changed
>> significantly recently, and I'm not entirely sure how to adapt the
>> Crossroads code to match. Is there documentation on its latest design
>> somewhere?
>>
>>
>>> On Thu, Dec 5, 2013 at 4:13 PM, Charles Remes <lists at chuckremes.com> wrote:
>>>
>>> Feel free to "rubber duck" with the list as much as you want. :)
>>>
>>> cr
>>>
>>> On Dec 5, 2013, at 3:08 PM, Lindley French <lindleyf at gmail.com> wrote:
>>>
>>> Disregard the question about timers, it appears this was a change made to
>>> Crossroads IO after the fork. I found the relevant commit:
>>>
>>> https://github.com/crossroads-io/libxs/commit/2df873a435ff139cf9d1b10b666d75e6dc6da442#diff-45c9686e09ed422318a2da658745dedd
>>> I will revert the code to the 0MQ way of doing things for now.
>>>
>>>
>>>> On Thu, Dec 5, 2013 at 3:48 PM, Lindley French <lindleyf at gmail.com> wrote:
>>>>
>>>> I'm trying to make sure I understand how the timer_ids work, since these
>>>> were not in the Crossroads IO code. It appears several classes define
>>>> timer_id constants as anonymous enums. Furthermore, the values of these
>>>> constants don't appear to follow any particular pattern, *except* that
>>>> constants with the same name (same purpose?) are assigned the same value.
>>>> Therefore, I expect I ought to adopt tx_timer_id and rx_timer_id from
>>>> pgm_sender without changing their values for udp_sender etc. Am I
>>>> understanding this correctly?
>>>>
>>>>
>>>> On Thu, Dec 5, 2013 at 3:06 PM, Lindley French <lindleyf at gmail.com>
>>>> wrote:
>>>>>
>>>>> It appears that at some point, activate_in was renamed to restart_input,
>>>>> and likewise with output. I'm assuming there are no functional changes other
>>>>> than the names in the UDP case. It also appears that zap_msg_available() can
>>>>> be ignored for now. Let me know if I'm wrong about these.
>>>>>
>>>>>
>>>>> On Thu, Dec 5, 2013 at 2:01 PM, Lindley French <lindleyf at gmail.com>
>>>>> wrote:
>>>>>>
>>>>>> Okay, I'm working on porting the UDP stuff to the latest ZMQ now. Most
>>>>>> of the changes necessary are purely a matter of namespace, but there are a
>>>>>> few others that seem to reflect design changes in ZMQ since XS was forked.
>>>>>> These are the ones I'm not immediately sure how to solve:
>>>>>>
>>>>>> 1) The XS UDP code references encoder_t and decoder_t, but ZMQ has
>>>>>> v1_encoder_t and v2_encoder_t etc instead.
>>>>>> 2) i_engine appears to have three pure virtual methods that are not
>>>>>> defined in the XS code: restart_input, restart_output, and
>>>>>> zap_msg_available().
>>>>>>
>>>>>> Any suggestions how to resolve these?
>>>>>>
>>>>>>
>>>>>> On Thu, Dec 5, 2013 at 10:52 AM, Lindley French <lindleyf at gmail.com>
>>>>>> wrote:
>>>>>>>
>>>>>>> Thanks, I found it. I don't see any reason given as to why UDP support
>>>>>>> was reverted, though. Are there issues with the code, philosophical
>>>>>>> objections, etc?
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Dec 4, 2013 at 5:16 PM, Ivan Pechorin
>>>>>>> <ivan.pechorin at gmail.com> wrote:
>>>>>>>>
>>>>>>>> UDP support was reverted in libxs just before 1.2.0 release: check
>>>>>>>> the commits history on github around June 13th, 2012.
>>>>>>>>
>>>>>>>> 05.12.2013 6:48 пользователь "Lindley French" <lindleyf at gmail.com>
>>>>>>>> написал:
>>>>>>>>
>>>>>>>>> Can you clarify where in the Crossroads IO library the UDP transport
>>>>>>>>> code lives? I've downloaded the 1.2.0 tarball here:
>>>>>>>>> http://download.crossroads.io/libxs-1.2.0.tar.gz
>>>>>>>>> but so far, I don't see a UDP transport in that code. I also checked
>>>>>>>>> the github version:
>>>>>>>>> https://github.com/crossroads-io/libxs
>>>>>>>>> but I don't see it there either.
>>>>>>>>>
>>>>>>>>> On Wed, Nov 27, 2013 at 11:26 AM, Charles Remes
>>>>>>>>> <lists at chuckremes.com> wrote:
>>>>>>>>>>
>>>>>>>>>> That defunct library (crossroads io) has the code that you want.
>>>>>>>>>> That lib was a fork of zeromq, so moving the UDP transport from that library
>>>>>>>>>> to zeromq should be easy (for varying degrees of easy). Once it makes it
>>>>>>>>>> into zeromq, it will be supported.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Nov 27, 2013, at 9:50 AM, Lindley French <lindleyf at gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> I've never used ZeroMQ before so writing up a new transport would
>>>>>>>>>> be just a bit ambitious right now. (I did write something in Java last year
>>>>>>>>>> that, in retrospect, was solving basically the same problem as ZeroMQ so I
>>>>>>>>>> have some familiarity with the problem space.)
>>>>>>>>>>
>>>>>>>>>> I'm also leery of adopting a defunct library for a new project.
>>>>>>>>>>
>>>>>>>>>> I'll keep the udp transport option in mind.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tue, Nov 26, 2013 at 1:18 PM, Pieter Hintjens <ph at imatix.com>
>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Hi Lindley,
>>>>>>>>>>>
>>>>>>>>>>> The right solution would be to make a UDP transport for ZeroMQ.
>>>>>>>>>>> It's
>>>>>>>>>>> not a trivial project but could start with, for instance, just
>>>>>>>>>>> pub/sub
>>>>>>>>>>> (like PGM).
>>>>>>>>>>>
>>>>>>>>>>> It might be worth looking at Crossroads.io for that, which is
>>>>>>>>>>> abandoned but had afair a UDP transport, and shared the same
>>>>>>>>>>> original
>>>>>>>>>>> codebase with ZeroMQ.
>>>>>>>>>>>
>>>>>>>>>>> -Pieter
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Nov 26, 2013 at 6:15 PM, Lindley French
>>>>>>>>>>> <lindleyf at gmail.com> wrote:
>>>>>>>>>>>> I have a networking application that I'd like to use ZeroMQ in.
>>>>>>>>>>>> However, my
>>>>>>>>>>>> use-case demands minimum latency even at the expense of lost
>>>>>>>>>>>> messages. I'm
>>>>>>>>>>>> weary of using ZeroMQ's TCP transport because if packets are
>>>>>>>>>>>> dropped, TCP
>>>>>>>>>>>> will block further messages until it has retransmitted the last
>>>>>>>>>>>> one, and I
>>>>>>>>>>>> don't want that behavior.
>>>>>>>>>>>>
>>>>>>>>>>>> I don't mind FEC codes or other strategies to improve
>>>>>>>>>>>> reliability by sending
>>>>>>>>>>>> more data up-front, but I do not need complete reliability and I
>>>>>>>>>>>> want to
>>>>>>>>>>>> avoid retransmission of messages, or at the least avoid blocking
>>>>>>>>>>>> later
>>>>>>>>>>>> messages if earlier ones need to be retransmitted.
>>>>>>>>>>>>
>>>>>>>>>>>> Is there an existing ZeroMQ transport that will provide the
>>>>>>>>>>>> behavior I want?
>>>>>>>>>>>> I was thinking maybe epgm would do the trick, even though I
>>>>>>>>>>>> don't really
>>>>>>>>>>>> need multicast. Ideally, I'd want a transport that uses pure UDP
>>>>>>>>>>>> for
>>>>>>>>>>>> messages, perhaps with some TCP "behind the scenes" for
>>>>>>>>>>>> out-of-band
>>>>>>>>>>>> handshaking.
>>>>>>>>>>>>
>>>>>>>>>>>> I may end up just using UDP myself for the time-critical
>>>>>>>>>>>> messages, and
>>>>>>>>>>>> ZeroMQ for less critical data, but I'd prefer to avoid
>>>>>>>>>>>> multiple-API creep.
>>>>>>>>>>>>
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> zeromq-dev mailing list
>>>>>>>>>>>> zeromq-dev at lists.zeromq.org
>>>>>>>>>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> zeromq-dev mailing list
>>>>>>>>>>> zeromq-dev at lists.zeromq.org
>>>>>>>>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> zeromq-dev mailing list
>>>>>>>>>> zeromq-dev at lists.zeromq.org
>>>>>>>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> zeromq-dev mailing list
>>>>>>>>>> zeromq-dev at lists.zeromq.org
>>>>>>>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> zeromq-dev mailing list
>>>>>>>>> zeromq-dev at lists.zeromq.org
>>>>>>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> zeromq-dev mailing list
>>>>>>>> zeromq-dev at lists.zeromq.org
>>>>>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>
>>> _______________________________________________
>>> zeromq-dev mailing list
>>> zeromq-dev at lists.zeromq.org
>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>
>>>
>>>
>>> _______________________________________________
>>> zeromq-dev mailing list
>>> zeromq-dev at lists.zeromq.org
>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>>
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
More information about the zeromq-dev
mailing list