[zeromq-dev] Using zeromq in a distributed load pipeline, can I get the address of the last server a message was sent to?

Pieter Hintjens ph at imatix.com
Sat Oct 2 04:56:52 CEST 2010


John,

Read the user guide, try the examples, and a lot of this will become clearer.

-Pieter

On Sat, Oct 2, 2010 at 4:47 AM, John Connor
<john.theman.connor at gmail.com> wrote:
> Pieter,
> Thank you for the response.  Your explanation is exactly what I
> needed, I can see that I am going to have to hit the docs pretty hard
> to get the hang of this :)
>
> --Connor
>
>
> On Fri, Oct 1, 2010 at 9:35 PM, Pieter Hintjens <ph at imatix.com> wrote:
>> John,
>>
>> There are several aspects here.
>>
>> - The tcp:// transport is disconnected.  You can send, and then later
>> a recipient can connect, and it will get the message.
>> - The send() function itself runs in the background and thus the
>> actual TCP send can happen much later even if the recipients are
>> already connected.  So you'd need some other API to get back the
>> sender address and some way to tie this to specific send() operations.
>> - If you're doing several million per second, that would become hugely
>> expensive to store and process.
>> - With some transports (multicast) there are no receivers.
>> - 0MQ hides the connected nodes unless you are doing explicit routing
>> with the XREP socket.  This is both for reasons of simplicity and
>> scalability.  E.g. a publisher that has to manage or track all its
>> subscribers is much less scalable than one that does not.
>>
>> -Pieter
>>
>> On Sat, Oct 2, 2010 at 4:13 AM, John Connor
>> <john.theman.connor at gmail.com> wrote:
>>> Thanks for the reply Joshua,
>>> But I still don't understand why the send function can't return the
>>> address that it sent the message to "because it is async".  Why should
>>> that have any effect on whether or not it exposes the address of the
>>> endpoint it sent to?  Once connected, is the address no longer stored
>>> in a usable form?
>>> I don't want to have to use req/rep just so that the receiver can tell
>>> the sender that it got the message and what its address is, that seems
>>> like way over kill.  TCP ensures that the receiver will get the
>>> message, and the sender already knows the address its sending to, so
>>> the whole req/rep thing seems redundant.  Its looking like the only
>>> solution is for me to maintain a list of separate sockets and iterate
>>> over them when sending messages so that I know what server I'm sending
>>> the message to.  Do you know if there would be a lot of overhead with
>>> creating multiple sockets instead of multiple connections?
>>> I appreciate your help,
>>> --Connor
>>>
>>> On Fri, Oct 1, 2010 at 8:26 PM,  <zeromq-dev-request at lists.zeromq.org> wrote:
>>>> No, ZeroMQ doesn't expose where it sends the data because it is asynchronous. If you need this information, you can possibly do it with a request/reply. The request being the msg sent to the server, the reply being the information from that specific server.
>>>>
>>>> You should also change from DOWNSTREAM/UPSTREAM to PUSH/PULL because the downstream/upstream naming is deprecated.
>>>>
>>>> Joshua
>>>>
>>>> On Oct 1, 2010, at 5:41 PM, John Connor wrote:
>>>>
>>>>> Hello,
>>>>> Sorry if this question is a dupe, but I'm just starting to use zeromq
>>>>> for a project, and I have looked everywhere for an answer to this
>>>>> question:
>>>>>
>>>>> If I set up a pipeline which distributes load across a cluster, I
>>>>> would like to log on the sender where its messages get sent. This is
>>>>> what I have in mind (python):
>>>>>
>>>>> import zmq
>>>>> context = zmq.Context()
>>>>> socket = context.socket(zmq.DOWNSTREAM)
>>>>> socket.connect("tcp://127.0.0.1:5000")
>>>>> socket.connect("tcp://127.0.0.1:6000")
>>>>>
>>>>> msg = "Hello World\0"
>>>>> connection_string = socket.send(msg)
>>>>> # should print "Sent message to tcp://127.0.0.1:5000"
>>>>> print "Sent message to", connection_string
>>>>>
>>>>> But I cant find anything that talks about this. Any help at all is appreciated.
>>>>>
>>>>> Thanks,
>>>>> --Connor
>>>>> _______________________________________________
>>>>> zeromq-dev mailing list
>>>>> zeromq-dev at lists.zeromq.org
>>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>> _______________________________________________
>>> zeromq-dev mailing list
>>> zeromq-dev at lists.zeromq.org
>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>
>>>
>>
>>
>>
>> --
>> -
>> Pieter Hintjens
>> iMatix - www.imatix.com
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>



-- 
-
Pieter Hintjens
iMatix - www.imatix.com



More information about the zeromq-dev mailing list