[zeromq-dev] zproxy and curve
Pieter Hintjens
ph at imatix.com
Mon Apr 27 10:50:50 CEST 2015
This is very kind of you. I've posted the code here:
https://gist.github.com/anonymous/6fb336d4cc7e5cf76eed
Hope that is OK.
-Pieter
On Mon, Apr 27, 2015 at 10:25 AM, Alan Ward <alan.ward at btinternet.com> wrote:
> Hi Pieter,
> Sorry for the delay, but I work only part-time on this.
>
> My overall architecture is based on the Asynchronous Client/Server pattern from the Guide.
>
> I have N (configurable) backend worker processes to provide solution scaling. They are processes rather than threads as they use postgres.
>
> Until recently I was using zmq_proxy to handle the distribution of work to these workers and the retrieval of their replies. I found I had to deal with the curve security before using the proxy, but that is OK in my case.
>
> When I moved to zsock, etc; zmq_proxy stopped working (as discussed previously) and I found there was no way of using zproxy with curve.
>
> I was already using a zpoller solution in my client, so I transferred that logic to the server to handle the workers. This also enabled me to clean up my close-down as this code was able to action the STOP message.
>
> As requested I have included the code I use to handle all this. [I have simplified it slightly and not tested it]
>
> -----------------------------------------------------------------------------------------------------------------------------------
>
> {
> char worker_endpoint [128], worker_endpoints [1024];
> int process_instance;
>
> worker_endpoints [0] = 0;
>
> // start async worker processes
> for ( process_instance = 0; process_instance < CFG_Backend_Threads; process_instance++ ) {
> pid_t pid = fork();
>
> if ( pid == 0 ) { /* now in the forked process */
> server_worker ( process_instance );
> _exit (0);
> }
> else
> // build string for worker IPC connections
> sprintf ( worker_endpoint, "ipc:///tmp/async%d.ipc", process_instance );
> strcat ( worker_endpoints, worker_endpoint );
> if ( process_instance < CFG_Backend_Threads - 1 ) strcat ( worker_endpoints, "," );
> }
>
> // Connect frontend socket that talks to clients
> zsock_t *frontend = zsock_new ( ZMQ_ROUTER );
> assert ( frontend );
>
> // set up 2-way curve security
> secure_connection ( frontend );
>
> int rc = zsock_bind (frontend, "tcp://*:%s", CFG_Server_Async_Port);
> assert ( rc != -1 );
>
> // connect socket for async backend workers
> zsock_t *backend = zsock_new ( ZMQ_DEALER );
> assert ( backend );
> rc = zsock_attach ( backend, worker_endpoints, 1 );
> assert ( rc != -1 );
>
> // Set up poller to handle 2-way traffic
> zpoller_t *poller = zpoller_new ( frontend,
> backend,
> NULL );
> assert (poller);
>
> while (1) {
>
> void *which = zpoller_wait (poller, -1 );
>
> if ( which == frontend ) {
>
> // get incoming client message
> zmsg_t *msg = zmsg_recv ( frontend );
> assert (msg);
>
> // test for STOP command
> zframe_t *content = zmsg_last (msg);
> assert (content);
> char *message = zframe_data (content);
> if (streq (message, "STOP") ) {
>
> // send STOP message to each worker, before exiting
> for ( process_instance = 0; process_instance < CFG_Backend_Threads; process_instance++ ) {
>
> // need to copy message as it is destroyed by send
> zmsg_t *msg_copy = zmsg_dup ( msg );
> rc = zmsg_send ( &msg_copy, backend );
> }
>
> zmsg_destroy ( &msg );
>
> // wait for workers to get message before destroying socket
> sleep (1);
> zsock_destroy ( &frontend );
> zsock_destroy ( &backend );
> return;
> }
>
> // not STOP, send message to worker process
> rc = zmsg_send ( &msg, backend );
> assert ( rc != -1 );
> }
>
> else if ( which == backend ) {
>
> // get reply mesage from backend process and send to client
> zmsg_t *msg = zmsg_recv ( backend );
> assert (msg);
> rc = zmsg_send ( &msg, frontend );
> assert ( rc != -1 );
> }
>
> else if ( zpoller_terminated ) break;
> }
> }
>
> ----------------------------------------------------------------------------------------------------------
>
> Regards,
> Alan
>
> -----Original Message-----
> From: zeromq-dev-bounces at lists.zeromq.org [mailto:zeromq-dev-bounces at lists.zeromq.org] On Behalf Of Pieter Hintjens
> Sent: 22 April 2015 14:36
> To: ZeroMQ development list
> Subject: Re: [zeromq-dev] zproxy and curve
>
> Nice. if you want to publish that piece of code, others may enjoy seeing how you did it.
>
> On Wed, Apr 22, 2015 at 12:43 PM, Alan Ward <alan.ward at btinternet.com> wrote:
>> Hi Pieter,
>>
>> Thanks for the replies.
>> In the end I cooked my own proxy using a zpoller loop.
>> This also allows me to handle close down more cleanly.
>>
>> Thanks,
>> Alan
>>
>> -----Original Message-----
>> From: zeromq-dev-bounces at lists.zeromq.org
>> [mailto:zeromq-dev-bounces at lists.zeromq.org] On Behalf Of Pieter
>> Hintjens
>> Sent: 20 April 2015 12:25
>> To: ZeroMQ development list
>> Subject: Re: [zeromq-dev] zproxy and curve
>>
>> Maybe I should wait another 5 minutes so you can answer the question
>> :-)
>>
>> Indeed, there is no support for configuring CURVE security on the zproxy sockets.
>>
>> Two options. A is to add this, which is a fun exercise for someone who wants to explore how CZMQ actors work. B, use zmq_proxy as you were before, using zsock_resolve () to get the ZMQ socket handle.
>>
>>
>>
>>
>>
>> On Mon, Apr 20, 2015 at 11:54 AM, Alan Ward <alan.ward at btinternet.com> wrote:
>>>
>>> Hi again,
>>>
>>>
>>>
>>> OK. Having delved a bit deeper it looks like I should be using zproxy rather than zmq_proxy.
>>>
>>>
>>>
>>> I now have a problem in that I can’t determine how to connect my frontend socket with CURVE security as the socket I wish to secure is within zproxy.
>>>
>>>
>>>
>>> Thanks
>>>
>>> Alan
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> From: zeromq-dev-bounces at lists.zeromq.org
>>> [mailto:zeromq-dev-bounces at lists.zeromq.org] On Behalf Of Alan Ward
>>> Sent: 20 April 2015 09:12
>>> To: zeromq-dev at lists.zeromq.org
>>> Subject: [zeromq-dev] zmq_proxy problem
>>>
>>>
>>>
>>> Hi,
>>>
>>>
>>>
>>> I have a system that has been using zmq_proxy for a while to distribute work across a number of worker processes..
>>>
>>>
>>>
>>> It was using zsocket etc.
>>>
>>>
>>>
>>> I am in the process of upgrading the system to use zsock, zactor etc and all is going well – except that zmq-proxy is no longer working.
>>>
>>>
>>>
>>> Apart from the change to zsock the code has not changed and zmq_proxy is just falling through with no error message.
>>>
>>>
>>>
>>> The frontend is a normal tcp socket from an external client; the backed is a set of 3 ipc sockets.
>>>
>>>
>>>
>>> Any ideas?
>>>
>>>
>>>
>>> Thanks,
>>>
>>> Alan
>>>
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> zeromq-dev mailing list
>>> zeromq-dev at lists.zeromq.org
>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
More information about the zeromq-dev
mailing list