[zeromq-dev] service discovery - problem description (was: Re: Poll on sockets OR child process)
Chris Laws
clawsicus at gmail.com
Thu Dec 26 23:24:32 CET 2013
I posted this info to the list a while ago and it seems appropriate to
mention it again now. I'll need to confirm that I can release the code to
show you the internals but here is the basic outline...
I implemented a service discovery mechanism a while ago that might be
useful as a model for this zservice activity. This method lets my system
applications find and make available any number of services. It is
influenced by zbeacon and czmq in general. While I chose to serialise the
UDP packet content using msgpack it could equally be simple strings.
Here is a brief overview of the discovery class module:
// Create a discovery service on a certain UDP port
sdisco_t * sdisco_new (char *peer_id);
// Request the discovery service to resolve the endpoints for the named
service.
void sdisco_lookup (sdisco_t *self, const char* service);
// Remove a service name from the list of unresolved services that are
// in the process of being resolved.
void sdisco_remove_lookup (sdisco_t *self, const char* service);
// Make a service name discoverable. Add an endpoint to the list of
endpoints
// associated with this named service. Services can bind multiple endpoints
// (Eg. ipc:// & tcp://)
void sdisco_add_service (sdisco_t *self, const char* service, const char
*endpoint);
// Remove the named service and its associated list of endpoints from
// the discovery mechanism. The service will no longer be discoverable.
void sdisco_remove_service (sdisco_t *self, const char* service);
// Get discovery socket, for polling or receiving messages
void * sdisco_socket (sdisco_t *self);
Applications wanting to make use of the discovery mechanism must first
create a discovery object using 'sdisco_new ("app_name")'. The application
would typically add the sdisco socket into the zloop so it's messages can
be processed.
The discovery mechanism binds a UDP socket to a port and listens for
discovery packets, similar to zbeacon.
Each discovery packets contains
version_id:
- type: str
- desc: Discovery packet version identification token
- example: 'DISCO01'
peer_id:
- type: str
- desc: Application name
- example 'app_1'
config_id:
- type: int
- desc: A system configuration identification number (defaults to user
id)
- example: getuid()
requests:
- type: str
- desc: a list of service lookups being requested by this application
- example: ['service_2', 'log_service']
services:
- type: {str: [str]}:
- desc: a hashmap of details for services made available from this
application
- example: {'service_1': ['ipc://service_1.ipc', 'tcp://192.168.1.1:5555
']}
The applications own packets are ignored. Only packets that match the
version_id and the config_id continue for further processing. The config_id
field provides the ability to run multiple system configurations ('dev'
'production', etc) on the same compute resources without interfering with
each other - and without needing to change the discovery port. The
config_id defaults to the user id returned from getuid() but can be
overridden by setting an environment variable. This capability was a key
requirement for my particular operating environment but may not be needed
in a zservice. Another approach could be to change the port for different
system configurations.
Applications that provide a service would bind a socket and add the bound
endpoint along with a service name to the discovery mechanism using the
'sdisco_add_service' call. I use a function that binds the socket and then
gets the LAST_ENDPOINT socket option to obtain the endpoint to pass to the
add_service funciton. The discovery mechanism will emit this information
after a brief interval (to minimise unnecessary emits in case the app is
registering lots of services right now). After the service is broadcast
once it is not broadcasted again until another application explicitly
requests it. This minimises unnecessary discovery packets on the network.
Applications that need the endpoints of a particular service ask the sdisco
object to resolve service name endpoints using the 'sdisco_lookup'
call. When emitting lookup requests, the discovery mechanism does not
populate the services dict.
The discovery mechanism emits lookup packets at a configurable interval
(typically 5 seconds) for as long as it has unresolved lookups. When all
unresolved lookups have been resolved then no more discovery packets are
emitted.
When the discovery mechanism resolves a lookup it returns the service name
and endpoint list on its socket (which was earlier placed into the
zloop/poll set) and removes the lookup from its list of unresolved lookups.
So far the method seems to work well for my system of C and Python
applications. It lets me create applications with numerous discoverable
services (eg. control, diagnostics, logging, etc) and it only emits
discovery messages when necessary.
Hopefully this was of interest and informative.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20131227/0b100786/attachment.htm>
More information about the zeromq-dev
mailing list