[zeromq-dev] Thinking out loud ...

Henry Baragar Henry.Baragar at instantiated.ca
Tue Jun 28 16:50:52 CEST 2011


Andrew,


On June 24, 2011 08:41:13 PM Andrew Hume wrote:
> henry,
> 
> 	i'm not sure i followed all your examples, but i detected a general meme
> which others have complained of in the past. i admit to being sensitised
> to this issue because i have worked in this area for over a decade now.
> 

I admit that I may have strayed into job scheduling, and even structured my 
arguments so that they led there, but it was supposed to be only for 
illustrative purposes.  

> 	the meme is that of conflating message routing with job scheduling. at
> least, this is what it seems to me. routing messages (fair share, load
> balanced, or whatever) is fundamentally a different thing than job
> scheduling. routing is all about asymptotic properties and is clearly
> aimed at the large number of smallish things case. thus, the issues of
> buffer-induced latency and starvation don't arise unless you stray from
> this niche.
> 

I think we agree on this definition.  

One of my questions was: why are there so many questions about routing on the 
mailing list and why is more than two-thirds of the Guide devoted to 
discussing how to to implement, in application code, custom routing for common 
patterns?  I was exploring for an answer to this question.

> 	on the other hand, job scheduling (outside the large number of cheap jobs
> niche) is a field of study in itself, but because the greedy heuristic
> works very well in most cases, it is well served by the simple use of
> workers asking a central dispatcher for a few jobs at a time.
> 
> 	i think this meme is quite common; nearly all instances of complaints
> about 0mq's scheduling and buffering, slow joiners and the like, are
> examples of this -- we want 0mq to trivially do job scheduling for us as
> well as all the other stuff. 

Does queue management fall into the realm of messaging or job scheduling?  If 
it falls into the realm of messaging, then shouldn't the queue be end-to-end?  
If it falls into the realm of job scheduling, then should 0mq have any queues 
at all?  

Henry

> maybe pieter should add a section to the
> guide discussing this issue and how generically one might deal with it.
> 
> 		andrew
> 
> On Jun 24, 2011, at 9:11 AM, Henry Baragar wrote:
> > I have followed the 0MQ mailing list for about a year, experimented with 
0MQ and contributed to the 0MQ adaptor for plack.  I like many of the features 
of 0MQ, including asynchronous I/O, multi-language support, fan-out/fan-in 
connections and end-point connection syntax.  But there are a number of things 
that I find frustrating and that hinder my use of 0MQ for more applications, 
including:
> ------------------
> Andrew Hume  (best -> Telework) +1 623-551-2845
> andrew at research.att.com  (Work) +1 973-236-2014
> AT&T Labs - Research; member of USENIX and LOPSA

-- 
Henry Baragar
Instantiated Software
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20110628/40694c2c/attachment.htm>


More information about the zeromq-dev mailing list