[zeromq-dev] [Openpgm-dev] latency increasing

[Martin Sustrik]

Steven,

Steven McCoy wrote:
> 2009/2/4 Pavol Malosek <malosek at fastmq.com>:
>> I have been measuring latency for different message sizes with following
>> results:
>> first column is message size
>>
>> libpgm
>> 4B: 72.2us
>> 256B: 259us
>> 1400B: 1330us
>> 
>> libpgmplus
>> 4B: 71.2us
>> 256B: 259us
>> 1400B: 1330us
>>
> 
> Oh dear, I can't read.  Your timing results do not indicate any
> variation between the faster assembler checksum routines and the C
> based ones which is rather unusual.

My impression is that there's something so slow involved here that the 
fact whether checksum computation is done in C or assembly becomes 
irrelevant.

We're speaking of 1.3 ms to send 1400B of data... Which limits the 
throughput of the whole thing to 8.6 Mb/s :(

> Modifying the test_partial_csum to add timing I only see up to 2us
> time for checksum calculation and up to 2us for kernel sending
> (debug):
> 
> TSDU size, min cksum time, avg cksum time, max cksum time, min send
> time, avg send time, max send time
> 4,0,0,1,1,1,29
> 256,0,0,4,2,3,29
> 1400,2,2,3,2,2,29
> 
> Or in ticks on a 3.2Ghz box:
> 
> 4,392,410,752,4256,6099,67272
> 256,1728,1810,2136,4840,6271,309760
> 1400,7080,7978,25152,8496,9038,309760

Yes, avg 2us for cksum + avg 2us for send = 2.8 Gb/s of overall 
throughput which is pretty good.

Maybe the slowdown is not related to checksums or sending. I've noticed 
there's a mutex synchronising access to the tx window. Maybe the sender 
thread has to wait for extended periods to get access to the critical 
section?

> So how are you calculating the times?

We're releasing new version of 0MQ today (with OpenPGM support built 
in). Once that's ready Pavol will have a look at the issue.

Martin