[zeromq-dev] PUB/SUB appears to be 'missing' first byte 3.2.2

Wed May 22 03:47:56 CEST 2013

Hi Trevor, Siam

Thanks for the responses. Rebuilt jzmq on the device and the problem disappeared.

Siam - I think you're right, the zmq.jar I was distributing with the code was not fully compatible with the jni layer/zmq installed on the target machine. Ah jni...

Thanks again,
Mark

________________________________
 From: Siam Rafiee <siamraf at gmail.com>
To: ZeroMQ development list <zeromq-dev at lists.zeromq.org> 
Sent: Tuesday, 21 May 2013, 16:48
Subject: Re: [zeromq-dev] PUB/SUB appears to be 'missing' first byte 3.2.2

Mark,

I've experienced something similar to you before - and in my case it came down to the version of the jzmq jar I was using on the machines. I found a jar built on one machine wouldn't necessarily work on another machine, especially if there is a difference in the jzmq/zmq version on each machine (as there is in your case).

Are your processes using a jzmq jar built specifically on the machine they're running on? Or are they using a jar retrieved from some common repository manager (e.g. nexus)?

Siam

On 20 May 2013 17:54, Trevor Bernard <trevor.bernard at gmail.com> wrote:

What version of JZMQ are you using?
>
>I ran your test program and received the correct results.
>
>JZMQ - 018a38d414af4e689c93e29d4d7a59a1f51fe84e
>libzmq 3.2.2
>
>
>On Mon, May 20, 2013 at 1:45 AM, Mark Sutheran <mark_sutheran at yahoo.com> wrote:
>> Hi Pieter,
>>
>> Thanks for the quick response - test case below.
>>
>> Tested it using ipc and tcp transports - no difference. The trigger appears
>> to be the flag used in the send() method. Setting this to 1 (ZMQ.NOBLOCK)
>> causes the unexpected behaviour on the second box.
>>
>> Logs:
>>
>> Machine 1 (Ubuntu 12.04 64, JZMQ/ZMQ 3.2.0)
>>
>> Blocking pub: [7, 0, 0, 13, -49, 0, 0]
>> ...received data: [7, 0, 0, 13, -49, 0, 0]
>> Non-blocking pub: [7, 0, 0, 13, -49, 0, 0]
>> ...received data: [7, 0, 0, 13, -49, 0, 0]
>>
>> Machine 2 (Ubuntu 12.04 64, JZMQ/ZMQ 3.2.2)
>>
>> Blocking pub: [7, 0, 0, 13, -49, 0, 0]
>> ...received data: [7, 0, 0, 13, -49, 0, 0]
>> Non-blocking pub: [7, 0, 0, 13, -49, 0, 0]
>> ...received data: [0, 0, 13, -49, 0, 0]
>>
>> Regards,
>> Mark
>>
>> Jzmq test case:
>>
>> /*************************************************************************************************************************/
>>
>> package zmqtest;
>>
>> import java.util.Arrays;
>>
>> import org.zeromq.ZMQ;
>>
>> public class ZMQTest {
>>
>> private static final ZMQ.Context context = ZMQ.context(1);
>> private static final ZMQ.Socket subscriber = context.socket(ZMQ.SUB);
>> private static final String tcpPort = "9999";
>>
>> private static final byte[] TEST_DATA = new byte[]{7, 0, 0, 13, -49, 0, 0};
>>
>> public static void main(String[] args) {
>>
>> System.out.println("Using ZMQ version: " + ZMQ.getVersionString());
>> subscriber.connect("tcp://localhost:" + tcpPort);
>> subscriber.subscribe(new byte[]{});
>> Thread t = new Thread(new SubscriptionDataHandler());
>> t.setName("SubscriptionDataHandlerThread");
>> t.setDaemon(true);
>> t.start();
>>
>> ZMQ.Socket publisher = context.socket(ZMQ.PUB);
>> publisher.bind("tcp://*:" + tcpPort);
>> try {
>>
>> for (int i = 0; i < 3; i++) {
>> System.out.println("Blocking pub:\t\t" + Arrays.toString(TEST_DATA));
>> publisher.send(TEST_DATA,0);
>> Thread.sleep(1000l);
>> System.out.println("Non-blocking pub:\t" + Arrays.toString(TEST_DATA));
>> publisher.send(TEST_DATA,ZMQ.NOBLOCK);
>> Thread.sleep(1000l);
>> }
>> Thread.sleep(500l);
>>
>> } catch (Exception e) {
>> e.printStackTrace();
>> }
>> }
>> private static final class SubscriptionDataHandler implements Runnable {
>>
>> @Override
>> public void run() {
>> System.out.println("SubscriptionDataHandler thread starting");
>> while (true) {
>> final byte[] rec = subscriber.recv(0);
>> System.out.println("...received data:\t" + Arrays.toString(rec));
>> }
>> }
>> }
>> }
>>
>>
>> ________________________________
>> From: Pieter Hintjens <ph at imatix.com>
>> To: Mark Sutheran <mark_sutheran at yahoo.com>; ZeroMQ development list
>> <zeromq-dev at lists.zeromq.org>
>> Sent: Sunday, 19 May 2013, 17:04
>> Subject: Re: [zeromq-dev] PUB/SUB appears to be 'missing' first byte 3.2.2
>>
>> Hi Mark,
>>
>> Would you provide a minimal test case that demonstrates the problem?
>>
>> Thanks
>> Pieter
>>
>> On Sat, May 18, 2013 at 6:56 PM, Mark Sutheran <mark_sutheran at yahoo.com>
>> wrote:
>>> Hi,
>>>
>>> I appear to have a issue with PUB/SUB apparently 'dropping' the first
>>> byte.
>>>
>>> Setup:
>>>  * Basic PUB/SUB running on same machine over TCP/localhost.
>>>  * Message has first 4 bytes as subscription id, rest payload
>>>
>>> Problem:
>>>  * On dev box runs fine
>>>  * On prod box the subscriber sees no messages
>>>  * The reason is that the received message is 'missing' the first byte
>>>
>>> Logs:
>>>
>>> Machine 1 (Ubuntu 12.04 64, JZMQ/ZMQ 3.2.0) runs fine, e.g.:
>>>
>>> 01:19:10:042 DEBUG [qtp1866572071-24] Publishing: [0, 0, 0, 13, -49, 0, 0,
>>> 1, 62, -72, -90, -29, 63, 7, -108, -53, 63, -12, -122, -13, 110, -8, 5,
>>> 96,
>>> -53, 64, 89, -9, -101, -128, 35, -90, -50, -53, 0, 0, 0, 0, 0, 0, 0, 0,
>>> -53,
>>> 64, 68, 0, 0, 0, 0, 0, 0, 0]
>>> 01:19:10:042 DEBUG [SubscriptionDataHandlerThread] Received data: [0, 0,
>>> 0,
>>> 13, -49, 0, 0, 1, 62, -72, -90, -29, 63, 7, -108, -53, 63, -12, -122, -13,
>>> 110, -8, 5, 96, -53, 64, 89, -9, -101, -128, 35, -90, -50, -53, 0, 0, 0,
>>> 0,
>>> 0, 0, 0, 0, -53, 64, 68, 0, 0, 0, 0, 0, 0, 0]
>>>
>>> Machine 2 (Ubuntu 12.04 64, JZMQ/ZMQ 3.2.2) and the subscriptions
>>> 'disappear'... digging into it the receiver appears to drop the first byte
>>> of the messages 100% of the time, e.g.:
>>>
>>> 05:20:01:603 DEBUG [qtp1038722314-22] Publishing: [0, 0, 0, 13, -49, 0, 0,
>>> 1, 62, -72, -90, 61, 85, 7, -108, -53, 63, -12, -122, -13, 110, -8, 5, 96,
>>> -53, 64, 89, -9, -101, -128, 35, -90, -50, -53, 0, 0, 0, 0, 0, 0, 0, 0,
>>> -53,
>>> 64, 68, 0, 0, 0, 0, 0, 0, 0]
>>> 05:20:01:603 DEBUG [SubscriptionDataHandlerThread] Received data: [0, 0,
>>> 13,
>>> -49, 0, 0, 1, 62, -72, -90, 61, 85, 7, -108, -53, 63, -12, -122, -13, 110,
>>> -8, 5, 96, -53, 64, 89, -9, -101, -128, 35, -90, -50, -53, 0, 0, 0, 0, 0,
>>> 0,
>>> 0, 0, -53, 64, 68, 0, 0, 0, 0, 0, 0, 0]
>>>
>>> Connecting Machine 1 (SUB) to Machine 2 (PUB) also produces the same
>>> results.
>>>
>>> It's presumably an issue with my code however, it's the same code running
>>> on
>>> both boxes. The one obvious difference is the ZMQ version, though I've not
>>> seen this before and it doesn't seem to correspond to anything I can see
>>> on
>>> the buglist. Any ideas?
>>>
>>> Thanks in advance,
>>> Mark
>>>
>>> _______________________________________________
>>> zeromq-dev mailing list
>>> zeromq-dev at lists.zeromq.org
>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>>
>>
>> _______________________________________________
>> zeromq-dev mailing list
>> zeromq-dev at lists.zeromq.org
>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>
>_______________________________________________
>zeromq-dev mailing list
>zeromq-dev at lists.zeromq.org
>http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>

_______________________________________________
zeromq-dev mailing list
zeromq-dev at lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20130521/5cd8601b/attachment.htm>