[zeromq-dev] 回复： pyzmq issue: _pickle.UnpicklingError: pickle data was truncated

[CZ]

The msg I am sending is actually a dict structure including some info. Something like,
arg = {
  'account': '12345',
  'asset': 'IO2312-C-3100.CEEX',
  'direction': 'LONG',
   'price': 10,
   'tick': 1
}
What I receive would be the same data structure including the trading result.


Yes, I am sure the client side and server side is running the same version of the project.


Yes, I agree with you it is not a good idea to use pickle with network data. But it is not my choice but zmq's choice. The pickle load and dump happen within zmq/sugar/socket.py.


CZ.
 


------------------ 原始邮件 ------------------
发件人:                                                                                                                        "ZeroMQ development list"                                                                                    <brederlo at q-leap.de>;
发送时间: 2023年10月12日(星期四) 晚上10:10
收件人: "zeromq-dev"<zeromq-dev at lists.zeromq.org>;

主题: Re: [zeromq-dev] pyzmq issue: _pickle.UnpicklingError: pickle data was truncated



Hi,

On 10/12/23 16:05, CZ wrote:
> I am running some python project based on client/server framework. It 
> is something like rpclient/rpcserver. pyzmq is a very important part 
> of this project. Now we are running into some trouble because of some 
> errors in pyzmq. Please refer to the following error msg:
> Exception in thread Thread-1:
> Traceback (most recent call last):
>   File 
> "C:\Users\DELL\AppData\Local\Programs\Python\Python37\lib\threading.py", 
> line 926, in _bootstrap_inner
>     self.run()
>   File 
> "C:\Users\DELL\AppData\Local\Programs\Python\Python37\lib\threading.py", 
> line 870, in run
>     self._target(*self._args, **self._kwargs)
>   File "D:\DellProjs\oly\Oly\olympos\q_core\rpc\__init__.py", line 
> 372, in run
>     topic, data = self.__socket_sub.recv_pyobj(flags=NOBLOCK)
>   File "C:\Dev\Py37venv\lib\site-packages\zmq\sugar\socket.py", line 
> 976, in recv_pyobj
>     return self._deserialize(msg, pickle.loads)
>   File "C:\Dev\Py37venv\lib\site-packages\zmq\sugar\socket.py", line 
> 834, in _deserialize
>     return load(recvd)
> _pickle.UnpicklingError: pickle data was truncated
>
> I am using Python3.7.9 (actually I tried python3.7.9 and. python3.11, 
> the same error msg) in windows 11, and pyzmq 25.1.1. The msg I am 
> sending is really small. No way it would blow the buffer. I am using 
> PUB at server side and SUB at client side (I always saw XPUB and XSUB 
> in the zmq.constants. Could they possibly be another choices). I saw 
> flags have 3 different values (corret me if I was wrong), NOWAIT, 
> NOBLOCK, SNDMORE. I am not sure if the choice of flags value could be 
> the reason.
>
> Any of your input would be highly appreciated.  Thanks!
>
> CZ.


You should print the string you are sending and the string you received. 
You also need to make sure the client and server use the same version of 
the project.


That said: You should never ever use pickle with network data. That's an 
instand remote code execution exploit as the string you unpickle can 
contain arbitrary code that you will just execute.


MfG

Goswin

_______________________________________________
zeromq-dev mailing list
zeromq-dev at lists.zeromq.org
https://lists.zeromq.org/mailman/listinfo/zeromq-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20231012/7b452b90/attachment.htm>