[zeromq-dev] ZeroMQ 4.2 release, planning

[Luca Boccassi]

I have sent a solution for the alignment problem that solves the sigbus
problem without breaking ABI compat (plus follow-up for VC++ - sorry
Windows guys https://github.com/zeromq/libzmq/pull/2179 ).

I tested the alignment and sigbus problem on x86_64 by enabling
alignment check with:

__asm__("pushf\norl $0x40000,(%rsp)\npopf");

All was fine.

I ran tests built from the zeromq4-1 repository against a shared lib
from the head of libzmq repo, and they all run fine minus the
ZMQ_REQ_CORRELATE one but that option was borken anyway.

This allows us to do a release now, and then when we are ready we can do
the ABI breakage, without blocking 4.2. Which is nice since it means it
might make it for Debian 9!

So, Doron et al, shall we do the bump this weekend?

On Thu, 2016-10-20 at 17:12 -0500, Thomas Rodgers wrote:
> I will have some time most likely the week of Nov6 (off for a week of C++
> Committee 'fun') to test different message size alternatives. I'll follow
> up with my results here for consideration the next time we are inclined to
> break the ABI compatibility :)
> 
> On Sunday, October 16, 2016, Brian Knox <bknox at digitalocean.com> wrote:
> 
> > A new stable version would definitely help me in my quest to get ZeroMQ
> > support enabled by default in rsyslog in distros.
> >
> > On Sun, Oct 16, 2016 at 2:40 PM Doron Somech <somdoron at gmail.com> wrote:
> >
> >> I say lets bump.
> >>
> >> On Oct 15, 2016 20:32, "Luca Boccassi" <luca.boccassi at gmail.com> wrote:
> >>
> >>> As Thomas said, false sharing would be a real issue with 96.
> >>>
> >>> So given a release is long due, at this point I'd say to drop this for
> >>> the moment.
> >>>
> >>> What do we do for the change to union for zmq_msg_t? Bump ABI version or
> >>> not?
> >>>
> >>> On Thu, 2016-10-06 at 09:53 +0300, Doron Somech wrote:
> >>> > No new socket type, I worked at the time on binary message type, might
> >>> > complete it sometime, but it is not urgent.
> >>> >
> >>> > If there is a lot of performance penalty we can give it up, I will
> >>> > find another solution for the Radio-Dish.
> >>> >
> >>> > What about 96 bytes? same penalty?
> >>> >
> >>> > Regarding the binding, I'm not sure.
> >>> >
> >>> > On Sat, Oct 1, 2016 at 9:14 PM, Luca Boccassi <luca.boccassi at gmail.com>
> >>> wrote:
> >>> > > On Tue, 2016-09-27 at 09:41 +0300, Doron Somech wrote:
> >>> > >> Sorry for the late response, increasing the msg_t structure will be
> >>> > >> great, however this will require changing a lot of binding.
> >>> > >
> >>> > > I think I remember we need it for the new socket types, is that
> >>> correct?
> >>> > >
> >>> > > There is a large performance penalty (intuitively due to not fitting
> >>> > > into a single cache line anymore, but haven't ran perf/cachegrind),
> >>> and
> >>> > > the throughput with vsm type messages goes down by 4% (min) and 20%
> >>> > > (max) for TCP, and 36% (min) 38 (max) for inproc, which is quite a
> >>> lot,
> >>> > > so we need to be sure it's worth it.
> >>> > >
> >>> > > Regarding the bindings, after a quick search on the Github org, I
> >>> could
> >>> > > only see:
> >>> > >
> >>> > > https://github.com/zeromq/lzmq/blob/master/src/lua/lzmq/
> >>> ffi/api.lua#L144
> >>> > > https://github.com/zeromq/clrzmq4/blob/master/lib/zmq.cs#L28
> >>> > > https://github.com/zeromq/pyczmq/blob/master/pyczmq/zmq.py#L177
> >>> > >
> >>> > > Other bindings just import zmq.h. Did I miss any?
> >>> > >
> >>> > >> Sorry for disappearing, baby and full time job is a lot :-),
> >>> hopefully
> >>> > >> I'm back...
> >>> > >
> >>> > > No worries, perfectly understandable :-)
> >>> > >
> >>> > >> On Mon, Aug 29, 2016 at 6:46 PM, Luca Boccassi <
> >>> luca.boccassi at gmail.com> wrote:
> >>> > >> > Sorry, I meant if we go with (1), not (2), we might bump the size
> >>> as
> >>> > >> > well, since we are already doing another ABI-breaking change.
> >>> > >> >
> >>> > >> > I agree on the solution as well.
> >>> > >> >
> >>> > >> > On Mon, 2016-08-29 at 17:12 +0200, Pieter Hintjens wrote:
> >>> > >> >> I'm confused between the (1) and (2) choices, and can't see where
> >>> > >> >> bumping the message size fits.
> >>> > >> >>
> >>> > >> >> Nonetheless, I think bumping the size, fixing the alignment
> >>> issues,
> >>> > >> >> and bumping the ABI version is the best solution here.
> >>> > >> >>
> >>> > >> >> On Fri, Aug 26, 2016 at 12:33 PM, Luca Boccassi <
> >>> luca.boccassi at gmail.com> wrote:
> >>> > >> >> > I've given some more thoughts and testing to the alignment
> >>> issue. I can
> >>> > >> >> > reproduce the problem by enabling alignment checks on x86 too.
> >>> > >> >> >
> >>> > >> >> > But most importantly, I think we cannot get away from bumping
> >>> the ABI
> >>> > >> >> > with this fix, however we rearrange it, simply because
> >>> applications need
> >>> > >> >> > to be rebuilt against the new header to be fixed. A simple
> >>> rebuild of
> >>> > >> >> > the libzmq.so is not enough. And the way to do this is to bump
> >>> the ABI
> >>> > >> >> > so that distros can schedule transitions and rebuilds and so
> >>> on.
> >>> > >> >> >
> >>> > >> >> > So the choice list is now restricted to:
> >>> > >> >> >
> >>> > >> >> > 1) Bump ABI
> >>> > >> >> > 2) Revert the fix and leave everything broken on sparc64 and
> >>> some
> >>> > >> >> > aarch64 (rpi3 seems not to be affected, must depend on the SoC
> >>> flavour)
> >>> > >> >> >
> >>> > >> >> > If we go with 2, we might as well get 2 birds with one stone
> >>> and bump
> >>> > >> >> > the zmq_msg_t size to 128 as we have talked about in the past.
> >>> > >> >> >
> >>> > >> >> > Doron, this would help with the new UDP based socket types
> >>> right?
> >>> > >> >> >
> >>> > >> >> > Pros of bumping msg size:
> >>> > >> >> >
> >>> > >> >> > - we can get rid of the malloc() in the lmsg type case as all
> >>> the data
> >>> > >> >> > will fit
> >>> > >> >> >
> >>> > >> >> > Cons:
> >>> > >> >> >
> >>> > >> >> > - for the vsm/cmsg type cases (for most architectures anyway)
> >>> it won't
> >>> > >> >> > fit anymore into a single cacheline
> >>> > >> >> >
> >>> > >> >> > Given all this, I'd say we should go for it.
> >>> > >> >> >
> >>> > >> >> > Opinions?
> >>> > >> >> >
> >>> > >> >> > On Sat, 2016-08-13 at 16:59 +0100, Luca Boccassi wrote:
> >>> > >> >> >> Hello,
> >>> > >> >> >>
> >>> > >> >> >> Trying to give some thoughts again on the libzmq 4.2 release.
> >>> It's
> >>> > >> >> >> really long overdue!
> >>> > >> >> >>
> >>> > >> >> >> The main issue from my point of view is this change:
> >>> > >> >> >>
> >>> > >> >> >> https://github.com/zeromq/libzmq/commit/
> >>> d9fb1d36ff2008966af538f722a1f4ab158dbf64
> >>> > >> >> >>
> >>> > >> >> >> -typedef struct zmq_msg_t {unsigned char _ [64];} zmq_msg_t;
> >>> > >> >> >>  +/* union here ensures correct alignment on architectures
> >>> that require
> >>> > >> >> >> it, e.g.
> >>> > >> >> >>  + * SPARC
> >>> > >> >> >>  + */
> >>> > >> >> >>  +typedef union zmq_msg_t {unsigned char _ [64]; void *p; }
> >>> zmq_msg_t;
> >>> > >> >> >>
> >>> > >> >> >>
> >>> > >> >> >> This is flagged by the common ABI checkers tools as an ABI
> >>> breakage
> >>> > >> >> >> (see: http://abi-laboratory.pro/tracker/timeline/zeromq/ ).
> >>> And it makes
> >>> > >> >> >> sense from this point of view: if some applications on some
> >>> > >> >> >> architectures are broken due to wrong alignment, they would
> >>> need to be
> >>> > >> >> >> rebuilt, and the way to ensure that is to bump the ABI
> >>> "current" digit
> >>> > >> >> >> to make sure maintainers do a rebuild.
> >>> > >> >> >>
> >>> > >> >> >> On the other hand, signaling an ABI breakage is a pain, and a
> >>> cause of
> >>> > >> >> >> major churn for packagers and maintainers. It means for
> >>> example a new
> >>> > >> >> >> package has to be created (eg: libzmq5 -> libzmq6), and a
> >>> transition has
> >>> > >> >> >> to be started and all reverse dependencies need to be
> >>> rebuilt. And if
> >>> > >> >> >> this is pointless for all save a few corner cases (eg SPARC64
> >>> as for
> >>> > >> >> >> above) it's all quite frustrating.
> >>> > >> >> >>
> >>> > >> >> >> So we have a choice to make before we release 4.2, four
> >>> possibilities as
> >>> > >> >> >> far as I can see:
> >>> > >> >> >>
> >>> > >> >> >> 1) Ignore the ABI checkers and get yelled at by maintainers
> >>> and
> >>> > >> >> >> packagers. Also the SPARC64 users will most likely NOT get
> >>> their bug
> >>> > >> >> >> fixed
> >>> > >> >> >> 2) Bump ABI revision to 6 and get yelled at by maintainers
> >>> and packagers
> >>> > >> >> >> 3) Revert the above change and postpone it to when we have a
> >>> more
> >>> > >> >> >> generally useful reason to break ABI (bump zmq_msg_t from 64
> >>> to 128
> >>> > >> >> >> bytes for example, Doron?)
> >>> > >> >> >> 4) Try to be clever and revert the above change and use
> >>> something like
> >>> > >> >> >> #pragma pack(8). This will fool the ABI checkers (I tried
> >>> it), and given
> >>> > >> >> >> that typedef is only used externally to allocate the right
> >>> size it
> >>> > >> >> >> shouldn't actually affect anything, apart from the users of
> >>> SPARC64
> >>> > >> >> >> which should get the bugfix with this too. This is very
> >>> sneaky :-)
> >>> > >> >> >>
> >>> > >> >> >> CC'ing Lazslo, the Debian maintainer, given what we choose to
> >>> do might
> >>> > >> >> >> result in a lot of work for him :-)
> >>> > >> >> >>
> >>> > >> >> >> Opinions?
> >>> > >> >> >>
> >>> > >> >> >> Kind regards,
> >>> > >> >> >> Luca Boccassi
> >>> > >> >> >>
> >>> > >> >> >> On Tue, 2016-05-03 at 10:39 +0200, Pieter Hintjens wrote:
> >>> > >> >> >> > Hi all,
> >>> > >> >> >> >
> >>> > >> >> >> > I'm just throwing some ideas on the table. We have a good
> >>> package of
> >>> > >> >> >> > work on master and it's probably time to make a 4.2 release.
> >>> > >> >> >> >
> >>> > >> >> >> > Luca has already back-ported the enable/disable draft
> >>> design from
> >>> > >> >> >> > zproject (CZMQ et al). Yay! So we can now release stable
> >>> master
> >>> > >> >> >> > safely, while continuing to refine and extend the draft API
> >>> sections.
> >>> > >> >> >> >
> >>> > >> >> >> > I propose:
> >>> > >> >> >> >
> >>> > >> >> >> > - to end with the stable fork policy; this was needed years
> >>> ago when
> >>> > >> >> >> > we had massively unstable masters. It's no longer a problem.
> >>> > >> >> >> > - to use the github release function for libzmq releases
> >>> and deprecate
> >>> > >> >> >> > the separate delivery of tarballs.
> >>> > >> >> >> > - we aim to make a 4.2.0 rc asap, then fix any issues we
> >>> get, with
> >>> > >> >> >> > patch releases as usual.
> >>> > >> >> >> > - we backport the release function to older maintained
> >>> releases (4.1,
> >>> > >> >> >> > 3.2) so that their tarballs are provided by github instead
> >>> of
> >>> > >> >> >> > downloads.zeromq.org.
> >>> > >> >> >> >
> >>> > >> >> >> > Problems:
> >>> > >> >> >> >
> >>> > >> >> >> > - this will break a few things that depend on
> >>> downloads.zeromq.org. To
> >>> > >> >> >> > be fixed as we go.
> >>> > >> >> >> > - github tarballs are not identical to source tarballs,
> >>> particularly
> >>> > >> >> >> > they lack `configure`. I propose changing our autotools
> >>> build
> >>> > >> >> >> > instructions so they always start with `./autogen,sh` no
> >>> matter where
> >>> > >> >> >> > the sources come from.
> >>> > >> >> >> >
> >>> > >> >> >> > I think this will work and also let us gracefully
> >>> deprecate/switch off
> >>> > >> >> >> > the downloads box.
> >>> > >> >> >> >
> >>> > >> >> >> > -Pieter
> >>> > >> >> >> > _______________________________________________
> >>> > >> >> >> > zeromq-dev mailing list
> >>> > >> >> >> > zeromq-dev at lists.zeromq.org
> >>> > >> >> >> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >>> > >> >> >>
> >>> > >> >> >>
> >>> > >> >> >
> >>> > >> >> >
> >>> > >> >> >
> >>> > >> >> > _______________________________________________
> >>> > >> >> > zeromq-dev mailing list
> >>> > >> >> > zeromq-dev at lists.zeromq.org
> >>> > >> >> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >>> > >> >> _______________________________________________
> >>> > >> >> zeromq-dev mailing list
> >>> > >> >> zeromq-dev at lists.zeromq.org
> >>> > >> >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >>> > >> >
> >>> > >> >
> >>> > >
> >>>
> >>> _______________________________________________
> >> zeromq-dev mailing list
> >> zeromq-dev at lists.zeromq.org
> >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> >
> >
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: This is a digitally signed message part
URL: <https://lists.zeromq.org/pipermail/zeromq-dev/attachments/20161028/85778255/attachment.sig>