[zeromq-dev] Testing framework

[Staffan Gimåker]

On Tue, 2012-01-24 at 01:14 +1100, john skaller wrote:
> On 23/01/2012, at 11:56 PM, Ian Barber wrote:
> 
> > On Mon, Jan 23, 2012 at 12:26 PM, Staffan Gimåker <staffan at spotify.com> wrote:
> > Hi,
> > 
> > Are there any plans to migrate libzmq to use a more fully-featured testing framework (e.g. googletest)? The current automake test system is a bit of a pain to work with since it's missing stuff like a nice way to group related tests, a standardized way to do fixtures, sensible messages from failed assertions, etc.
> > 
> > Aside from an added (compile-time) dependency, are there any reasons why the automake way is preferable?
> > 
> > /S
> > 
> > It's proved tricky to unit test, though there have been some attempts - if you check the list archive you'll find a few discussions, Steve-o's is probably the best summary of issues: http://comments.gmane.org/gmane.network.zeromq.devel/11587
> 
>  I am struggling with a closely related problem:
> how to check the Guide examples in Felix actually work. 
> 
> My belief here: for systems like 0MQ units tests are all but worthless.
> The behaviour of the functions depends critically other functions,
> so you can't really test any of them independently. Worse, you cannot
> test communications and asynchronous behaviour with an isolated
> subroutine.

I don't disagree that automated or semi-automated
integration/performance tests would be nice to have, BUT unit tests have
a role to fill as well.

Traditional unit tests work perfectly well for detecting a lot of errors
(the easy ones). For example, it's a good fit to test that filtering
works as it should or that someone didn't accidentally break the REQ-REP
functionality. They're easy to write in comparison, easy to fully
automate and it's very easy to spot where a regression was introduced
("assertion failed, line X, file Y").

/S