[zeromq-dev] How should test cases be written?

[Pieter Hintjens]

We don't really have formal rules for test cases. It could be fun to
document what we do in practice.

Technically a test must fail or pass, it may not hang.

Here are the guidelines I use for my test code:

- should be written to be reusable by developers looking for examples
- should thus be in C, more than C++
- should test every single API functionality that is documented
- should be tuned to not do long waits or pauses
- should use 127.0.0.1 as interface for all tests

-Pieter

On Sat, Jan 18, 2014 at 7:19 AM, Goswin von Brederlow <goswin-v-b at web.de> wrote:
> Hi,
>
> having written a patch for connection notification on STREAM sockets I
> wonder about the test case for stream sockets.
>
> Currently there are 3 tests with STREAM sockets (tested in that order):
>
> 1) test_stream: (uses recv/send)
>    + test connecting a STREAM and DEALER socket
>    + test connecting a STREAM and STREAM socket
>
>    Tests connection notification, sending messages, closing sockets
>    and disconnect notification all in one.
>
> 2) test_stream_disconnect_notifications:  (uses recv/send_msg)
>    + test connecting a STREAM and STREAM socket with polling
>
> 3) test_stream_empty: (uses recv/send_msg)
>    connecting a STREAM and DEALER socket
>    + testing zmq_msg_close on an empty message?
>
> As it is test_stream will fail if anything is wrong with STREAM
> sockets regardless of what it is.
>
> I wonder: Is the line number of the failure enough of a clue what is
> wrong? Or should there be more tests that test things step by step?
>
> Esspecially test_stream contains 2 completly seperate tests:
> STREAM/DEALER and STREAM/STREAM. Shouldn't that at least be split?
>
>
> Secondly I wonder if the tests should care about blocking. They don't
> set a timeout on the sockets, don't use non-blocking calls and most
> don't set LINGER to 0 before closing sockets and destroying the
> context. So if anything goes wrong, like the connect notification not
> being send, the test simply hangs. It doesn't fail. Shouldn't they?
>
>
> So what is the rule for this in the zmq project?
>
> MfG
>         Goswin
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev at lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev