Archive for the ‘Unit Testing’ Category

Helper macros for Check

August 6, 2007

During my vacation I have had the opportunity to add unit testing to SCEW (Simple C Expat Wrapper). I looked at various C unit testing frameworks and I decided to use Check. Most of them follow the xUnit approach, but I chose Check because tests run in a separate address space other than the test runner.

I found that writing test cases was a bit hard using Check’s syntax, for example following the manual you can write this integer check:

fail_unless (money_amount (m) == 5,
             "Amount not set correctly on creation");

This is fine if you are reading the code, but if the check fails the output doesn’t show you what the actual or expected values are, so the manual suggests changing it for:

fail_unless(money_amount (m) == 5,
            "Amount was %d, instead of 5", money_amount (m));

which is quite better than the first one, but to painful if you have to write it for every check. So, why not write a helper macro that checks for integers, prints the actual and expected values and also shows you the check that is being done?

#define CHECK_U_INT(A, B, MSG, ...)                                     \\
  do                                                                    \\
    {                                                                   \\
      enum { MAX_BUFFER = 250 };                                        \\
      static char buffer[MAX_BUFFER];                                   \\
      sprintf (buffer, MSG, ##__VA_ARGS__);                             \\
      unsigned int v_a = (A);                                           \\
      unsigned int v_b = (B);                                           \\
      fail_unless (v_a == v_b,                                          \\
                   "(%s) == (%s) \\n  Actual: %d \\n  Expected: %d \\n  %s", \\
                   #A, #B, v_a, v_b, buffer);                           \\
    }                                                                   \\
  while (0)

With this macro you can now write code like this:

CHECK_U_INT (money_amount (m), 5, "Money amount mismatch");

which is really easy to read and in the test’s output you can see the actual and expected values, the performed test and the user message clarifying the intention of the check.

check.c:97:F:Core:test_amount:0: (money_amount (m)) == (5) 
  Actual: 2 
  Expected: 5
  Money amount mismatch

The same happens with strings, so instead of writing this:

fail_if (strcmp (money_currency (m), "USD") != 0,
         "Currency not set correctly on creation");

or this:

if (strcmp (money_currency (m), "USD") != 0)
  {
    fail ("Currency not set correctly on creation");
  }

we can write a string checking macro that shows the actual and expected strings and the check being done:

#define CHECK_STR(A, B, MSG, ...)                                       \\
  do                                                                    \\
    {                                                                   \\
      char const *str_a = (A);                                          \\
      char const *str_b = (B);                                          \\
      if (strcmp (str_a, str_b) != 0)                                   \\
        {                                                               \\
          enum { CHECK_MAX_BUFFER = 250 };                              \\
          static char buffer[CHECK_MAX_BUFFER];                         \\
          sprintf (buffer, MSG, ##__VA_ARGS__);                         \\
          fail ("(%s) == (%s) \\n  Actual: %s \\n  Expected: %s \\n  %s",  \\
                #A, #B, str_a, str_b, buffer);                          \\
        }                                                               \\
    }                                                                   \\
  while (0)

As before, this would be the new output:

check.c:205:F:Core:test_currency:0: (money_currency (m)) == (USD) 
  Actual: EUR
  Expected: USD
  Currency not set correctly on creation

Well, this is not a big deal, but I have found it quite useful. Below, is the list of macros I am using right now:

#define CHECK_U_INT(A, B, MSG, ...)                                     \\
  do                                                                    \\
    {                                                                   \\
      enum { MAX_BUFFER = 250 };                                        \\
      static char buffer[MAX_BUFFER];                                   \\
      sprintf (buffer, MSG, ##__VA_ARGS__);                             \\
      unsigned int v_a = (A);                                           \\
      unsigned int v_b = (B);                                           \\
      fail_unless (v_a == v_b,                                          \\
                   "(%s) == (%s) \\n  Actual: %d \\n  Expected: %d \\n  %s", \\
                   #A, #B, v_a, v_b, buffer);                           \\
    }                                                                   \\
  while (0)

#define CHECK_S_INT(A, B, MSG, ...)                                     \\
  do                                                                    \\
    {                                                                   \\
      enum { MAX_BUFFER = 250 };                                        \\
      static char buffer[MAX_BUFFER];                                   \\
      sprintf (buffer, MSG, ##__VA_ARGS__);                             \\
      int v_a = (A);                                                    \\
      int v_b = (B);                                                    \\
      fail_unless (v_a == v_b,                                          \\
                   "(%s) == (%s) \\n  Actual: %d \\n  Expected: %d \\n  %s", \\
                   #A, #B, v_a, v_b, buffer);                           \\
    }                                                                   \\
  while (0)

#define CHECK_BOOL(A, B, MSG, ...) CHECK_U_INT (A, B, MSG, ##__VA_ARGS__)

#define CHECK_STR(A, B, MSG, ...)                                       \\
  do                                                                    \\
    {                                                                   \\
      char const *str_a = (A);                                          \\
      char const *str_b = (B);                                          \\
      if (strcmp (str_a, str_b) != 0)                                   \\
        {                                                               \\
          enum { CHECK_MAX_BUFFER = 250 };                              \\
          static char buffer[CHECK_MAX_BUFFER];                         \\
          sprintf (buffer, MSG, ##__VA_ARGS__);                         \\
          fail ("(%s) == (%s) \\n  Actual: %s \\n  Expected: %s \\n  %s",  \\
                #A, #B, str_a, str_b, buffer);                          \\
        }                                                               \\
    }                                                                   \\
  while (0)

#define CHECK_PTR(A, MSG, ...)                                          \\
  do                                                                    \\
    {                                                                   \\
      enum { MAX_BUFFER = 250 };                                        \\
      static char buffer[MAX_BUFFER];                                   \\
      sprintf (buffer, MSG, ##__VA_ARGS__);                             \\
      fail_unless ((A) != NULL, "(%s) != NULL \\n  %s", #A, buffer);     \\
    }                                                                   \\
  while (0)

#define CHECK_NULL_PTR(A, MSG, ...)                                     \\
  do                                                                    \\
    {                                                                   \\
      enum { MAX_BUFFER = 250 };                                        \\
      static char buffer[MAX_BUFFER];                                   \\
      sprintf (buffer, MSG, ##__VA_ARGS__);                             \\
      fail_unless ((A) == NULL, "(%s) == NULL \\n  %s", #A, buffer);     \\
    }                                                                   \\
  while (0)

Update 2007/10/05: Fix: using macro parameters more than once might cause multiple unnecessary function calls.

Update 2007/08/11: I have updated the macros so variable number of arguments are allowed (see variadic macros).

Advertisements

Unit Testing tools

November 9, 2006

Most of developers know how great and useful is unit testing. What is even greater is the bunch of libraries you can find out there for almost any language around (C, C++, Objective-C, Java, Pyhton, Perl…).

All these tools are great (I have used some of the ones I have mentioned), but what happens when you start working in a big and commercial project and your bosses have decided to buy a fantastic tool (I’m not going to mention the one I will talk about) which costs a lot of money and they have never try it? What happens is that you, the developer, start working in this fantastic tool and start feeling sort of depressing.

We started our project a year ago and by now, I have had to send so many e-mails to the unit testing software company that I can’t remember. The tool looked promising at first: unit testing, static analysis, user interfaces to speed up the process, cross platform (we use a cross compiler) and many other things. I started to use the GUI without much success, I don’t even remember how it looked like. After a couple of days we decided (we had it already in mind) to create the tests by hand (with help of some elisp code). After a month or so, we had our first problem: the parser for static analysis did not support some of the C99 syntax, first e-mail and first example to show the problem. It took like two months or more to fix the problem, this means that we had to code without using some of the syntax we wanted to use, and of course we had to remember it each time. After some months of tranquility, second problem: a header file provided by the unit testing software had some errors, so second e-mail with a fix suggestion. This time (or even before) I started thinking: “why have I to use this tool? I’m only using the basic macros all the free libraries already give me and no one in the company is looking at the metrics generated by the software”. Nevertheless, I kept on using the tool. A month ago (and after 6000 € paid for support), third problem: I got linking errors using the static analysis again. I prepared another test and sent it to the company again. They started by sending me an update that did not fix the C99 issue I mentioned above, and of course didn’t fix the current one. Then, I recognize it, I made a mistake and I saw a new issue when there really was no one, despite that the current problem was not fixed. This morning, I received a new version of the executables which were supposed to fix the problem. Again, no luck, no problem resolved. May be I made a mistake again, but no one has complained by now. Is any one trying the fix before they send it? Am I too exigent?

My conclusion is: whenever you can, avoid the tools you don’t know and use the ones you do. Commercial tools may not worth the money.

A co-worker told me that one can write its own code to perform unit testing, and his is totally right. If the problem is that the free libraries are not certified or whatever, write your own code and it will be certified as it will be the rest of the code.

Happy testing!