A difficult simple test framework

I'm working on writing a few unit tests for the latest of my projects and have come up with a fairly simple test code template that allows both easy implementation as well as straightforward extensibility.

The general idea is to put all the test cases into an array of function pointers. The following is an example of the code:

// Define the shape of the function pointer
typedef void(*fn_ptr)(DataType&);
#define TESTCASE(fn) void (fn)(DataType& param)

TESTCASE(test1); // Declare a testcase
TESTCASE(test2); // Declare another

// Define the global array
const fn_ptr test_array[] =
{
   test1,
   test2,
};
const int NumOfCases = sizeof(test_array) / sizeof(fn_ptr);

// implement the test case
TESTCASE(test1)
{
   // Do something
}

// The main test driver
int main()
{
   ...
   for (int i=0; i<NumOfCases; i++)
   {
      // do some test case initialization
      (*test_array[i])(param);
      // clean up the test case resources
   }
   ...
}

The benefit of this code is that once the test case initialization and cleanup are taken care of, the developer/tester can concentrate on implementing test cases rather than trying to worry about those mundanities in every test case. The test case writer just comes up with a good name for a test case, uses the TESTCASE() macro to declare the function, adds the new function to test_array[], and then adds the test implementation. The test will automatically be run the next time through, plus there is no reason why tests can't be injected directly into the array rather than appended to the end. One other benefit that is realizable with this setup is that with good planning, it may be possible to break the array into discrete test areas such that only the test cases specified by offset are run (I may add this functionality later if it is useful and necessary).

The drawback is (besides its unreadability) is that it is tedious to add a function declaration AND add that new function to test_array[]. What I am trying to think of here is a way to condense those two operations into a single operation which would simplify this test program template even further. It does no one any good if a test case is written but is never entered into the test_array.

Visual Studio 2003 has macro capabilities, and I may end up relying on that, but I'd much rather be able to implement something using the standard C++ preprocessor and compiler stack rather than anything IDE-specific. As of now, I'm doing it by hand and it is error prone and difficult once the list of test cases becomes longer than half the screen size.

If there is a better way to do this, I'm all ears.

How would you test this code?

This past weekend I went through a hellish interview with the home office PM for one of the positions I am interviewing for. Making the interview harder than usual was the fact that it was held early in the morning to synchronize our schedules. Also, it was a phone interview and that brought with it all the bad reception problems that that brings along.

Groggy and unable to hear half the interview, I plowed through the best I could. Unfortunately, I don't think I made the best impression I could.

One question I got especially hung up on had to do with testing of code. First, I was asked to code up a fairly simple set of functions that translated some data from one format to another. Then I was grilled for 15 minutes on the intricacies of the C function call conventions. He asked me why the incremented parameter in the subfunction was not also updating the passed variable in the calling function and I couldn't wrap my head around what he was asking. Pass by value, pass by value, pass by value. It's all I could think of to answer. After 15 minutes of trying to dig deeper into the question, I finally figured out that he was trying to get me to explain how pass by value works and how it uses the callstack, etc. That was a big minus, I think.

After answering the question to his satisfaction (as satisfied he could be with an answer that took 15 minutes to cajole out of me), he asked me how I would test this code. I don't think I have a good answer for that.

Obviously there are some general criteria for any testing. Internal consistency (parameters OK), external consistency (return values OK), performance criteria, thread safety, correct processing, correct error handling, etc. However, none of these came to mind due to the frazzled state of mind. What did come to mind were specific test cases that exercised the function as much as necessary, but I don't think that's what he was looking for.

To be frank, I believe myself to be undertrained in this general area. I am looking to improve my code testing processes, and am wondering where to begin looking. Kaner, et al. is where most of my testing knowledge comes from, but that generally approaches the testing process from a tester's point of view rather than a programmer's point of view.

I am aware of methodologies such as Agile, but I am looking for something more basic that can give me a good idea about what code testing is and the useful techniques for approaching the problem. The ultimate goal isn't just to pass some interview, but rather to improve my efficiency as a programmer.

Any ideas where to start looking?