This weekend I have finally been able to start coding for my SoC project: the Automated Testing Framework for NetBSD. To my disliking, this has been delayed too much... but I was so busy with my PFC that I couldn't find any other chance to get my hands on it.

I've started by working on two core components:

• libatf: The C/C++ library that will provide the interface to easily write test cases and test suites. Among its features will be the ability to report the results of each test, a way to define meta-data for each test, etc.
  
• libatfmain: A library that provides a default entry point for test programs that takes care to run the test cases in a controlled environment — e.g. it captures all signals and deals with them gracefully — and provides a standard command-line interface for them.

Soon after I started coding, I realized that I could need to write my own C code to deal with data collections and safe strings. I hate that, because that's a very boring task — it is not related to the problem at hand at all — and because it involves reinventing the wheel: virtually all other languages provide these two features for-free. But wait! NetBSD has a C++ compiler, and atf won't be a host tool¹. So... I can take advantage of C++, and I'll try to. Calm down! I'll try to avoid some of the "complex" C++ features as much as possible to keep the executables' size small enough. You know how the binaries' size blows up when using templates... Oh, and by the way: keep in mind that test cases will be typically written in POSIX shell script, so in general you won't need to deal with the C++ interface.

Furthermore, I see no reason for atf to be tied to NetBSD. The test cases will surely be, but the framework needn't. Thus I'm thinking of creating a standalone package for atf itself and distributing it as a completely independent project (under the TNF² umbrella), which will later be imported into the NetBSD source tree as we currently do for other third-party projects such as Postfix. In fact, I've already started work on this direction by creating the typical infrastructure to use the GNU auto-tools. Of course this separation could always be done at a later step in the development, but doing it from the very beginning ensures the code is free of NetBSD-isms, emphasizes the portability desire and keeps the framework self-contained.

I'd like to hear your comments about these "decisions" :-)

¹ A host tool is a utility that is built with the operating system's native tools instead of with the NetBSD's tool-chain: i.e. host tools are what build.sh tools builds. Such tools need to be highly portable because they have to be accepted by old compilers and bizarre build environments.

² TNF = The NetBSD Foundation.