Posts: Lutok

lutok-0.3, kyua-testers-0.2 and kyua-cli-0.8

Yesterday was release day: I pushed out Lutok 0.3, Kyua Testers 0.2 and Kyua CLI 0.8.

There are not a lot of changes in these new releases. The reason I cut them was to publish the new TAP-compliant tester and make it available for use in FreeBSD as soon as possible. I will be using this new feature as part of the FreeBSD Test Suite in order to hook existing test programs without having to rewrite them to use the ATF libraries (or at least not as a first step).

December 8, 2013 · Tags: kyua, lutok, release
Continue reading (about 2 minutes)

Lutok 0.3 released

Lutok 0.3 was released yesterday evening. The packages in pkgsrc-current, Fedora 19 and Fedora rawhide have all been updated accordingly.

The major highlight of this new release is support for Lua 5.2 while retaining backwards compatibility with Lua 5.1. The incompatible changes between 5.1 and 5.2 only affected a small subset of the functionality in Lutok, which made this dual support possible.

For those that don't know what this project is about: Lutok is a lightweight C++ API for Lua. It is lightweight because it's an almost 1:1 mapping of the C API into C++. However, the major design criterion behind Lutok is to provide an interface that is fully C++ native and that is safe to use in the face of exceptions. For this reason, Lutok is not as performant as the native C library, which is OK for many use cases out there.

June 15, 2013 · Tags: lutok, release
Continue reading (about 2 minutes)

Projects migrated to Git

I finally took the plunge. Yesterday night, I migrated the Kyua and Lutok repositories from Subversion to Git. And this morning I migrated ATF from Monotone and custom hosting to Git and Google Code; oh, and this took way longer than expected.

Migration of Kyua and Lutok

Migrating these two projects was straightforward. After preparing a fresh local Git repository following the instructions posted yesterday, pushing to Google Code is a simple matter:

$ git remote add googlecode https://code.google.com/p/your-project
$ git push googlecode --all
$ git push googlecode --tags

February 26, 2012 · Tags: atf, git, kyua, lutok
Continue reading (about 3 minutes)

Switching projects to Git

The purpose of this post is to tell you the story of the Version Control System (VCS) choices I have made while maintaining my open source projects ATF, Kyua and Lutok. It also details where my thoughts are headed to these days.

This is not a description of centralized vs. distributed VCSs, and it does not intend to be one. This does not intend to compare Monotone to Git either, although you'll probably feel like it while reading the text. Note that I have fully known the advantages of DVCSs over centralized systems for many years, but for some reason or another I have been "forced" to use centralized systems on and off. The Subversion hiccup explained below is... well... regrettable, but it's all part of the story!

Hope you enjoy the read.


Looking back at Monotone (and ATF)

I still remember the moment I discovered Monotone in 2004: simply put, it blew my mind. It was clear to me that Distributed Version Control Systems (DVCSs) were going to be the future, and I eagerly adopted Monotone for my own projects. A year later, Git appeared and it took all the praise for DVCSs: developers all around started migrating en masse to Git, leaving behind other (D)VCSs. Many of these developers then went on to make Git usable (it certainly wasn't at first) and well-documented. (Note: I really dislike Git's origins... but I won't get into details; it has been many years since that happened.)

One of the projects in which I chose to use Monotone was ATF. That might have been a good choice at the time despite being very biased, but it has caused problems over time. These have been:

• Difficulty to get Monotone installed: While most Linux distributions come with a Monotone binary package these days, it was not the case years ago. But even nowadays if all Linux distributions have binary packages, the main consumers of ATF are NetBSD users, and their only choice is to build their own binaries. This generates discomfort because there is a lot of FUD surrounding C++ and Boost.
• High entry barrier to potential contributors: It is a fact that Monotone is not popular, which means that nobody is familiar with it. Monotone's CLI is very similar to CVS, and I'd say the knowledge transition for basic usage is trivial, but the process of cloning a remote project was really convoluted until "recently". The lack of binary packages, combined with complex instructions on just how to fetch the sources of a project only help in scaring people away.
• Missing features: Despite years have passed, Monotone still lacks some important features that impact its usability. For example, to my knowledge, it's still not possible to do work-directory merges and, while the interactive merges offered by the tool seem like a cool idea, they are not really practical as you get no chance to validate the merge. It is also not possible, for example, to reference the parent commit of any given commit without looking at the parent's ID. (Yeah, yeah, in a DAG there may be more than one parent, but that's not the common case.) Or know what a push/pull operation is going to change on both sides of the connection. And key management and trust has been broken since day one and is still not fixed. Etc, etc, etc.
• No hosting: None of the major project hosting sites support Monotone. While there are some playground hosting sites, they are toys. I have also maintained my own servers sometimes, but it's certainly inconvenient and annoying.
• No tools support: Pretty much no major development tools support Monotone as a VCS backend. Consider Ohloh, your favorite bug tracking system or your editor/IDE. (I attempted to install Trac with some alpha plugin to add Monotone support and it was a huge mess.)
• No more active development: This is the drop that spills the cup. The developers of Monotone that created the foundations of the project left years ago. While the rest of the developers did a good job in coming up with a 1.0 release by March 2011, nothing else has happened since then. To me, it looks like a dead project at this point :-(

• ATF continues to be hosted in a Monotone repository, and Kyua depends on ATF. You can spot the problem, can't you? It's a nightmare to check out all the dependencies of Kyua, using different tools, just to get the thing working.
• As of today, Git is as popular, if not more, than Subversion. All the major operating systems have binary packages for Git and/or bundle Git in their base installation (hello, OS X!). Installing Git on NetBSD is arguably easier (at least faster!) than Subversion. Developers are used to Git. Or let me fix that: developers love Git.
• Subversion gets on the way more than it helps; it really does once you have experienced what other VCSs have to offer. I currently maintain independent checkouts of the repository (appropriately named 1, 2 and 3) so that I can develop different patches on each before committing the changes. This gets old really quickly. Not to mention when I have to fly for hours, as being stuck without an internet connection and plain-old Subversion... is suboptimal. Disconnected operation is key.

February 11, 2012 · Tags: atf, git, kyua, lutok, monotone, vcs
Continue reading (about 8 minutes)

Kyua: Weekly status report

• Created an RPM package for Lutok for inclusion in Fedora.
• Created a preliminary RPM spec for ATF for Fedora. Now in discussions with the FPC to figure out how to install the tests on a Fedora system, as /usr/tests may not be appropriate.
• No activity on Kyua itself though, unfortunately.

February 7, 2012 · Tags: atf, kyua, lutok, report
Continue reading (about 1 minute)

Kyua: Weekly status report

This comes two days late... but anyway, I felt like posting it now instead of waiting until next Sunday/Monday.

The activity past week focused mostly on implementing support for the require.memory test-case metadata property recently introduced into ATF. This was non-trivial due to the need to write some tricky Autoconf code to make this "slightly portable". Seriously: It's scary to see how hard it is to perform, in a portable manner, an operation as simple as "query the amount of physical memory"... but oh well, such are the native Unix APIs...

I later spent the weekend on Lutok, preparing and publishing the project's first release and writing my first RPM library spec for it. This was a prerequisite for the upcoming 0.3 release of Kyua and thus deserved special attention!

February 1, 2012 · Tags: kyua, lutok, report
Continue reading (about 1 minute)

Lutok 0.1 available

A few months ago, I introduced the Lutok project, a simple C++ API for Lua. To recap: the major goal of this API, which does not mimic the Lua C API bit by bit, is to enforce correct coding practices on the client side. This is done by (ab)using the RAII programming pattern to automatically free resources when not needed and to ensure that the Lua stack is correctly managed. The library also adheres to common C++ programming idioms and exposes exceptions for error management and uses the pimpl idiom to completely hide the Lua C API from clients of Lutok (unless you use the c_gate backdoor!).

Today, I am pleased to announce that the first formal release of Lutok, obviously named 0.1, is available for download! You can obtain this release by visiting the lutok-0.1.tar.gz download page.

Also, in preparation for this release, I have spent the weekend writing some little example programs to demonstrate the usage of Lutok, and also some installation instructions.

Hope you find this release useful and please do send me any comments you may have!

Hint: Yes, releasing Lutok 0.1 was a prerequisite for Kyua 0.3. So stay tuned ;-)

January 29, 2012 · Tags: lutok, release
Continue reading (about 1 minute)

Kyua: Weekly status report (db designdoc edition!)

• Moved the code of utils::lua to a new project, Lutok.
• Attempted to integrate a copy of Lutok into the Kyua source code to simplify installing Kyua. I have been playing with Subversion externals and Autoconf/Automake hacks to make this happen, but unfortunately haven't got a pleasant solution yet.
• Modified Lutok to not expose the Lua C API at all from header files and cleaned up the Kyua code to cope with the changes.
• Been chewing through the first chapters of the "Using SQLite" book to refresh my SQL skills.
• And, most importantly, wrote a preliminary design document for the database store of Kyua and the reporting features. Comments certainly welcome! Be aware that this is how atf-report will be replaced, so once this is done we should be able to finally kill atf-run and atf-report altogether :-)

September 18, 2011 · Tags: kyua, lutok, report
Continue reading (about 1 minute)

Name your C++ auto-cleaners

As you may already know, RAII is a very powerful and popular pattern in the C++ language. With RAII, you can wrap non-stack-managed resources into a stack-managed object such that, when the stack-managed object goes out of scope, it releases the corresponding non-stack-managed object. Smart pointers are just one example of this technique, but so are IO streams too.

Before getting into the point of the article, bear with me for a second while I explain what the  stack_cleaner object of Lutok is. The "stack cleaner" takes a reference to a Lua state and records the height of the Lua stack on creation. When the object is destroyed (which happens when the declaring function exits), the stack is returned to its previous height thus ensuring it is clean. It is always a good idea for a function to prevent side-effects by leaving its outside world as it was — and, like it or not, the Lua state is part of the outside world because it is an input/output parameter to many functions.

Let's consider a piece of code without using the stack cleaner:

void
my_function(lutok::state& state, const int foo)
{
    state.push_integer(foo);

    ... do something else in the state ...
    const int bar = state.to_integer();
    if (bar != 3) {
        state.pop(1); 
        throw std::runtime_error("Invalid data!");
    }
    state.pop(1);

}

void
my_function(lutok::state& state, const int foo)
{
    state.push_integer(foo);
    try { 
        ... do something else in the state ...
        const int bar = state.to_integer();
        if (bar != 3 
           throw std::runtime_error("Invalid data!");
    } catch (...) {
        state.pop(1);
        throw;
    }
    state.pop(1);
}

void
my_function(lutok::state& state, const int foo)
{
    lutok::stack_cleaner cleaner(state);

    state.push_integer(foo);
    ... do something else in the state ...
    const int bar = state.to_integer();
    if (bar != 3) 
        throw std::runtime_error("Invalid data!"); 
}

lutok::stack_cleaner(state);

lutok::stack_cleaner ANY_NAME_HERE(state);

September 17, 2011 · Tags: cxx, lua, lutok
Continue reading (about 4 minutes)

Introducing Lutok: A lightweight C++ API for Lua

It has finally happened. Lutok is the result of what was promised in the "Splitting utils::lua from Kyua" web post.

Quoting the project web page:

Lutok provides thin C++ wrappers around the Lua C API to ease the interaction between C++ and Lua. These wrappers make intensive use of RAII to prevent resource leakage, expose C++-friendly data types, report errors by means of exceptions and ensure that the Lua stack is always left untouched in the face of errors. The library also provides a small subset of miscellaneous utility functions built on top of the wrappers.

Lutok focuses on providing a clean and safe C++ interface; the drawback is that it is not suitable for performance-critical environments. In order to implement error-safe C++ wrappers on top of a Lua C binary library, Lutok adds several layers or abstraction and error checking that go against the original spirit of the Lua C API and thus degrade performance.

Lutok was originally developed within Kyua but was later split into its own project to make it available to general developers.

September 15, 2011 · Tags: announce, cxx, kyua, lua, lutok
Continue reading (about 2 minutes)