Posts: Lua

Three productive days on the Kyua front

This being Thanksgiving week in the U.S. and Google giving us Thursday and Friday off, I decided to take Monday to Wednesday off as well to spend some time hacking on Kyua — yes, finally, after months of being inactive. And what a three productive days!

Here comes a little briefing on the three fronts in which I made progress. (This puts on hold the header files series until next Monday... but most of you are probably away anyway. Enjoy the holidays if they apply to you!)

November 28, 2013 · Tags: atf, fedora, freebsd, kyua, lua
Continue reading (about 6 minutes)

Exposing a configuration tree through Lua

In the previous post, I discussed the type-safe tree data structure that is now in the Kyua codebase, aimed at representing the configuration of the program. In this post, we'll see how this data structure ties to the parsing of the configuration file.

One goal in the design of the configuration file was to make its contents a simple key/value association (i.e. assigning values to predetermined configuration variables). Of course, the fact that the configuration file is just a Lua script means that additional constructions (conditionals, functions, etc.) can be used to compute these values before assignment, but in the end all we want to have is a collection of values for known keys. The tree data structure does exactly the latter: maintain the mapping of keys to values, and ensuring that only a set of "valid" keys can be set. But, as a data structure, it does not contain any of the "logic" involved in computing those values: that is the job of the script.

Now, consider that we have the possible following syntaxes in the configuration file:

simple_variable = "the value"
complex.nested.variable = "some other value"

These assignments map, exactly, to a tree::set() function call: the name of the key is passed as the first argument to tree::set() and the value is passed as the second argument. (Let's omit types for simplicity.) What we want to do is modify the Lua environment so that these assignments are possible, and that when such assignments happen, the internal tree object gets updated with the new values.

In order to achieve this, the configuration library modifies the Lua environment as follows:

• The newindex metatable method of _G is overridden so that an assignment causes a direct call to the set method of the referenced key. The key name is readily available in the newindex arguments, so no further magic is needed. This handles the case of "a = b" (top-level variables).
• The index metatable method of _G is overridden so that, if the indexed element is not found, a new table is generated and injected into _G. This new table has a metatable of its own that performs the same operations as the newindex and index herein described. This handles the case of "a.b = c", as this trick causes the intermediate tables (in this case "a") to be transparently created.
• Each of the tables created by index has a "key" metatable field that contains the fully qualified key of the node the table corresponds to. This is necessary to be able to construct the full key to pass to the set method.
• There is further magic to ensure that values pre-populated in the tree (aka default values) can be queried from within Lua, and that variables can be set more than once. These details are uninteresting though.

June 2, 2012 · Tags: cxx, kyua, lua
Continue reading (about 3 minutes)

Rethinking Kyua's configuration system

In the previous blog post, I described the problems that the implementation of the Kyua configuration file parsing and in-memory representation posed. I also hinted that some new code was coming and, after weeks of work, I'm happy to say that it has just landed in the tree!

May 28, 2012 · Tags: cxx, kyua, lua
Continue reading (about 4 minutes)

Name your C++ auto-cleaners

As you may already know, RAII is a very powerful and popular pattern in the C++ language. With RAII, you can wrap non-stack-managed resources into a stack-managed object such that, when the stack-managed object goes out of scope, it releases the corresponding non-stack-managed object. Smart pointers are just one example of this technique, but so are IO streams too.

Before getting into the point of the article, bear with me for a second while I explain what the  stack_cleaner object of Lutok is. The "stack cleaner" takes a reference to a Lua state and records the height of the Lua stack on creation. When the object is destroyed (which happens when the declaring function exits), the stack is returned to its previous height thus ensuring it is clean. It is always a good idea for a function to prevent side-effects by leaving its outside world as it was — and, like it or not, the Lua state is part of the outside world because it is an input/output parameter to many functions.

Let's consider a piece of code without using the stack cleaner:

void
my_function(lutok::state& state, const int foo)
{
    state.push_integer(foo);

    ... do something else in the state ...
    const int bar = state.to_integer();
    if (bar != 3) {
        state.pop(1); 
        throw std::runtime_error("Invalid data!");
    }
    state.pop(1);

}

void
my_function(lutok::state& state, const int foo)
{
    state.push_integer(foo);
    try { 
        ... do something else in the state ...
        const int bar = state.to_integer();
        if (bar != 3 
           throw std::runtime_error("Invalid data!");
    } catch (...) {
        state.pop(1);
        throw;
    }
    state.pop(1);
}

void
my_function(lutok::state& state, const int foo)
{
    lutok::stack_cleaner cleaner(state);

    state.push_integer(foo);
    ... do something else in the state ...
    const int bar = state.to_integer();
    if (bar != 3) 
        throw std::runtime_error("Invalid data!"); 
}

lutok::stack_cleaner(state);

lutok::stack_cleaner ANY_NAME_HERE(state);

September 17, 2011 · Tags: cxx, lua, lutok
Continue reading (about 4 minutes)

Introducing Lutok: A lightweight C++ API for Lua

It has finally happened. Lutok is the result of what was promised in the "Splitting utils::lua from Kyua" web post.

Quoting the project web page:

Lutok provides thin C++ wrappers around the Lua C API to ease the interaction between C++ and Lua. These wrappers make intensive use of RAII to prevent resource leakage, expose C++-friendly data types, report errors by means of exceptions and ensure that the Lua stack is always left untouched in the face of errors. The library also provides a small subset of miscellaneous utility functions built on top of the wrappers.

Lutok focuses on providing a clean and safe C++ interface; the drawback is that it is not suitable for performance-critical environments. In order to implement error-safe C++ wrappers on top of a Lua C binary library, Lutok adds several layers or abstraction and error checking that go against the original spirit of the Lua C API and thus degrade performance.

Lutok was originally developed within Kyua but was later split into its own project to make it available to general developers.

September 15, 2011 · Tags: announce, cxx, kyua, lua, lutok
Continue reading (about 2 minutes)

Splitting utils::lua from Kyua

If you remember a post from January titled C++ interface to Lua for Kyua (wow, time flies), the Kyua codebase includes a small library to wrap the native Lua C library into a more natural C++ interface. You can take a look at the current code as of r129.

Quoting the previous post:

The utils::lua library provides thin C++ wrappers around the Lua C API to ease the interaction between C++ and Lua. These wrappers make intensive use of RAII to prevent resource leakage, expose C++-friendly data types, report errors by means of exceptions and ensure that the Lua stack is always left untouched in the face of errors. The library also provides a place (the operations module) to add miscellaneous utility functions built on top of the wrappers.

September 3, 2011 · Tags: cxx, kyua, lua
Continue reading (about 3 minutes)

Injecting C++ functions into Lua

The C++ interface to Lua implemented in Kyua exposes a lua::state class that wraps the lower-level lua_State* type. This class completely hides the internal C type of Lua to ensure that all calls that affect the state go through the lua::state class.

Things get a bit messy when we want to inject native functions into the Lua environment. These functions follow the prototype represented by the lua_CFunction type:

typedef int (*lua_CFunction)(lua_State*);

int
awesome_native_function(lua_State* state)
{
    // Uh, we have access to s, so we bypass the lua::state!
    ... do something nasty ...

    // Oh, and we can throw an exception here...
    //with bad consequences.
}

void
setup(...)
{
    lua::state state;
    state.push_c_function(awesome_native_function);
    state.set_global("myfunc");
    ... run some script ...
}

typedef int (*cxx_function)(lua::state&)

template< cxx_function Function >
int
wrap_cxx_function(lua_State* state)
{
    try {
        lua::state state_wrapper(state);
        return Function(state_wrapper);
    } catch (...) {
        luaL_error(state, "Geez, don't go into C's land!");
    }
}

int
awesome_native_function(lua::state& state)
{
    // See, we cannot access lua_State* now.

    ... do something ...
    throw std::runtime_error("And we can even do this!");
}

void
setup(...)
{
    lua::state state;
    state.push_c_function(
        wrap_cxx_function< awesome_native_function >);
    state.set_global("myfunc");
    ... run some script ...
}

January 17, 2011 · Tags: cxx, kyua, lua
Continue reading (about 2 minutes)

Error handling in Lua: the Kyua approach

About a week ago, I detailed the different approaches I encountered to deal with errors raised by the Lua C API. Later, I announced the new C++ interface for Lua implemented within Kyua. And today, I would like to talk about the specific mechanism I implemented in this library to deal with the Lua errors.

The first thing to keep in mind is that the whole purpose of Lua in the context of Kyua is to parse configuration files. This is an infrequent operation, so high performance does not matter: it is more valuable to me to be able to write robust algorithms fast than to have them run at optimal speed. The other key point to consider is that I want Kyua to be able to use prebuilt Lua libraries, which are built as C binaries.

The approach I took is to wrap every single unsafe Lua C API call in a "thin" (FSVO thin depending on the case) wrapper that gets called by lua_pcall. Anything that runs inside the wrapper is safe to Lua errors, as they are caught and safely reported to the caller.

Lets examine how this works by taking a look at an example: the wrapping of lua_getglobal. We have the following code (copy pasted from the utils/lua/wrap.cpp file but hand-edited for publishing here):

static int
protected_getglobal(lua_State* state)
{
    lua_getglobal(state, lua_tostring(state, -1));
    return 1;
}

void
lua::state::get_global(const std::string& name)
{
    lua_pushcfunction(_pimpl->lua_state, protected_getglobal);
    lua_pushstring(_pimpl->lua_state, name.c_str());
    if (lua_pcall(_pimpl->lua_state, 1, 1, 0) != 0)
        throw lua::api_error::from_stack(_pimpl->lua_state,
            "lua_getglobal");
}

January 14, 2011 · Tags: cxx, kyua, lua
Continue reading (about 3 minutes)

C++ interface to Lua for Kyua

Finally! After two weeks of holidays work, I have finally been able to submit Kyua's r39: a generic library that implements a C++ interface to Lua. The code is hosted in the utils/lua/ subdirectory.

From the revision description:

The utils::lua library provides thin C++ wrappers around the Lua C API to ease the interaction between C++ and Lua. These wrappers make intensive use of RAII to prevent resource leakage, expose C++-friendly data types, report errors by means of exceptions and ensure that the Lua stack is always left untouched in the face of errors. The library also provides a place (the operations module) to add miscellaneous utility functions built on top of the wrappers.

January 8, 2011 · Tags: cxx, kyua, lua
Continue reading (about 2 minutes)

Error handling in Lua

Some of the methods of the Lua C API can raise errors. To get an initial idea on what these are, take a look at the Functions and Types section and pay attention to the third field of a function description (the one denoted by 'x' in the introduction).

my_array = nil
return my_array["test"]

lua_getglobal(state, "my_array");
lua_pushstring(state, "test");
lua_gettable(state, -2);

January 7, 2011 · Tags: c, cxx, lua
Continue reading (about 6 minutes)

Using RAII to clean up temporary values from a stack

For the last couple of days, I have been playing around with the Lua C API and have been writing a thin wrapper library for C++. The main purpose of this auxiliary library is to ensure that global interpreter resources such as the global state or the execution stack are kept consistent in the presence of exceptions — and, in particular, that none of these are leaked due to programming mistakes when handling error codes.

To illustrate this point, let's forget about Lua and consider a simpler case. Suppose we lost the ability to pass arguments and return values from functions in C++ and all we have is a stack that we pass around. With this in mind, we could implement a multiply function as follows:

void multiply(std::stack< int >& context) {
    const int arg1 = context.top();
    context.pop();
    const int arg2 = context.top();
    context.pop();
    context.push(arg1 * arg2);
}

std::stack< int > context;
context.push(5);
context.push(6);
multiply(context);
const int result = s.top();
s.pop();

void magic(std::stack< int >& context) {
    const int arg1 = context.top();
    context.pop();
    const int arg2 = context.top();
    context.pop();

    context.push(arg1 * arg2);
    context.push(arg1 / arg2);
    try {
        ... do something with the two values on top ...

        context.push(arg1 - arg2);
        try {
            ... do something with the three values on top ...
        } catch (...) {
            context.pop();  // arg1 - arg2
            throw;
        }
        context.pop();
    } catch (...) {
        context.pop();  // arg1 / arg2
        context.pop();  // arg1 * arg2
        throw;
    }
    context.pop();
    context.pop();
}

class temp_stack {
    std::stack< int >& _stack;
    int _pop_count;

public:
    temp_stack(std::stack< int >& stack_) :
        _stack(stack_), _pop_count(0) {}

    ~temp_stack(void)
    {
        while (_pop_count-- > 0)
            _stack.pop();
    }

    void push(int i)
    {
        _stack.push(i);
        _pop_count++;
    }
};

void magic(std::stack< int >& context) {
    const int arg1 = context.top();
    context.pop();
    const int arg2 = context.top();
    context.pop();

    temp_stack temp(context);

    temp_stack.push(arg1 * arg2);
    temp_stack.push(arg1 / arg2);
    ... do something with the two values on top ...

    temp_stack.push(arg1 - arg2);
    ... do something with the three values on top ...

    // Yes, we can return now.  No need to do manual pop()s!
}

December 27, 2010 · Tags: c, cxx, kyua, lua
Continue reading (about 6 minutes)

Kyua: Design of the configuration system

Over a week ago, I mostly finished the implementation of the runtime engine for test cases of Kyua and, along the way, realized that it is imperative to write a configuration system right now before the configuration code becomes messier than it already is.

To that end, I spent the last week working on a design document for the configuration system. Summarizing, the document describes what the requirements for the configuration files of Kyua are, what the possible alternatives to implement them are, and advocates the use of Lua — a tiny embedded programming language — to bring these configuration files to life.

It is your chance to get involved in the early stages of the development of Kyua! :-) Take a look at the email to kyua-discuss asking for comments and feel free to join the mailing list ;-)

December 22, 2010 · Tags: kyua, lua
Continue reading (about 1 minute)