lutok-0.3, kyua-testers-0.2 and kyua-cli-0.8
Yesterday was release day: I pushed out Lutok 0.3, Kyua Testers 0.2 and Kyua CLI 0.8. There are not a lot of changes in these new releases. The reason I cut them was to publish the new TAP-compliant tester and make it available for use in FreeBSD as soon as possible. I will be using this new feature as part of the FreeBSD Test Suite in order to hook existing test programs without having to rewrite them to use the ATF libraries (or at least not as a first step).
December 8, 2013
·
Tags:
<a href="/tags/kyua">kyua</a>, <a href="/tags/lutok">lutok</a>, <a href="/tags/release">release</a>
Continue reading (about
2 minutes)
Lutok 0.3 released
Lutok 0.3 was released yesterday evening. The packages in pkgsrc-current, Fedora 19 and Fedora rawhide have all been updated accordingly. The major highlight of this new release is support for Lua 5.2 while retaining backwards compatibility with Lua 5.1. The incompatible changes between 5.1 and 5.2 only affected a small subset of the functionality in Lutok, which made this dual support possible. For those that don't know what this project is about: Lutok is a lightweight C++ API for Lua. It is lightweight because it's an almost 1:1 mapping of the C API into C++. However, the major design criterion behind Lutok is to provide an interface that is fully C++ native and that is safe to use in the face of exceptions. For this reason, Lutok is not as performant as the native C library, which is OK for many use cases out there.
June 15, 2013
·
Tags:
<a href="/tags/lutok">lutok</a>, <a href="/tags/release">release</a>
Continue reading (about
2 minutes)
Projects migrated to Git
I finally took the plunge. Yesterday night, I migrated the Kyua and Lutok repositories from Subversion to Git. And this morning I migrated ATF from Monotone and custom hosting to Git and Google Code; oh, and this took way longer than expected. Migration of Kyua and Lutok Migrating these two projects was straightforward. After preparing a fresh local Git repository following the instructions posted yesterday, pushing to Google Code is a simple matter: $ git remote add googlecode https://code.google.com/p/your-project $ git push googlecode --all $ git push googlecode --tags One of the nice things I discovered while doing this is that a Google Code project supports multiple repositories when the VCS system is set to Git or Mercurial. By default, the system creates the default and wiki repositories, but you can add more at will. This is understandable given that, in Subversion, you have the ability to check out individual directories of a project whereas you cannot do that in the other supported VCSs: you actually need different repositories to group different parts of the project. I performed the full migration under a Linux box so that I could avail of the most recent Git version along with a proven binary package. The migration went alright, but I encountered a little problem when attempting a fresh checkout from NetBSD: git under NetBSD will not work correctly against SSL servers because it lacks the necessary CA certificates. The solution is to install the security/mozilla-rootcerts package and follow the instructions printed during installation; why this does not happen automatically escapes my mind. Migration of ATF I had been having doubts about migrating ATF itself, although if Kyua was moved to Git, it was a prerequisite to move ATF as well. Certainly I could convert the repository to Git, but where could I host it afterwards? Creating a new Google Code project just for this seemed too much of a hassle. My illumination came when I found out, as above, that Google Code supports an arbitrary amount of repositories in a particular project when converting it to Git. So, for ATF, I just ran mtn git_export with appropriate flags, created a new atf repository on the Kyua site, and pushed the contents there. Along the way, I also decided to kill the home-grown ATF web site and replace it by a single page containing all the relevant information. At this point, ATF and Kyua are supposed to work together quite tightly (in the sense that ATF is just a "subcomponent" of Kyua), so coupling the two projects on the same site makes sense. Now, let's drink the kool aid.
February 26, 2012
·
Tags:
<a href="/tags/atf">atf</a>, <a href="/tags/git">git</a>, <a href="/tags/kyua">kyua</a>, <a href="/tags/lutok">lutok</a>
Continue reading (about
3 minutes)
Switching projects to Git
The purpose of this post is to tell you the story of the Version Control System (VCS) choices I have made while maintaining my open source projects ATF, Kyua and Lutok. It also details where my thoughts are headed to these days. This is not a description of centralized vs. distributed VCSs, and it does not intend to be one. This does not intend to compare Monotone to Git either, although you'll probably feel like it while reading the text. Note that I have fully known the advantages of DVCSs over centralized systems for many years, but for some reason or another I have been "forced" to use centralized systems on and off. The Subversion hiccup explained below is... well... regrettable, but it's all part of the story! Hope you enjoy the read. Looking back at Monotone (and ATF) I still remember the moment I discovered Monotone in 2004: simply put, it blew my mind. It was clear to me that Distributed Version Control Systems (DVCSs) were going to be the future, and I eagerly adopted Monotone for my own projects. A year later, Git appeared and it took all the praise for DVCSs: developers all around started migrating en masse to Git, leaving behind other (D)VCSs. Many of these developers then went on to make Git usable (it certainly wasn't at first) and well-documented. (Note: I really dislike Git's origins... but I won't get into details; it has been many years since that happened.) One of the projects in which I chose to use Monotone was ATF. That might have been a good choice at the time despite being very biased, but it has caused problems over time. These have been: Difficulty to get Monotone installed: While most Linux distributions come with a Monotone binary package these days, it was not the case years ago. But even nowadays if all Linux distributions have binary packages, the main consumers of ATF are NetBSD users, and their only choice is to build their own binaries. This generates discomfort because there is a lot of FUD surrounding C++ and Boost.High entry barrier to potential contributors: It is a fact that Monotone is not popular, which means that nobody is familiar with it. Monotone's CLI is very similar to CVS, and I'd say the knowledge transition for basic usage is trivial, but the process of cloning a remote project was really convoluted until "recently". The lack of binary packages, combined with complex instructions on just how to fetch the sources of a project only help in scaring people away.Missing features: Despite years have passed, Monotone still lacks some important features that impact its usability. For example, to my knowledge, it's still not possible to do work-directory merges and, while the interactive merges offered by the tool seem like a cool idea, they are not really practical as you get no chance to validate the merge. It is also not possible, for example, to reference the parent commit of any given commit without looking at the parent's ID. (Yeah, yeah, in a DAG there may be more than one parent, but that's not the common case.) Or know what a push/pull operation is going to change on both sides of the connection. And key management and trust has been broken since day one and is still not fixed. Etc, etc, etc.No hosting: None of the major project hosting sites support Monotone. While there are some playground hosting sites, they are toys. I have also maintained my own servers sometimes, but it's certainly inconvenient and annoying.No tools support: Pretty much no major development tools support Monotone as a VCS backend. Consider Ohloh, your favorite bug tracking system or your editor/IDE. (I attempted to install Trac with some alpha plugin to add Monotone support and it was a huge mess.)No more active development: This is the drop that spills the cup. The developers of Monotone that created the foundations of the project left years ago. While the rest of the developers did a good job in coming up with a 1.0 release by March 2011, nothing else has happened since then. To me, it looks like a dead project at this point :-(Despite all this, I have been maintaining ATF in its Monotone repository, but I have felt the pain points above for years. Furthermore, the few times some end user has approached ATF to offer some contribution, he has had tons of trouble getting a fresh checkout of the repository and given up. So staying with Monotone hurts the project more than it helps. The adoption of Subversion (in Kyua) To fix this mess, when I created the Kyua project two years ago, I decided to use Subversion instead of a DVCS. I knew upfront that it was a clear regression from a functionality point of view, but I was going to live with it. The rationale for this decision was to make the entry barrier to Kyua much lower by using off-the-shelf project hosting. And, because NetBSD developers use CVS (shrugh), choosing Subversion was a reasonable choice because of the workflow similarities to CVS and thus, supposedly, the low entry barrier. Sincerely, the choice of Subversion has not fixed anything, and it has introduced its own trouble. Let's see why: ATF continues to be hosted in a Monotone repository, and Kyua depends on ATF. You can spot the problem, can't you? It's a nightmare to check out all the dependencies of Kyua, using different tools, just to get the thing working.As of today, Git is as popular, if not more, than Subversion. All the major operating systems have binary packages for Git and/or bundle Git in their base installation (hello, OS X!). Installing Git on NetBSD is arguably easier (at least faster!) than Subversion. Developers are used to Git. Or let me fix that: developers love Git.Subversion gets on the way more than it helps; it really does once you have experienced what other VCSs have to offer. I currently maintain independent checkouts of the repository (appropriately named 1, 2 and 3) so that I can develop different patches on each before committing the changes. This gets old really quickly. Not to mention when I have to fly for hours, as being stuck without an internet connection and plain-old Subversion... is suboptimal. Disconnected operation is key.The fact that Subversion is slowing down development, and the fact that it really does not help in getting new contributors more than Git would, make me feel it is time to say Subversion goodbye. The migration to Git At this point, I am seriously considering switching all of ATF, Lutok and Kyua to Git. No Mercurial, no Bazaar, no Fossil, no anything else. Git. I am still not decided, and at this point all I am doing is toying around the migration process of the existing Monotone and Subversion repositories to Git while preserving as much of the history as possible. (It's not that hard, but there are a couple of details I want to sort out first.) But why Git?First and foremost, because it is the most popular DVCS. I really want to have the advantages of disconnected development back. (I have tried git-svn and svk and they don't make the cut.)At work, I have been using Git for a while to cope with the "deficiencies" of the centralized VCS of choice. We use the squashing functionality intensively, and I find this invaluable to constantly and shamelessly commit incomplete/broken pieces of code that no-one will ever see. Not everything deserves being in the recorded history!Related to the above, I've grown accustomed to keeping unnamed, private branches in my local copy of the repository. These branches needn't match the public repository. In Monotone, you had this functionality in the form of "multiple heads for a given branch", but this approach is not as flexible as named private branches.Monotone is able to export a repository to Git, so the transition is easy for ATF. I have actually been doing this periodically so that Ohloh can gather stats for ATF.Lutok and ATF are hosted in Google Code, and this hosting platform now supports Git out of the box.No Mercurial? Mercurial looks a lot like Monotone, and it is indeed very tempting. However, the dependency on Python is not that appropriate in the NetBSD context. Git, without its documentation, builds very quickly and is lightweight enough. Plus, if I have to change my habits, I would rather go with Git given that the other open source projects I am interested in use Git.No Bazaar? No, not that popular. And the fact that this is based on GNU arch makes me cringe.No Fossil? This tool looks awesome and provides much more than DVCS functionality: think about distributed wiki and bug tracking; cool, huh? It also appears to be a strong contender in the current discussions of what system should NetBSD choose to replace CVS. However, it is a one-man effort, much like Monotone was. And few people are familiar with it, so Fossil wouldn't solve the issue of lowering the entry barrier. Choosing Fossil would mean repeating the same mistake as choosing Monotone.So, while Git has its own deficiencies — e.g. I still don't like the fact that it is unable to record file moves (heuristics are not the same) — it seems like a very good choice. The truth is, it will ease development by a factor of a million (OK, maybe not that much) and, because the only person (right?) that currently cares about the upstream sources for any of these projects is me, nobody should be affected by the change. The decision may seem a bit arbitrary given that the points above don't provide too much rationale to compare Git against the other alternatives. But if I want to migrate, I have to make a choice and this is the one that seems most reasonable. Comments? Encouragements? Criticisms?
February 11, 2012
·
Tags:
<a href="/tags/atf">atf</a>, <a href="/tags/git">git</a>, <a href="/tags/kyua">kyua</a>, <a href="/tags/lutok">lutok</a>, <a href="/tags/monotone">monotone</a>, <a href="/tags/vcs">vcs</a>
Continue reading (about
8 minutes)
Kyua: Weekly status report
Created an RPM package for Lutok for inclusion in Fedora.Created a preliminary RPM spec for ATF for Fedora. Now in discussions with the FPC to figure out how to install the tests on a Fedora system, as /usr/tests may not be appropriate.No activity on Kyua itself though, unfortunately.
February 7, 2012
·
Tags:
<a href="/tags/atf">atf</a>, <a href="/tags/kyua">kyua</a>, <a href="/tags/lutok">lutok</a>, <a href="/tags/report">report</a>
Continue reading (about
1 minute)
Kyua: Weekly status report
This comes two days late... but anyway, I felt like posting it now instead of waiting until next Sunday/Monday. The activity past week focused mostly on implementing support for the require.memory test-case metadata property recently introduced into ATF. This was non-trivial due to the need to write some tricky Autoconf code to make this "slightly portable". Seriously: It's scary to see how hard it is to perform, in a portable manner, an operation as simple as "query the amount of physical memory"... but oh well, such are the native Unix APIs... I later spent the weekend on Lutok, preparing and publishing the project's first release and writing my first RPM library spec for it. This was a prerequisite for the upcoming 0.3 release of Kyua and thus deserved special attention!
February 1, 2012
·
Tags:
<a href="/tags/kyua">kyua</a>, <a href="/tags/lutok">lutok</a>, <a href="/tags/report">report</a>
Continue reading (about
1 minute)
Lutok 0.1 available
A few months ago, I introduced the Lutok project, a simple C++ API for Lua. To recap: the major goal of this API, which does not mimic the Lua C API bit by bit, is to enforce correct coding practices on the client side. This is done by (ab)using the RAII programming pattern to automatically free resources when not needed and to ensure that the Lua stack is correctly managed. The library also adheres to common C++ programming idioms and exposes exceptions for error management and uses the pimpl idiom to completely hide the Lua C API from clients of Lutok (unless you use the c_gate backdoor!). Today, I am pleased to announce that the first formal release of Lutok, obviously named 0.1, is available for download! You can obtain this release by visiting the lutok-0.1.tar.gz download page. Also, in preparation for this release, I have spent the weekend writing some little example programs to demonstrate the usage of Lutok, and also some installation instructions. Hope you find this release useful and please do send me any comments you may have! Hint: Yes, releasing Lutok 0.1 was a prerequisite for Kyua 0.3. So stay tuned ;-)
January 29, 2012
·
Tags:
<a href="/tags/lutok">lutok</a>, <a href="/tags/release">release</a>
Continue reading (about
1 minute)
Kyua: Weekly status report (db designdoc edition!)
Moved the code of utils::lua to a new project, Lutok.Attempted to integrate a copy of Lutok into the Kyua source code to simplify installing Kyua. I have been playing with Subversion externals and Autoconf/Automake hacks to make this happen, but unfortunately haven't got a pleasant solution yet.Modified Lutok to not expose the Lua C API at all from header files and cleaned up the Kyua code to cope with the changes.Been chewing through the first chapters of the "Using SQLite" book to refresh my SQL skills. And, most importantly, wrote a preliminary design document for the database store of Kyua and the reporting features. Comments certainly welcome! Be aware that this is how atf-report will be replaced, so once this is done we should be able to finally kill atf-run and atf-report altogether :-)
September 18, 2011
·
Tags:
<a href="/tags/kyua">kyua</a>, <a href="/tags/lutok">lutok</a>, <a href="/tags/report">report</a>
Continue reading (about
1 minute)
Name your C++ auto-cleaners
As you may already know, RAII is a very powerful and popular pattern in the C++ language. With RAII, you can wrap non-stack-managed resources into a stack-managed object such that, when the stack-managed object goes out of scope, it releases the corresponding non-stack-managed object. Smart pointers are just one example of this technique, but so are IO streams too. Before getting into the point of the article, bear with me for a second while I explain what the stack_cleaner object of Lutok is. The "stack cleaner" takes a reference to a Lua state and records the height of the Lua stack on creation. When the object is destroyed (which happens when the declaring function exits), the stack is returned to its previous height thus ensuring it is clean. It is always a good idea for a function to prevent side-effects by leaving its outside world as it was — and, like it or not, the Lua state is part of the outside world because it is an input/output parameter to many functions. Let's consider a piece of code without using the stack cleaner: void my_function(lutok::state& state, const int foo) { state.push_integer(foo); ... do something else in the state ... const int bar = state.to_integer(); if (bar != 3) { state.pop(1); throw std::runtime_error("Invalid data!"); } state.pop(1); } Note that we have had to call state.pop(1) from "all" exit points of the function to ensure that the stack is left unmodified upon return of my_function. Also note that "all exit points" may not be accurate: in a language that supports exceptions, any statement may potentially raise an exception so to be really safe we should do: void my_function(lutok::state& state, const int foo) { state.push_integer(foo); try { ... do something else in the state ... const int bar = state.to_integer(); if (bar != 3 throw std::runtime_error("Invalid data!"); } catch (...) { state.pop(1); throw; } state.pop(1); } ... which gets old very quickly. Writing this kind of code is error-prone and boring. With an "auto-cleaner" object such as the stack_cleaner, we can simplify our code like this: void my_function(lutok::state& state, const int foo) { lutok::stack_cleaner cleaner(state); state.push_integer(foo); ... do something else in the state ... const int bar = state.to_integer(); if (bar != 3) throw std::runtime_error("Invalid data!"); } And we leave the boring task of determining when to actually call state.pop(1) to the compiler and the runtime environment. In this particular case, no matter how the my_function terminates, we ensure that the Lua stack will be left as the same size as it was before. But, as I said earlier, all this was just an introduction to the idea that made me write this post. When you declare an auto-cleaner object of any kind, be sure to give it a name. It has happened to me a few times already that I have written the following construct: lutok::stack_cleaner(state); ... which is syntactically correct, harmless and "looks good" if you don't look closely. The compiler will chew along just fine because, even though we are declaring an anonymous object, its constructor and destructor may be doing who-knows-what, so their code must be called and thus the "unused variable" warning cannot really be raised. However this does not give us the desired behavior. The cleaner object will be constructed and destructed in the same statement without having a chance to wrap any of the following code, because its scope is just the statement in which it was defined. In other words, the cleaner will have absolutely no effect on the rest of the function and thus will be useless. So, moral of the story: always give a name to your auto-cleaner objects so that their scope is correctly defined and their destructor is run when you actually expect: lutok::stack_cleaner ANY_NAME_HERE(state);
September 17, 2011
·
Tags:
<a href="/tags/cxx">cxx</a>, <a href="/tags/lua">lua</a>, <a href="/tags/lutok">lutok</a>
Continue reading (about
4 minutes)
Introducing Lutok: A lightweight C++ API for Lua
It has finally happened. Lutok is the result of what was promised in the "Splitting utils::lua from Kyua" web post. Quoting the project web page: Lutok provides thin C++ wrappers around the Lua C API to ease the interaction between C++ and Lua. These wrappers make intensive use of RAII to prevent resource leakage, expose C++-friendly data types, report errors by means of exceptions and ensure that the Lua stack is always left untouched in the face of errors. The library also provides a small subset of miscellaneous utility functions built on top of the wrappers. Lutok focuses on providing a clean and safe C++ interface; the drawback is that it is not suitable for performance-critical environments. In order to implement error-safe C++ wrappers on top of a Lua C binary library, Lutok adds several layers or abstraction and error checking that go against the original spirit of the Lua C API and thus degrade performance. Lutok was originally developed within Kyua but was later split into its own project to make it available to general developers.Coming up with a name for this project was quite an odyssey, and is what has delayed is release more than I wanted. My original candidate was "luawrap" which, although not very original, was to-the-point and easy to understand. Unfortunately, that name did not clear with the legal department and I had to propose several other names, some of which were not acceptable either. Eventually, I settled with "Lutok", which comes from "LUa TOolKit". At this point, the source tree of Lutok provides pretty much the same code as the utils::lua module of Kyua. While it may be enough to get you started, I'm pretty sure you will lack some functions in the state class. If that is the case, don't hesitate to file a bug report to let me know what is missing. In case you missed the link above, the project page is here: Lutok in Google Code.
September 15, 2011
·
Tags:
<a href="/tags/announce">announce</a>, <a href="/tags/cxx">cxx</a>, <a href="/tags/kyua">kyua</a>, <a href="/tags/lua">lua</a>, <a href="/tags/lutok">lutok</a>
Continue reading (about
2 minutes)
Featured software
Featured posts
- Fast machines, slow machines
- EndBASIC 0.10: Core language, evolved
- Farewell, Microsoft; hello, Snowflake!
- Rust is hard, yes, but does it matter?
- Rust traits and dependency injection
- A year on Windows: Introduction
- Always be quitting
- How does Google keep build times low?
- How does Google avoid clean builds?
- Unit-testing a console app (a text editor)
- More...
