Shtk - Julio Merino (jmmv.dev)

End-to-end tool testing with Bazel and shtk

If you use Bazel, your project is of moderate size. And because your project is of moderate size, it almost-certainly builds one or more binaries, at least one of which is a CLI tool. But let’s face it: you don’t have end-to-end testing for those tools, do you? I’m sure you have split the binary’s main function into its own file so that the rest of the tool can be put in a library, and I’m extra-sure that you have unit tests for such library. But… those tests do little to verify the functionality and quality of the tool as experienced by the end user. Consider: What exactly does the tool print to the console on success? Does it show errors nicely when they happen, or does it dump internal stack traces? How does it handle unknown flags or bad arguments? Is the built-in help message nicely rendered when your terminal is really wide? What if the terminal is narrow? You must write end-to-end tests for your tools but, usually, that isn’t easy to do. Until today. Combining shtk with Bazel via the new rules_shtk ruleset makes it trivial to write tests that verify the behavior of your CLI tools—no matter what language they are written in—and in this article I’m going to show you how.

November 4, 2023 · Tags: <a href="/tags/bazel">bazel</a>, <a href="/tags/blogsystem5">blogsystem5</a>, <a href="/tags/shell">shell</a>, <a href="/tags/shtk">shtk</a>, <a href="/tags/testing">testing</a>
Continue reading (about 7 minutes)

Unit-testing shell scripts and tools with shtk

While working on this static blog a few days ago, I made a change to its templates that warranted an automated test. I could have written a trivial shell script to do it, but instead I reached out for shtk’s unit-testing module. I tweeted about it right away to just say that you can, in fact, write tests in shell because lots of developers are skeptical about any script longer than 10 lines of code. Interestingly, this reply came through: a pointer to a contemporary, under-development library for writing tests in Bash. Which made me think: “Hey, I had already done that years ago… but nobody knows about it. Gotta fix that with a blog post!” But first, I had to bring shtk back from its ashes because I had not touched it for more than 6 years and it wasn’t read for show and tell. So I did something that I wanted to do back in the day but never did: I put together a website for shtk to host its reference manual and I fixed a few obvious rough edges. With those tweaks out of the way, we come to this article. In here, I want to show you how writing decent tests in shell is entirely possible and how shtk’s testing platform provides unique features to do integration testing of CLI apps written in any language.

October 11, 2023 · Tags: <a href="/tags/shell">shell</a>, <a href="/tags/shtk">shtk</a>, <a href="/tags/testing">testing</a>
Continue reading (about 12 minutes)

Shell readability: function parameters

The shell supports defining functions, which, as we learned in the previous post, you should embrace and use. Unfortunately, they are fairly primitive and their use can, paradoxically, introduce other readability problems. One specific problem is that function parameters are numbered, not named, so the risk of cryptic code is high. Let’s see why this is a problem.

March 2, 2018 · Tags: <a href="/tags/readability">readability</a>, <a href="/tags/shell">shell</a>, <a href="/tags/shtk">shtk</a>
Continue reading (about 3 minutes)

Shell readability: main

Our team develops Bazel, a Java-based tool. We do have, however, a significant amount of shell scripting. The percentage is small at only 3.6% of our codebase… but given the size of our project, that’s about 130,000 lines—a lot, really. Pretty much nobody likes writing these integration tests in shell. Leaving aside that our infrastructure is clunky, the real problem is that the team at large is not familiar with writing shell per se. Few people are these days actually.

February 26, 2018 · Tags: <a href="/tags/readability">readability</a>, <a href="/tags/shell">shell</a>, <a href="/tags/shtk">shtk</a>
Continue reading (about 2 minutes)

shtk 1.6 is now available

The Shell Toolkit, or shtk for short, is a little project I introduced back in August of 2008 to support other tools such as sysbuild and sysupgrade. Since that time, the project has seen little activity because it did not have much to offer and because shtk's public interface was not documented (hence making it impossible for developers to get started with shtk). Well, both are changing today with the brand-new release of shtk 1.6:

November 17, 2014 · Tags: <a href="/tags/announce">announce</a>, <a href="/tags/release">release</a>, <a href="/tags/shtk">shtk</a>
Continue reading (about 2 minutes)

'Hello GitHub!' say shtk, sysbuild and sysupgrade

Over a year ago, I developed two command line utilities for NetBSD (sysbuild and sysupgrade) and a supporting library for the two tools (shtk). These were all introduced in their corresponding blog posts —Introducing sysbuild for NetBSD, Introducing sysupgrade for NetBSD and Introducing shtk— and since then I have heard good comments about them. About a couple of weeks ago, I started working on the much-needed rewrite of pkg_comp and a supporting standalone tool. I was using the same development methodology as with the other three projects: putting all the code in pkgsrc and implementing the build system from the package's Makefile. Along the way, this became increasingly annoying to the point where I could not stand it any more. pkgsrc is a packaging system, not a development platform. Developing a project within it is difficult due to the indirections between the outer Makefile and the work directory, and the expected workflow of working with packages. A separate issue is that having the source code in pkgsrc prevents the distribution of the packages as standalone tools for third-party operating systems. (Mind you: I've recently started building NetBSD from Linux and FreeBSD and missed sysbuild dearly.) It's hard and ugly (but not impossible) to generate tarballs for the sources in pkgsrc that can later be hosted elsewhere... and, even if doing that, using pkgsrc as the master tree for the code would seem backwards.

July 28, 2013 · Tags: <a href="/tags/github">github</a>, <a href="/tags/release">release</a>, <a href="/tags/shtk">shtk</a>, <a href="/tags/sysbuild">sysbuild</a>, <a href="/tags/sysupgrade">sysupgrade</a>
Continue reading (about 3 minutes)

Introducing shtk

Have you ever wanted to have a collection of ready-to-use modules for shell scripts? I have, particularly because I keep reimplementing the same functions over and over and over and over again whenever I write non-trivial shell scripts, and I'm tired of doing so. That's why I have just abstracted all the common code in the aforementioned tools and put it into a new package called the "Shell Toolkit", or shtk for short. Yeah, this name sounds very pretentious but, really, I don't intend this to be anything big. The only thing I want to do is simplify my life when implementing shell scripts, and hope that other people might find the modules useful. So far, I have taken the generic (and common!) code from sysbuild and sysupgrade, reconciled a few tiny divergences, and moved it into this new shtk package. In reality, writing something like shtk is sin-borderline. I really should not be using shell scripting for the kind of tools I am implementing (they deserve better data structures and better error checking than what shell provides, for example). However, shell scripting is incredible convenient to get reasonably-good implementations of such tools with minimal effort, and is the only scripting language available in NetBSD's base system. (Yes, yes, there is Lua, but my limited knowledge of Lua would not let me write these tools in any decent manner nor in any reasonable time.) So, what's in shtk? There are a few functions to deal with command lines (error/warning reporting and such things), some trivial stuff to deal with lists, a bunch of code to interact with cvs and, what I like the most, a module to implement configuration files with some kind of key/value validation. At the moment, shtk can only be found in pkgsrc under pkgsrc/devel/shtk and I don't currently have any plans to make it more widely available. If there are enough people interested in that with real needs, I could reconsider, but the maintenance effort would be non-trivial. To showcase the features of shtk, I have updated the sysbuild and sysupgrade packages to depend on this new toolkit while at the same time dropping all this duplicate supporting code. It's a good thing that I wrote exhaustive tests for all possible code paths, because the migration from the built-in modules to shtk was riddled with subtleties that would have impacted end users otherwise. Now... time to really consider taking the task of rewriting pkg_comp in a more maintainable style so that I can add the features I have wished for for many years (like OS X support).

August 15, 2012 · Tags: <a href="/tags/announce">announce</a>, <a href="/tags/netbsd">netbsd</a>, <a href="/tags/shell">shell</a>, <a href="/tags/shtk">shtk</a>, <a href="/tags/sysbuild">sysbuild</a>, <a href="/tags/sysupgrade">sysupgrade</a>
Continue reading (about 3 minutes)

Posts: Shtk