In Unix-like systems, “everything is a file and a file is defined as a byte stream you can open, read from, write to, and ultimately close”… right? Right? Well, not quite. It’s better to say file descriptors provide access to almost every system that the kernel provides, but not that they can all be manipulated with the same quartet of system calls, nor that they all behave as byte streams.

Because you see: network connections are manipulated via file descriptors indeed, but you don’t open them: you bind, listen/accept and/or connect to them. And then you don’t read from and write to network connections: you somehow send to and recv from them. Device drivers are similar: yes, hardware devices are represented as “virtual files” in the /dev hierarchy and many support read and write… but these two system calls are not sufficient to access the breath of functionality that the hardware drivers provide. No, you need ioctl.

ioctl is the poster child of the system call that breaks Unix’s “everything is a file” paradigm. ioctl is the API that allows out-of-band communication with the kernel side of an open file descriptor. To see cool examples, refer back to my previous article where I demonstrated how to drive graphics from the console without X11: in that post, we had to open the console device, but then we had to use ioctl to obtain the properties of the framebuffer, and then we had to mmap the device’s content for direct access: no reads nor writes involved.

All the code I showed you in that earlier post was written in C to keep the graphics article to-the-point, but the code I’m really working on is part of EndBASIC, and thus it is all Rust. And the thing is, ioctls are not easy to issue from Rust. In fact, after 7 years of Rust-ing, it’s the first time I’ve had to reach for unsafe code blocks, and there was no good documentation on how to deal with ioctl. So this posts aims to fix that by presenting what ways there are to call ioctls from Rust… and, of course, diving a bit deeper into what ioctls actually are.

Our target

For all examples below, I’ll be using a relatively simple ioctl from NetBSD’s wsdisplay(4) driver. This API is available via the console device file, typically /dev/ttyE0, and is named WSDISPLAYIO_GINFO. Here is what the manual page has to say:

The following ioctl(2) calls are provided by the wsdisplay driver or by
devices which use it.  Their definitions are found in
<dev/wscons/wsconsio.h>.

...

    WSDISPLAYIO_GINFO (struct wsdisplay_fbinfo)
        Retrieve basic information about a framebuffer display.
        The returned structure is as follows:
    
            struct wsdisplay_fbinfo {
                    u_int   height;
                    u_int   width;
                    u_int   depth;
                    u_int   cmsize;
            };
    
        The height and width members are counted in pixels.  The
        depth member indicates the number of bits per pixel, and
        cmsize indicates the number of color map entries accessible
        through WSDISPLAYIO_GETCMAP and WSDISPLAYIO_PUTCMAP.  This
        call is likely to be unavailable on text-only displays.

Calling this API from a C program would be trivial and look like this:

#include <dev/wscons/wsconsio.h>
#include <ioctl.h>

// ... open `/dev/ttyE0` as fd ...

struct wsdisplay_fbinfo fbi;
if (ioctl(fd, WSDISPLAYIO_GINFO, &fbi) == -1) {
    // Handle error.
}

// fbi now contains the data returned by the kernel.

The reason I’m picking WSDISPLAYIO_GINFO specifically to talk about ioctls in this article is three-fold:

it returns a small structure with platform-dependent integers, so the sample code in the article will be relatively short;
it relies on platform-specific integer types, so we’ll have to account for that in Rust; and
it is “rare enough” (we are talking about NetBSD after all) that it is not going to be supported by any of the common Rust crates like libc or nix, so we’ll have to do extra work to call it.

What is an ioctl anyway?

The manual page helpfully provides us a copy of the data structure returned by the ioctl, and the BSD manual pages are typically awesome, but it’s worth double-checking that the code snippet actually matches the code it documents. Peeking into /usr/include/dev/wscons/wsconsio.h as the manual page directs us, we find:

/* Basic display information.  Not applicable to all display types. */
struct wsdisplay_fbinfo {
    u_int   height;             /* height in pixels */
    u_int   width;              /* width in pixels */
    u_int   depth;              /* bits per pixel */
    u_int   cmsize;             /* color map size (entries) */
};
#define WSDISPLAYIO_GINFO _IOR('W', 65, struct wsdisplay_fbinfo)

OK, great, the wsdisplay_fbinfo structure perfectly aligns with the manual page contents. But what’s more interesting is the #define, which says that WSDISPLAYIO_GINFO is:

an ioctl that reads from the kernel (_IOR),
that invokes function number 65 from the W class (which probably stands for “the Wscons device driver”), and
that places the returned data into a structure of type wsdisplay_fbinfo (not to be confused with wsdisplayio_fbinfo).

In a way, this is just like any other function or system call, except that it’s not defined as such and is instead funneled through a single API. ioctl is, therefore, “just” a grab bag of arbitrary functionality, and what can be invoked on a given file descriptor depends on what the file descriptor represents.

The reasons for this design are historical and, of course, there could have been other options.

For example: you know how regular files have an internal structure, right? The vast majority of file formats out there contain a header, which then specifies various sections within the file, which then contain data. The same could have been done with device drivers: their virtual files could have predefined some internal format such that, e.g. the wsdisplay_fbinfo structure always appeared at offset 0x1000 of the virtual file. read and write would have been sufficient for this design, albeit you’d almost-certainly wanted to combine it with mmap for more efficient access.

Or another example: device drivers could have used an RPC-like mechanism where each write to the file descriptor is a “message” that requests a specific function, and that the kernel responds to with an answer. read and write would have been sufficient for this design.

Or yet another example: the requests to the device driver could have been intermixed with the data, such that if the data contained a specific sequence, the kernel would invoke a function instead of processing data. Sounds crazy, right? But that’s what pseudo-terminals do: all those control sequences to change colors and the like are telling the terminal driver to do something special.

In any case, these are all alternate designs and… I’m sure they all live in some form or another in current systems. There is no consistency in how /dev pseudo-files expose their behavior, and ioctls are just one of the options we have to deal with. So without further ado, let’s look at three different ways of calling these services from Rust.

Option 1: nix

The first option to call an ioctl from Rust is to leverage the neat nix crate, which provides idiomatic access to Unix primitives. This crate is not to be confused with NixOS, with which it has no relation.

To use nix to invoke ioctls, we need to do two things.

First, we need to define the data structure used by the ioctl. In C, we would just #include <dev/wscons/wsconsio.h>, but in Rust we don’t have access to the C-style headers. Instead, we have to do extra work to define the same memory layout of the C wsdisplay_fbinfo structure, but in Rust:

use std::ffi::c_uint;

#[repr(C)]
struct WsDisplayFbInfo {
    height: c_uint,
    width: c_uint,
    depth: c_uint,
    cmsize: c_uint,
}

It is very important to declare the structure as having a C representation so that its memory layout matches what the C compiler produces for the same structure. The kernel expects C semantics in its system call boundary, and we must adhere to that. Additionally, we must ensure that the types of each field match the C definitions. Rust only has fixed-size integer types like i16 and u32, but C provides platform-dependent integer types like int or unsigned long and these are sometimes used in public kernel interfaces (a mistake, if you ask me). Fear not, though: the std::ffi module provides aliases for those C types.

And second, we have to do something unique to the nix crate: we have to define a wrapper function for the ioctl so that we can invoke the ioctl as if it were any other function. nix makes this very easy by providing macros that mimic the syntax of the C #define we saw earlier on:

use nix::ioctl_read;

ioctl_read!(wsdisplayio_ginfo, b'W', 65, WsDisplayFbInfo);

And with that, we are ready to put everything together in a fully-fledged program:

use std::ffi::c_uint;
use std::io;
use std::mem;
use std::os::fd::{AsRawFd, FromRawFd, OwnedFd};

use nix::fcntl;
use nix::ioctl_read;
use nix::sys::stat;

#[repr(C)]
#[derive(Debug)]
#[allow(unused)]
struct WsDisplayFbInfo {
    height: c_uint,
    width: c_uint,
    depth: c_uint,
    cmsize: c_uint,
}

ioctl_read!(wsdisplayio_ginfo, b'W', 65, WsDisplayFbInfo);

fn main() -> io::Result<()> {
    let mut oflag = fcntl::OFlag::empty();
    oflag.insert(fcntl::OFlag::O_RDWR);
    oflag.insert(fcntl::OFlag::O_NONBLOCK);
    oflag.insert(fcntl::OFlag::O_EXCL);

    let fd = {
        let raw =
            fcntl::open("/dev/ttyE0", oflag, stat::Mode::empty())?;
        unsafe { OwnedFd::from_raw_fd(raw) }
    };

    let mut fbi: WsDisplayFbInfo;
    unsafe {
        fbi = mem::zeroed();
        wsdisplayio_ginfo(fd.as_raw_fd(), &mut fbi).unwrap();
    }
    eprintln!("fbinfo: {:?}", fbi);

    Ok(())
}

There is one surprising detail in this code though: if we went through the hassle of defining a wrapper function for WSDISPLAYIO_GINFO via the idiomatic nix crate, and idiomatic nix usage doesn’t require unsafe blocks… why did we have to wrap the call to wsdisplayio_ginfo in an unsafe block? The reason may be that ioctl can do whatever to the running process and Rust needs to be over-conservative.

In any case, the above is clean and it works. But… using nix comes with a cost:

$ cargo build
   Compiling libc v0.2.169
   Compiling cfg_aliases v0.2.1
   Compiling bitflags v2.8.0
   Compiling cfg-if v1.0.0
   Compiling nix v0.29.0
   Compiling ioctls-rust-nix v0.1.0 (/home/jmmv/os/homepage/static/src/ioctls-rust/nix)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 1.71s
$ █

We have pulled 5 crates into the project just to open a file and invoke an ioctl. Not a huge deal in this day and age but… this contributes to the perception that the Rust ecosystem is a mess of bloated dependencies. Can we do differently?

Option 2: libc

What if we bypassed nix altogether and invoked libc directly? After all, we can see that nix depends on libc anyway, so we might as well use it at the expense of losing nix’s idiomatic representation of Unix’s interfaces.

Sure, we can do that: we can invoke the libc::ioctl function directly, which has this prototype:

pub fn ioctl(fd: c_int, request: Ioctl, ...) -> c_int;

Alright then: we need a file descriptor as the first argument, which we have. And then we need an Ioctl as the second argument, which we… wait, what is this Ioctl type? If we look for its definition in the libc source code, we find that Ioctl is an alias for an integer type (unsigned long or int depending on the platform), and this matches the C definition of ioctl. OK, nothing special.

But then… what do we pass in this second argument? If we were writing C, we would use the WSDISPLAY_GINFO constant, but we don’t have that in Rust because we don’t get access to the C header files. So what is WSDISPLAY_GINFO? Remember that we previously saw that it is defined as such:

#define WSDISPLAYIO_GINFO _IOR('W', 65, struct wsdisplay_fbinfo)

… which doesn’t help us much at this point. But we can chase the definition of _IOR, end up in /usr/include/sys/ioccom.h, and see:

#define _IOC(inout, group, num, len) \
    ((inout) | (((len) & IOCPARM_MASK) << IOCPARM_SHIFT) | \
    ((group) << IOCGROUP_SHIFT) | (num))
#define _IOR(g,n,t)     _IOC(IOC_OUT,   (g), (n), sizeof(t))

Ugh. We are combining the various arguments to _IOR into a number. This is hard to decipher by just reading the code, so we can ask the compiler to tell us the actual value of the constant instead:

#include <dev/wscons/wsconsio.h>
#include <stdio.h>

int main(void) {
    printf("%x\n", WSDISPLAYIO_GINFO);
}

And if we run the program, we get that WSDISPLAYIO_GINFO is 0x40105741. Knowing that, it’s an SMOP to call the ioctl using the libc crate alone:

use std::ffi::{c_char, c_uint};
use std::io;
use std::mem;
use std::os::fd::{AsRawFd, FromRawFd, OwnedFd};

const WSDISPLAYIO_GINFO: u64 = 0x40105741;

#[repr(C)]
#[derive(Debug)]
#[allow(unused)]
struct WsDisplayFbInfo {
    height: c_uint,
    width: c_uint,
    depth: c_uint,
    cmsize: c_uint,
}

fn main() -> io::Result<()> {
    let fd = {
        let result = unsafe {
            libc::open(
                "/dev/ttyE0\0".as_ptr() as *const c_char,
                libc::O_RDWR | libc::O_NONBLOCK | libc::O_EXCL,
                0,
            )
        };
        if result == -1 {
            return Err(io::Error::last_os_error());
        }
        unsafe { OwnedFd::from_raw_fd(result) }
    };

    let mut fbi: WsDisplayFbInfo;
    unsafe {
        fbi = mem::zeroed();
        let result = libc::ioctl(
            fd.as_raw_fd(),
            WSDISPLAYIO_GINFO,
            &mut fbi as *mut WsDisplayFbInfo,
        );
        if result == -1 {
            return Err(io::Error::last_os_error());
        }
    }
    eprintln!("fbinfo: {:?}", fbi);

    Ok(())
}

As you can see from this code snippet, we also have to define the WsdisplayFbInfo structure to match the kernel’s, so avoiding nix didn’t really make things simpler for us—and in fact, it made them uglier because now we have to deal with libc’s oddities like raw C strings, global errno values, and an opaque constant for the ioctl number. Not great.

Option 3: FFI

Which makes one wonder… can we avoid replicating the C interfaces in Rust and instead leverage the system-provided header files? Yes we can. Instead of trying to invoke the ioctls from Rust, we can invoke them via some custom C glue code. Rust is going to call into the system-provided libc anyway when we invoke a system call, so we might as well switch to C a bit “earlier”.

The idea in this case, as in any other computing problem, is to add a layer of abstraction: instead of dealing with the kernel-defined data structures from Rust, we define our own structures and APIs to detach the Rust world from the C world. Here, look:

#include <dev/wscons/wsconsio.h>
#include <sys/ioctl.h>

struct my_ginfo {
    unsigned int height;
    unsigned int width;
    unsigned int depth;
};

int get_ginfo(int fd, struct my_ginfo* gi) {
    struct wsdisplay_fbinfo fbi;

    int result = ioctl(fd, WSDISPLAYIO_GINFO, &fbi);
    if (result == -1) {
        return result;
    }

    gi->height = fbi.height;
    gi->width = fbi.width;
    gi->depth = fbi.depth;

    return 0;
}

We start by declaring our own version of wsdisplay_ginfo, which I’ve called my_ginfo, that only includes the few fields we want to propagate to Rust. Yes, in this example the indirection is utterly pointless because we go from 4 to 3 fields so we haven’t made our lives easier, but there are ioctls that return larger structures from which we might only need a few values. Then we define a trivial function to wrap the ioctl and transform its return value into our own structure.

Then, we go to Rust, re-define our my_ginfo structure as MyGinfo (both of which we fully control so we can easily verify that they match) and we call the wrapping function:

use std::ffi::{c_char, c_int, c_uint};
use std::io;
use std::mem;
use std::os::fd::{AsRawFd, FromRawFd, OwnedFd};

#[repr(C)]
#[derive(Debug)]
#[allow(unused)]
struct MyGInfo {
    height: c_uint,
    width: c_uint,
    depth: c_uint,
}

extern "C" {
    fn get_ginfo(fd: c_int, gi: *mut MyGInfo) -> c_int;
}

fn main() -> io::Result<()> {
    let fd = {
        let result = unsafe {
            libc::open(
                "/dev/ttyE0\0".as_ptr() as *const c_char,
                libc::O_RDWR | libc::O_NONBLOCK | libc::O_EXCL,
                0,
            )
        };
        if result == -1 {
            return Err(io::Error::last_os_error());
        }
        unsafe { OwnedFd::from_raw_fd(result) }
    };

    let mut gi: MyGInfo;
    unsafe {
        gi = mem::zeroed();
        let result = get_ginfo(fd.as_raw_fd(), &mut gi as *mut MyGInfo);
        if result == -1 {
            return Err(io::Error::last_os_error());
        }
    }
    eprintln!("my_ginfo: {:?}", gi);

    Ok(())
}

The trick now is to link the C code with the Rust code together, and for that, we create a build.rs script. In here, we leverage the cc crate to put things together, which is an extra dependency that is only used at build time:

fn main() {
    if cfg!(target_os = "netbsd") {
        println!("cargo::rerun-if-changed=src/ffi.c");
        cc::Build::new().file("src/ffi.c").compile("ffi");
    } else {
        println!("cargo::rerun-if-changed=src/dummy.c");
        cc::Build::new().file("src/dummy.c").compile("ffi");
    }
}

And with that, we are done.

Which option wins?

Well, let’s see:

$ cargo build --release
   ...
   Compiling ioctls-rust-ffi v0.1.0 (/home/jmmv/ioctls-rust/ffi)
   Compiling ioctls-rust-libc v0.1.0 (/home/jmmv/ioctls-rust/libc)
   Compiling ioctls-rust-nix v0.1.0 (/home/jmmv/ioctls-rust/nix)
    Finished `release` profile [optimized] target(s) in 1.89s
$ ls -lh target/release/ioctls-rust-* | grep -v \\.d
-rwxr-xr-x 2 jmmv jmmv 455K Feb 12 08:04 target/release/ioctls-rust-ffi
-rwxr-xr-x 2 jmmv jmmv 455K Feb 12 08:04 target/release/ioctls-rust-libc
-rwxr-xr-x 2 jmmv jmmv 464K Feb 12 08:04 target/release/ioctls-rust-nix
$ strip -s target/release/ioctls-rust-*
$ ls -lh target/release/ioctls-rust-* | grep -v \\.d
-rwxr-xr-x 2 jmmv jmmv 353K Feb 12 08:05 target/release/ioctls-rust-ffi
-rwxr-xr-x 2 jmmv jmmv 353K Feb 12 08:05 target/release/ioctls-rust-libc
-rwxr-xr-x 2 jmmv jmmv 358K Feb 12 08:05 target/release/ioctls-rust-nix
$ █

From a binary size perspective, there are no meaningful differences. As expected, the usage of the nix crate results in slightly more code than the other alternatives because nix has to do extra work to translate global errno values into Rust Result types and the like. But the libc and FFI alternatives seem identical.

At runtime, however, we should expect the FFI option to perform a teeny tiny bit worse (though good luck measuring that) than the libc option because we have to convert between the kernel structure and our own structure in the happy path… all for dubious benefit.

All in all, I’ll take option 1. I do not like the fact of having to manually replicate the kernel structures in my own code so, if I had the time, I’d try to upstream the definitions to the well-tested libc crate or write another reusable crate with just those. But, barring that, the idiomatic nix interfaces make calling Unix primitives a breeze.

ioctls from Rust