Some obscure C features you might not know about
Date: 2018-01-25
Git: https://gitlab.com/mort96/blog/blob/published/content/00000-home/00010-obscure-c-features.md
I have been working on Snow, a unit testing library for C. I wanted to see how close I could come to making a DSL (domain specific language) with its own syntax and features, using only the C preprocessor and more obscure C features and GNU extensions. I will not go into detail about how Snow works unless it's directly relevant, so I recommend taking a quick look at the readme on the GitHub page.
Sending blocks as arguments to macros
Let's start with the trick that's probably both the most useful in everyday code, and the least technically complicated.
Originally, I defined macros like describe, subdesc, and it similar to
this:
#define describe(name, block) \
void test_##name() { \
/* some code, omitted for brevity */ \
block \
/* more code */ \
}
The intended use would then be like this:
describe(something, {
/* code */
});
The C preprocessor doesn't really understand the code; it only
copies and pastes strings around. It splits the string between the opening (
and the closing ) by comma; that means, in this case, something would be
sent in as the first argument, and { /* code */ } as the second argument
(pretend /* code */ is actual code; the preprocessor actually strips out
comments). The C preprocessor is smart enough to know that you might want to
pass function calls to macros, and function calls contain commas, so
parentheses will "guard" the commas they contain.
describe(something, foo(10, 20)) would therefore pass something as the
first argument, and foo(10, 20) as the second argument.
Now, we're not passing in function calls, but blocks. The preprocessor only
considers parentheses; braces { } or brackets [ ] don't guard their
contents. That means this call will fail:
describe(something, {
int a, b;
/* code */
});
The preprocessor will interpret something as the first argument, { int a as
the second argument, and b; /* code */ } as the third argument, but
describe only takes two arguments! The preprocessor will halt and show an
error message.
So, how do we fix this? Not being able to write commas outside of parentheses
in our blocks is quite the limitation. Not only does it prevent us from
declaring multiple variables in one statement, it also messes with array
declarations like int foo[] = { 10, 20, 30 };.
Well, the preprocessor supports variadic macros; macros which can take an
unlimited amount of arguments. The way they are implemented is that any
extra arguments (indicated by ... in the macro definition) are made available
through the __VA_ARGS__ identifier; __VA_ARGS__ is replaced with all the
extra arguments separated by commas. So, what happens if we define the macro
like this?
#define describe(name, ...) \
void test_##name() { \
/* some code, omitted for brevity */ \
__VA_ARGS__ \
/* more code */ \
}
Let's call describe like we did above:
describe(something, {
int a, b;
/* code */
});
Now, the arguments will be interpreted the same way as before; something will
be the first argument, { int a will be the second argument, and
b; /* code */ } will be the third. However, __VA_ARGS__ will be replaced by
the second and third argument with a comma inbetween, and together they
produce { int a, b; /* code */ }, just as we intended. The entire describe
call will be expanded into this (with added newlines and indentation for
clarity; the actual preprocessor would put it all on one line):
void test_something() {
/* some code, omitted for brevity */
{
int a, b;
/* code */
}
/* more code */
}
And just like that, we successfully passed a block of code, with unguarded commas, to a macro.
Credit for this solution goes to this stackoverflow answer.
Generic macros with _Generic
I wanted to be able to use one set of macros, asserteq and assertneq, to be
able to do most simple equality checks, instead of having to write
asserteq_str for strings, asserteq_int for integers, etc. The C11 standard
added the _Generic keyword, which sounds like it's perfect for that; given a
list of types and expressions, _Generic will choose the expression whose
associated type is compatible with a controlling expression. For example, this
code will print "I am an int":
_Generic(10,
int: printf("I am an int\n"),
char *: printf("I am a string\n")
);
By itself, _Generic isn't terribly useful, but it can be used to make
faux-generic function-like macros.
The cppreference.com page
uses the example of a generic cbrt (cube root) macro:
#define cbrt(x) _Generic((x), \
long double: cbrtl, \
float: cbrtf, \
default: cbrt)(x)
Calling cbrt on a long double will now call cbrtl, while calling cbrt on
a double will call the regular cbrt function, etc. Note that _Generic is
not part of the preprocessor; the preprocessor will just spit out the
_Generic syntax with x replaced with the macro's argument, and it's the
actual compiler's job to figure out what type the controlling expression is and
choose the appropriate expression.
I have a bunch of asserteq functions for the various types;
asserteq_ptr(void *a, void *b), asserteq_int(intmax_t a, intmax_t b),
asserteq_str(const char *a, const char *b), etc.
(In reality, the function signatures are a lot uglier, and they're prefixed
with _snow_, but for the sake of this article, I'll pretend they look like
void asserteq_<suffix>(<type> a, <type> b)).
At first glance, _Generic looks perfect for this use case; just define an
asserteq macro like this:
#define asserteq(a, b) _Generic((b), \
const char *: asserteq_str, \
char *: asserteq_str, \
void *: asserteq_ptr, \
int: asserteq_int)(a, b)
It's sadly not that simple. _Generic will match only specific types; int
matches only int, not long. void * matches void pointers, not any other
form of pointer. There's no way to say "match every pointer type", for example.
However, there is a default clause, just like in switch statements. My first
solution was to just pass anything not otherwise specified to asserteq_int,
and use _Pragma (like #pragma, but can be used inside macros) to ignore the
warnings:
#define asserteq(a, b) \
do { \
_Pragma("GCC diagnostic push") \
_Pragma("GCC diagnostic ignored \"-Wint-conversion\"") \
_Generic((b), \
const char *: asserteq_str, \
char *: asserteq_str, \
default: asserteq_int)(a, b) \
_Pragma("GCC diagnostic pop") \
} while (0)
That solution worked but it's not exactly nice. I assume it would eventually
break, either due to compiler optimizations or due to weird systems where an
intmax_t is smaller than a pointer or whatever. Luckily, the good people over
in ##C@freenode had an answer: subtracting a pointer from a pointer results
in a ptrdiff_t! That means we can nest _Generics, and appropriately choose
asserteq_int for any integer types, or asserteq_ptr for any pointer types:
#define asserteq(a, b) _Generic((b), \
const char *: asserteq_str, \
char *: asserteq_str, \
default: _Generic((b) - (b), \
ptrdiff_t: asserteq_ptr(a, b), \
default: asserteq_int(a, b)))(a, b)
Defer, label pointers, and goto *(void *)
I once saw a demonstration of Golang's defer statement, and fell in love. It immediately struck me as a much better way to clean up than relying solely on the try/catch stuff we've been used to ever since 1985. Naturally, I wanted to use that for tearing down test cases in Snow, but there's not exactly any obvious way to implement it in C.
For those unfamiliar with it, in Go, defer is basically a way to say, "run
this expression once the function returns". It works like a stack; when the
function returns, the most recently deferred expression will be executed
first, and the first deferred expression will be executed last. The beautiful
part is that even if the function returns early, either because some steps can
be skipped, or because something failed, all the appropriate deferred
expressions, and only the appropriate deferred expressions, will be executed.
Replace "function" with "test case", and it sounds perfect for tearing down
tests.
So, how would you implement that in C? Well, it turns out that GCC has two useful non-standard extensions (which are also supported by Clang by the way): local labels, and labels as values.
Local labels are basically regular labels which you can jump to with goto,
but instead of being global to the entire function, they're only available in
the block they're declared in. That's fairly straightforward. You declare that
a label should be block scoped by just putting __label__ label_name; at the
top of the block, and then you can use label_name: anywhere within the block
to actually create the label. A goto label_name from anywhere within the
block will then go to the label, as expected.
Labels as values is weirder. GCC adds a new unary && operator, which gets a
pointer to a label as a void *. Moreover, if you save that pointer in a
variable which is accessible outside the block, you can jump back in to that
block from outside of it, even though it's a local label. This will print
"hello" in an infinite loop:
{
void *somelabel;
{
__label__ lbl;
lbl:
somelabel = &&lbl;
printf("hello\n");
}
goto *somelabel;
}
Yes, the somelabel is a void *. Yes, we dereference somelabel to go to
it. I don't know how that works, but the important part is that it does. Other
than being dereferencable, the void * we get from the unary && works
exactly like any other void *, and can even be in an array. Knowing this,
implementing defer isn't too hard; here's a simplified implementation of the
it(description, block) macro (using the __VA_ARGS__ trick from before)
which describes one test case, and the defer(expr) macro which can be used
inside the it block:
#define it(description, ...) \
do { \
__label__ done_label; \
void *defer_labels[32]; \
int defer_count = 0; \
int run_defer = 0; \
__VA_ARGS__ \
done_label: \
run_defer = 1; \
if (defer_count > 0) { \
defer_count -= 1; \
goto *defer_labels[defer_count]; \
} \
} while (0)
#define defer(expr) \
do { \
__label__ lbl; \
lbl: \
if (run_defer) { \
expr; \
/* Go to the previous defer, or the end of the `it` block */ \
if (defer_count > 0) { \
defer_count -= 1; \
goto *defer_labels[defer_count]; \
} else { \
goto done_label; \
} \
} else { \
defer_labels[defer_count] = &&lbl; \
defer_count += 1; \
} \
} while (0)
That might not be the most understandable code you've ever seen, but let's break it down with an example.
it("whatever", {
printf("Hello World\n");
defer(printf("world\n"));
defer(printf("hello "));
});
Running that through the preprocessor, we get this code:
do {
__label__ done_label;
void *defer_labels[32];
int defer_count = 0;
int run_defer = 0;
{
printf("Hello World\n");
do {
__label__ lbl;
lbl:
if (run_defer) {
printf("world\n");
/* Go to the previous defer, or the end of the `it` block */
if (defer_count > 0) {
defer_count -= 1;
goto *defer_labels[defer_count];
} else {
goto done_label;
}
} else {
defer_labels[defer_count] = &&lbl;
defer_count += 1;
}
} while (0);
do {
__label__ lbl;
lbl:
if (run_defer) {
printf("hello ");
/* Go to the previous defer, or the end of the `it` block */
if (defer_count > 0) {
defer_count -= 1;
goto *defer_labels[defer_count];
} else {
goto done_label;
}
} else {
defer_labels[defer_count] = &&lbl;
defer_count += 1;
}
} while (0);
}
done_label:
run_defer = 1;
if (defer_count > 0) {
defer_count -= 1;
goto *defer_labels[defer_count];
}
} while (0);
That's still not extremely obvious on first sight, but it's at least more
obvious than staring at the macro definitions. The first time through,
run_defer is false, so both the defer blocks will just add their labels to
the defer_labels array and increment defer_count. Then, just through normal
execution (without any goto), we end up at the label called done_label,
where we set run_defer to true. Because defer_count is 2, we decrement
defer_count and jump to defer_labels[1], which is the last defer.
This time, because run_defer is true, we run the deferred expression
printf("hello "), decrement defer_count again, and jump to
defer_labels[0], which is the first defer.
The first defer runs its expression, printf("world\n"), but because
defer_count is now 0, we jump back to done_label. defer_count is of
course still 0, so we just exit the block.
The really nice thing about this system is that a failing assert can at any
time just say goto done_label, and only the expressions which were deferred
before the goto will be executed.
(Note: in the actual implementation in Snow, defer_labels is of course a
dynamically allocated array which is realloc'd when necessary. It's also
global to avoid an allocation and free for every single test case. I omitted
that part because it's not that relevant, and would've made the example code
unnecessarily complicated.)
Update: A bunch of people on Reddit and Hacker News have suggested ways to
accomplish this. I ended up using the __attribute__((constructor))
function attribute,
which makes a given function execute before the main function. Basically, each
describe creates a function called test_##name, and a constructor
function called _snow_constructor_##name whose only job is to add
test_##name to a global list of functions. Here's the code:
https://github.com/mortie/snow/blob/7ee25ebbf0edee519c6eb6d36b82d784b0fdcbfb/snow/snow.h#L393-L421
Automatically call all functions created by describe
The describe macro is meant to be used at the top level, outside of
functions, because it creates functions. It's basically just this:
#define describe(name, ...) \
void test_##name() { \
__VA_ARGS__ \
}
Calling describe(something, {}) will create a function called
test_something. Currently, that function has to be called manually, because
no other part of Snow knows what the function is named. If you have used the
describe macro to define the functions test_foo, test_bar, and
test_baz, the main function will look like this:
snow_main({
test_foo();
test_bar();
test_baz();
})
I would have loved it if snow_main could just know what functions are
declared by describe, and automatically call them. I will go over a couple of
ways I tried, which eventually turned out to not be possible, and then one way
which would definitely work, but which is a little too crazy, even for me.
Static array of function pointers
What if, instead of just declaring functions with describe, we also appended
them to an array of function pointers? What if snow.h contained code like
this:
void (*described_functions[512])();
#define describe(name, ...) \
void test_##name() { \
__VA_ARGS__ \
} \
described_functions[__COUNTER__] = &test_##name
__COUNTER__ is a special macro which starts at 0, and is incremented by one
every time it's referenced. That means that assuming nothing else uses
__COUNTER__, this solution would have worked, and would have been
relatively clean, if only it was valid syntax. Sadly, you can't set the value
of an index in an array like that in the top level in C, only inside functions.
Appending to a macro
What if we had a macro which we appended test_##name(); to every time a
function is declared by describe? It turns out that this is almost possible
using some obscure GCC extensions. I found this solution
on StackOverflow:
#define described_functions test_foo();
#pragma push_macro("described_functions")
#undef described_functions
#define described_functions _Pragma("pop_macro(\"described_functions\")") described_functions test_bar();
#pragma push_macro("described_functions")
#undef described_functions
#define described_functions _Pragma("pop_macro(\"described_functions\")") described_functions test_baz();
described_functions // expands to test_foo(); test_bar(); test_baz();
This is actually a way to append a string to a macro which works, at least in
GCC. Snow could have used that... except for one problem: you of course can't
use #define from within a macro, and we would have needed to do this from
within the describe macro. I have searched far and wide for a way, even a
weird GCC-specific possibly pragma-related way, to redefine a macro from within
another macro, but I haven't found anything. Close, but no cigar.
The way which actually works
I mentioned that there is actually one way to do it. Before I show you, I
need to cover dlopen and dlsym.
void *dlopen(const char *filename, int flags) opens a binary (usually a
shared object... usually), and returns a handle. Giving dlopen NULL as the
file name gives us a handle to the main program.
void *dlsym(void *handle, const char *symbol) returns a pointer to a symbol
(for example a function) in the binary which handle refers to.
We can use dlopen and dlsym like this:
#include <stdio.h>
#include <dlfcn.h>
void foo() {
printf("hello world\n");
}
int main() {
void *h = dlopen(NULL, RTLD_LAZY);
void *fptr = dlsym(h, "foo");
void (*f)() = fptr;
f();
dlclose(h);
}
Compile that code with gcc -Wl,--export-dynamic -ldl -o something something.c,
and run ./something, and you'll see it print hello world to the terminal.
That means we can actually call functions dynamically based on an arbitrary
string at runtime. (The -Wl,--export-dynamic is necessary to tell the linker
to export the symbols, such that they're available to us through dlsym).
Being able to run functions based on a runtime C string, combined with our
friend __COUNTER__, opens up some interesting possibilities. We could write a
program like this:
#include <stdio.h>
#include <dlfcn.h>
/* Annoyingly, the concat_ and concat macros are necessary to
* be able to use __COUNTER__ in an identifier name */
#define concat_(a, b) a ## b
#define concat(a, b) concat_(a, b)
#define describe(...) \
void concat(test_, __COUNTER__)() { \
__VA_ARGS__ \
}
describe({
printf("Hello from function 0\n");
})
describe({
printf("Hi from function 1\n");
})
int main() {
void *h = dlopen(NULL, RTLD_LAZY);
char symbol[32] = { '\0' };
for (int i = 0; i < __COUNTER__; ++i) {
snprintf(symbol, 31, "test_%i", i);
void *fptr = dlsym(h, symbol);
void (*f)() = fptr;
f();
}
dlclose(h);
}
Run that through the preprocessor, and we get:
void test_0() {
{ printf("Hello from function 0\n"); }
}
void test_1() {
{ printf("Hi from function 1\n"); }
}
int main() {
void *h = dlopen(NULL, RTLD_LAZY);
char symbol[32] = { '\0' };
for (int i = 0; i < 2; ++i) {
snprintf(symbol, 31, "test_%i", i);
void *fptr = dlsym(h, symbol);
void (*f)() = fptr;
f();
}
dlclose(h);
}
That for loop in our main function will first call test_0(), then test_1().
I hope you understand why even though this technically works, it's not exactly something I want to include in Snow ;)
