home
projects
blurbs

Some obscure C features you might not know about

Date: 2018-01-25
Git: https://gitlab.com/mort96/blog/blob/published/content/00000-home/00010-obscure-c-features.md

I have been working on Snow, a unit testing library for C. I wanted to see how close I could come to making a DSL (domain specific language) with its own syntax and features, using only the C preprocessor and more obscure C features and GNU extensions. I will not go into detail about how Snow works unless it's directly relevant, so I recommend taking a quick look at the readme on the GitHub page.

Sending blocks as arguments to macros

Let's start with the trick that's probably both the most useful in everyday code, and the least technically complicated.

Originally, I defined macros like describe, subdesc, and it similar to this:

#define describe(name, block) \
	void test_##name() { \
		/* some code, omitted for brevity */ \
		block \
		/* more code */ \
	}

The intended use would then be like this:

describe(something, {
	/* code */
});

The C preprocessor doesn't really understand the code; it only copies and pastes strings around. It splits the string between the opening ( and the closing ) by comma; that means, in this case, something would be sent in as the first argument, and { /* code */ } as the second argument (pretend /* code */ is actual code; the preprocessor actually strips out comments). The C preprocessor is smart enough to know that you might want to pass function calls to macros, and function calls contain commas, so parentheses will "guard" the commas they contain. describe(something, foo(10, 20)) would therefore pass something as the first argument, and foo(10, 20) as the second argument.

Now, we're not passing in function calls, but blocks. The preprocessor only considers parentheses; braces { } or brackets [ ] don't guard their contents. That means this call will fail:

describe(something, {
	int a, b;
	/* code */
});

The preprocessor will interpret something as the first argument, { int a as the second argument, and b; /* code */ } as the third argument, but describe only takes two arguments! The preprocessor will halt and show an error message.

So, how do we fix this? Not being able to write commas outside of parentheses in our blocks is quite the limitation. Not only does it prevent us from declaring multiple variables in one statement, it also messes with array declarations like int foo[] = { 10, 20, 30 };.

Well, the preprocessor supports variadic macros; macros which can take an unlimited amount of arguments. The way they are implemented is that any extra arguments (indicated by ... in the macro definition) are made available through the __VA_ARGS__ identifier; __VA_ARGS__ is replaced with all the extra arguments separated by commas. So, what happens if we define the macro like this?

#define describe(name, ...) \
	void test_##name() { \
		/* some code, omitted for brevity */ \
		__VA_ARGS__ \
		/* more code */ \
	}

Let's call describe like we did above:

describe(something, {
	int a, b;
	/* code */
});

Now, the arguments will be interpreted the same way as before; something will be the first argument, { int a will be the second argument, and b; /* code */ } will be the third. However, __VA_ARGS__ will be replaced by the second and third argument with a comma inbetween, and together they produce { int a, b; /* code */ }, just as we intended. The entire describe call will be expanded into this (with added newlines and indentation for clarity; the actual preprocessor would put it all on one line):

void test_something() {
	/* some code, omitted for brevity */
	{
		int a, b;
		/* code */
	}
	/* more code */
}

And just like that, we successfully passed a block of code, with unguarded commas, to a macro.

Credit for this solution goes to this stackoverflow answer.

Generic macros with _Generic

I wanted to be able to use one set of macros, asserteq and assertneq, to be able to do most simple equality checks, instead of having to write asserteq_str for strings, asserteq_int for integers, etc. The C11 standard added the _Generic keyword, which sounds like it's perfect for that; given a list of types and expressions, _Generic will choose the expression whose associated type is compatible with a controlling expression. For example, this code will print "I am an int":

_Generic(10,
	int: printf("I am an int\n"),
	char *: printf("I am a string\n")
);

By itself, _Generic isn't terribly useful, but it can be used to make faux-generic function-like macros. The cppreference.com page uses the example of a generic cbrt (cube root) macro:

#define cbrt(x) _Generic((x), \
	long double: cbrtl, \
	float: cbrtf, \
	default: cbrt)(x)

Calling cbrt on a long double will now call cbrtl, while calling cbrt on a double will call the regular cbrt function, etc. Note that _Generic is not part of the preprocessor; the preprocessor will just spit out the _Generic syntax with x replaced with the macro's argument, and it's the actual compiler's job to figure out what type the controlling expression is and choose the appropriate expression.

I have a bunch of asserteq functions for the various types; asserteq_ptr(void *a, void *b), asserteq_int(intmax_t a, intmax_t b), asserteq_str(const char *a, const char *b), etc. (In reality, the function signatures are a lot uglier, and they're prefixed with _snow_, but for the sake of this article, I'll pretend they look like void asserteq_<suffix>(<type> a, <type> b)).

At first glance, _Generic looks perfect for this use case; just define an asserteq macro like this:

#define asserteq(a, b) _Generic((b), \
	const char *: asserteq_str, \
	char *: asserteq_str, \
	void *: asserteq_ptr, \
	int: asserteq_int)(a, b)

It's sadly not that simple. _Generic will match only specific types; int matches only int, not long. void * matches void pointers, not any other form of pointer. There's no way to say "match every pointer type", for example.

However, there is a default clause, just like in switch statements. My first solution was to just pass anything not otherwise specified to asserteq_int, and use _Pragma (like #pragma, but can be used inside macros) to ignore the warnings:

#define asserteq(a, b) \
	do { \
		_Pragma("GCC diagnostic push") \
		_Pragma("GCC diagnostic ignored \"-Wint-conversion\"") \
		_Generic((b), \
			const char *: asserteq_str, \
			char *: asserteq_str, \
			default: asserteq_int)(a, b) \
		_Pragma("GCC diagnostic pop") \
	} while (0)

That solution worked but it's not exactly nice. I assume it would eventually break, either due to compiler optimizations or due to weird systems where an intmax_t is smaller than a pointer or whatever. Luckily, the good people over in ##C@freenode had an answer: subtracting a pointer from a pointer results in a ptrdiff_t! That means we can nest _Generics, and appropriately choose asserteq_int for any integer types, or asserteq_ptr for any pointer types:

#define asserteq(a, b) _Generic((b), \
	const char *: asserteq_str, \
	char *: asserteq_str, \
	default: _Generic((b) - (b), \
		ptrdiff_t: asserteq_ptr(a, b), \
		default: asserteq_int(a, b)))(a, b)

Defer, label pointers, and goto *(void *)

I once saw a demonstration of Golang's defer statement, and fell in love. It immediately struck me as a much better way to clean up than relying solely on the try/catch stuff we've been used to ever since 1985. Naturally, I wanted to use that for tearing down test cases in Snow, but there's not exactly any obvious way to implement it in C.

For those unfamiliar with it, in Go, defer is basically a way to say, "run this expression once the function returns". It works like a stack; when the function returns, the most recently deferred expression will be executed first, and the first deferred expression will be executed last. The beautiful part is that even if the function returns early, either because some steps can be skipped, or because something failed, all the appropriate deferred expressions, and only the appropriate deferred expressions, will be executed. Replace "function" with "test case", and it sounds perfect for tearing down tests.

So, how would you implement that in C? Well, it turns out that GCC has two useful non-standard extensions (which are also supported by Clang by the way): local labels, and labels as values.

Local labels are basically regular labels which you can jump to with goto, but instead of being global to the entire function, they're only available in the block they're declared in. That's fairly straightforward. You declare that a label should be block scoped by just putting __label__ label_name; at the top of the block, and then you can use label_name: anywhere within the block to actually create the label. A goto label_name from anywhere within the block will then go to the label, as expected.

Labels as values is weirder. GCC adds a new unary && operator, which gets a pointer to a label as a void *. Moreover, if you save that pointer in a variable which is accessible outside the block, you can jump back in to that block from outside of it, even though it's a local label. This will print "hello" in an infinite loop:

{
	void *somelabel;

	{
		__label__ lbl;
		lbl:
		somelabel = &&lbl;
		printf("hello\n");
	}

	goto *somelabel;
}

Yes, the somelabel is a void *. Yes, we dereference somelabel to go to it. I don't know how that works, but the important part is that it does. Other than being dereferencable, the void * we get from the unary && works exactly like any other void *, and can even be in an array. Knowing this, implementing defer isn't too hard; here's a simplified implementation of the it(description, block) macro (using the __VA_ARGS__ trick from before) which describes one test case, and the defer(expr) macro which can be used inside the it block:

#define it(description, ...) \
	do { \
		__label__ done_label; \
		void *defer_labels[32]; \
		int defer_count = 0; \
		int run_defer = 0; \
		__VA_ARGS__ \
		done_label: \
		run_defer = 1; \
		if (defer_count > 0) { \
			defer_count -= 1; \
			goto *defer_labels[defer_count]; \
		} \
	} while (0)

#define defer(expr) \
	do { \
		__label__ lbl; \
		lbl: \
		if (run_defer) { \
			expr; \
			/* Go to the previous defer, or the end of the `it` block */ \
			if (defer_count > 0) { \
				defer_count -= 1; \
				goto *defer_labels[defer_count]; \
			} else { \
				goto done_label; \
			} \
		} else { \
			defer_labels[defer_count] = &&lbl; \
			defer_count += 1; \
		} \
	} while (0)

That might not be the most understandable code you've ever seen, but let's break it down with an example.

it("whatever", {
	printf("Hello World\n");
	defer(printf("world\n"));
	defer(printf("hello "));
});

Running that through the preprocessor, we get this code:

do {
	__label__ done_label;
	void *defer_labels[32];
	int defer_count = 0;
	int run_defer = 0;

	{
		printf("Hello World\n");

		do {
			__label__ lbl;
			lbl:
			if (run_defer) {
				printf("world\n");

				/* Go to the previous defer, or the end of the `it` block */
				if (defer_count > 0) {
					defer_count -= 1;
					goto *defer_labels[defer_count];
				} else {
					goto done_label;
				}
			} else {
				defer_labels[defer_count] = &&lbl;
				defer_count += 1;
			}
		} while (0);

		do {
			__label__ lbl;
			lbl:
			if (run_defer) {
				printf("hello ");

				/* Go to the previous defer, or the end of the `it` block */
				if (defer_count > 0) {
					defer_count -= 1;
					goto *defer_labels[defer_count];
				} else {
					goto done_label;
				}
			} else {
				defer_labels[defer_count] = &&lbl;
				defer_count += 1;
			}
		} while (0);
	}

	done_label:
	run_defer = 1;
	if (defer_count > 0) {
		defer_count -= 1;
		goto *defer_labels[defer_count];
	}
} while (0);

That's still not extremely obvious on first sight, but it's at least more obvious than staring at the macro definitions. The first time through, run_defer is false, so both the defer blocks will just add their labels to the defer_labels array and increment defer_count. Then, just through normal execution (without any goto), we end up at the label called done_label, where we set run_defer to true. Because defer_count is 2, we decrement defer_count and jump to defer_labels[1], which is the last defer.

This time, because run_defer is true, we run the deferred expression printf("hello "), decrement defer_count again, and jump to defer_labels[0], which is the first defer.

The first defer runs its expression, printf("world\n"), but because defer_count is now 0, we jump back to done_label. defer_count is of course still 0, so we just exit the block.

The really nice thing about this system is that a failing assert can at any time just say goto done_label, and only the expressions which were deferred before the goto will be executed.

(Note: in the actual implementation in Snow, defer_labels is of course a dynamically allocated array which is realloc'd when necessary. It's also global to avoid an allocation and free for every single test case. I omitted that part because it's not that relevant, and would've made the example code unnecessarily complicated.)

Update: A bunch of people on Reddit and Hacker News have suggested ways to accomplish this. I ended up using the __attribute__((constructor)) function attribute, which makes a given function execute before the main function. Basically, each describe creates a function called test_##name, and a constructor function called _snow_constructor_##name whose only job is to add test_##name to a global list of functions. Here's the code: https://github.com/mortie/snow/blob/7ee25ebbf0edee519c6eb6d36b82d784b0fdcbfb/snow/snow.h#L393-L421

Automatically call all functions created by describe

The describe macro is meant to be used at the top level, outside of functions, because it creates functions. It's basically just this:

#define describe(name, ...) \
	void test_##name() { \
		__VA_ARGS__ \
	}

Calling describe(something, {}) will create a function called test_something. Currently, that function has to be called manually, because no other part of Snow knows what the function is named. If you have used the describe macro to define the functions test_foo, test_bar, and test_baz, the main function will look like this:

snow_main({
	test_foo();
	test_bar();
	test_baz();
})

I would have loved it if snow_main could just know what functions are declared by describe, and automatically call them. I will go over a couple of ways I tried, which eventually turned out to not be possible, and then one way which would definitely work, but which is a little too crazy, even for me.

Static array of function pointers

What if, instead of just declaring functions with describe, we also appended them to an array of function pointers? What if snow.h contained code like this:

void (*described_functions[512])();

#define describe(name, ...) \
	void test_##name() { \
		__VA_ARGS__ \
	} \
	described_functions[__COUNTER__] = &test_##name

__COUNTER__ is a special macro which starts at 0, and is incremented by one every time it's referenced. That means that assuming nothing else uses __COUNTER__, this solution would have worked, and would have been relatively clean, if only it was valid syntax. Sadly, you can't set the value of an index in an array like that in the top level in C, only inside functions.

Appending to a macro

What if we had a macro which we appended test_##name(); to every time a function is declared by describe? It turns out that this is almost possible using some obscure GCC extensions. I found this solution on StackOverflow:

#define described_functions test_foo();

#pragma push_macro("described_functions")
#undef described_functions
#define described_functions _Pragma("pop_macro(\"described_functions\")") described_functions test_bar();

#pragma push_macro("described_functions")
#undef described_functions
#define described_functions _Pragma("pop_macro(\"described_functions\")") described_functions test_baz();

described_functions // expands to test_foo(); test_bar(); test_baz();

This is actually a way to append a string to a macro which works, at least in GCC. Snow could have used that... except for one problem: you of course can't use #define from within a macro, and we would have needed to do this from within the describe macro. I have searched far and wide for a way, even a weird GCC-specific possibly pragma-related way, to redefine a macro from within another macro, but I haven't found anything. Close, but no cigar.

The way which actually works

I mentioned that there is actually one way to do it. Before I show you, I need to cover dlopen and dlsym.

void *dlopen(const char *filename, int flags) opens a binary (usually a shared object... usually), and returns a handle. Giving dlopen NULL as the file name gives us a handle to the main program.

void *dlsym(void *handle, const char *symbol) returns a pointer to a symbol (for example a function) in the binary which handle refers to.

We can use dlopen and dlsym like this:

#include <stdio.h>
#include <dlfcn.h>

void foo() {
	printf("hello world\n");
}

int main() {
	void *h = dlopen(NULL, RTLD_LAZY);

	void *fptr = dlsym(h, "foo");
	void (*f)() = fptr;
	f();

	dlclose(h);
}

Compile that code with gcc -Wl,--export-dynamic -ldl -o something something.c, and run ./something, and you'll see it print hello world to the terminal. That means we can actually call functions dynamically based on an arbitrary string at runtime. (The -Wl,--export-dynamic is necessary to tell the linker to export the symbols, such that they're available to us through dlsym).

Being able to run functions based on a runtime C string, combined with our friend __COUNTER__, opens up some interesting possibilities. We could write a program like this:

#include <stdio.h>
#include <dlfcn.h>

/* Annoyingly, the concat_ and concat macros are necessary to
 * be able to use __COUNTER__ in an identifier name */
#define concat_(a, b) a ## b
#define concat(a, b) concat_(a, b)

#define describe(...) \
	void concat(test_, __COUNTER__)() { \
		__VA_ARGS__ \
	}

describe({
	printf("Hello from function 0\n");
})

describe({
	printf("Hi from function 1\n");
})

int main() {
	void *h = dlopen(NULL, RTLD_LAZY);
	char symbol[32] = { '\0' };

	for (int i = 0; i < __COUNTER__; ++i) {
		snprintf(symbol, 31, "test_%i", i);
		void *fptr = dlsym(h, symbol);
		void (*f)() = fptr;
		f();
	}

	dlclose(h);
}

Run that through the preprocessor, and we get:

void test_0() {
	{ printf("Hello from function 0\n"); }
}
void test_1() {
	{ printf("Hi from function 1\n"); }
}

int main() {
	void *h = dlopen(NULL, RTLD_LAZY);
	char symbol[32] = { '\0' };

	for (int i = 0; i < 2; ++i) {
		snprintf(symbol, 31, "test_%i", i);
		void *fptr = dlsym(h, symbol);
		void (*f)() = fptr;
		f();
	}

	dlclose(h);
}

That for loop in our main function will first call test_0(), then test_1().

I hope you understand why even though this technically works, it's not exactly something I want to include in Snow ;)

Read More