I've changed my mind already. The other versions of fuzzpuzz all have
different command-line interfaces anyway, so I think the best approach
is to just accept that and decide that precisely how fuzzpuzz works
isn't a defined API. Fuzzing is inherently not an end-user activity, so
I think it's acceptable to make it a bit inconsistent.
This means that in Clang builds you get the non-libFuzzer version of
fuzzpuzz by default (so you can use it with other fuzzers), but if you
turn on WITH_LIBFUZZER then you'll get the libFuzzer version instead.
I've separated out the various versions of main(), which has helped a
little bit. I've also stopped using fmemopen() since libFuzzer might
work on Windows. But I think I probably still have something
fundamentally wrong in my approach.
For AFL++ and Honggfuzz, our approach is to build a standard fuzzpuzz
binary with extra hooks for interacting with an external fuzzer. This
works well for AFL++ and tolerably for Honggfuzz. LibFuzzer, though,
provides its own main() so that the resulting program has a very
different command-line interface from the normal one. Also, since
libFuzzer is a standard part of Clang, we can't decide whether to use it
based on the behaviour of the compiler.
So what I've done, at least for now, is to have CMake detect when we're
using Clang and in that case build a separate binary called
"fuzzpuzz-libfuzzer" which is built with -fsanitize=fuzzer, while the
ordinary fuzzpuzz is built without. I'm not sure if this is the right
approach, though.
Unlike AFL, Honggfuzz's compiler wrapper doesn't provide a convenient
preprocessor macro, so we have to have CMake detect the existence of
HF_ITER. Also the resulting program can't run outside of Honggfuzz, so
maybe some additional cleverness is called for there as well. Still, it
makes Honggfuzz go ten times faster, which is nice.
GCC warns about that character sequence in a comment. I shouldn't have
assumed that having only edited a comment meant I could get away without
a test build.
There's now a function, fuzz_one(), that processes a single save file,
and main() arranges to call this a suitable number of times depending
on whether we're in AFL persistent mode or not. This makes things a
bit cleaner, and will probably make adding good support for other
fuzzers, or just switching entirely to the horrible but popular
libFuzzer interface, simpler.
Now if the input save file loads correctly, fuzzpuzz asks the back-end
to draw the puzzle. All the drawing operations are no-ops, but this
tests the drawing code in the back-end.
When reporting that the game name in a save file isn't recognised,
don't include the name from the save file in the error message, partly
to avoid the complexity of freeing it properly on two different code
paths and partly because including unsanitized data from a
fuzzer-supplied save file in the error message just seems dangerous.
And properly sanitising it would waste the fuzzer's time exploring the
sanitising code.
Thanks to Ben Hutchings for reporting the bug.
Now there's a single version of the main loop that runs once in normal
mode and repeatedly in AFL++ persistent mode. In persistent mode,
fmemopen() allows the loop to read the shared-memory buffer as though
it were a stdio stream. fmemopen() is POSIX-only, but so is AFL++.
Rather than a save file from standard input and then exiting, this reads
it from a shared memory buffer and then loops. This makes fuzzing
_much_ faster: one core on my laptop can now load about 30,000 save
files per second.
