16 Commits

Author SHA1 Message Date
80de73a6aa Try to clean up fuzzpuzz a bit
I've separated out the various versions of main(), which has helped a
little bit.  I've also stopped using fmemopen() since libFuzzer might
work on Windows.  But I think I probably still have something
fundamentally wrong in my approach.
2023-02-23 11:34:26 +00:00
5ba227031c Rough support for fuzzing with libFuzzer
For AFL++ and Honggfuzz, our approach is to build a standard fuzzpuzz
binary with extra hooks for interacting with an external fuzzer.  This
works well for AFL++ and tolerably for Honggfuzz.  LibFuzzer, though,
provides its own main() so that the resulting program has a very
different command-line interface from the normal one.  Also, since
libFuzzer is a standard part of Clang, we can't decide whether to use it
based on the behaviour of the compiler.

So what I've done, at least for now, is to have CMake detect when we're
using Clang and in that case build a separate binary called
"fuzzpuzz-libfuzzer" which is built with -fsanitize=fuzzer, while the
ordinary fuzzpuzz is built without.  I'm not sure if this is the right
approach, though.
2023-02-23 11:34:20 +00:00
fccd2a55e7 Revert "Stop persistent-mode fuzzpuzz exiting prematurely"
That was completely wrong: a "continue" at the end of the loop is
unnecessary.

This reverts commit b91f9824b6f73290051025317f3387c7212fa05f.
2023-02-18 20:23:23 +00:00
b91f9824b6 Stop persistent-mode fuzzpuzz exiting prematurely
In the transition to fuzz_one() I'd lost a "continue".
2023-02-18 13:56:10 +00:00
150c05a298 Support Honggfuzz's persistent mode in fuzzpuzz
Unlike AFL, Honggfuzz's compiler wrapper doesn't provide a convenient
preprocessor macro, so we have to have CMake detect the existence of
HF_ITER.  Also the resulting program can't run outside of Honggfuzz, so
maybe some additional cleverness is called for there as well.  Still, it
makes Honggfuzz go ten times faster, which is nice.
2023-02-18 13:56:10 +00:00
1717d5b685 Adjust fuzzpuzz sample shell commands to not include "/*"
GCC warns about that character sequence in a comment.  I shouldn't have
assumed that having only edited a comment meant I could get away without
a test build.
2023-02-16 23:43:50 +00:00
a1f1d7c247 Update and expand comment at the head of fuzzpuzz
It now correctly describes what fuzzpuzz does.  It also provides an
example of how to use it with AFL++.
2023-02-16 23:26:43 +00:00
100cfd2e99 Separate fuzzing and harness in fuzzpuzz
There's now a function, fuzz_one(), that processes a single save file,
and main() arranges to call this a suitable number of times depending
on whether we're in AFL persistent mode or not.  This makes things a
bit cleaner, and will probably make adding good support for other
fuzzers, or just switching entirely to the horrible but popular
libFuzzer interface, simpler.
2023-02-16 22:57:23 +00:00
ffe2fa169f Extend fuzzpuzz to test more code
Now if the input save file loads correctly, fuzzpuzz asks the back-end
to draw the puzzle.  All the drawing operations are no-ops, but this
tests the drawing code in the back-end.
2023-02-13 21:23:58 +00:00
a2bf0508c7 Reserialise the game in fuzzpuzz
This means that the serialising code gets tested, and also provides a
convenient way to canonicalise a (valid) save file.
2023-02-13 21:19:07 +00:00
4b5422181d Fix use-after-free in fuzzpuzz
When reporting that the game name in a save file isn't recognised,
don't include the name from the save file in the error message, partly
to avoid the complexity of freeing it properly on two different code
paths and partly because including unsanitized data from a
fuzzer-supplied save file in the error message just seems dangerous.
And properly sanitising it would waste the fuzzer's time exploring the
sanitising code.

Thanks to Ben Hutchings for reporting the bug.
2023-01-16 10:43:41 +00:00
1a48d76fcc Remember to free the game name in fuzzpuzz 2023-01-12 22:21:45 +00:00
5fa1931560 Don't leak midends in fuzzpuzz
If deserialising a save file fails, the midend still needs to be freed.
2023-01-12 22:21:45 +00:00
311d227ba6 Merge the two versions of fuzzpuzz back together
Now there's a single version of the main loop that runs once in normal
mode and repeatedly in AFL++ persistent mode.  In persistent mode,
fmemopen() allows the loop to read the shared-memory buffer as though
it were a stdio stream.  fmemopen() is POSIX-only, but so is AFL++.
2023-01-12 22:21:45 +00:00
69924f376b AFL-specific shared-memory fuzzing mode
Rather than a save file from standard input and then exiting, this reads
it from a shared memory buffer and then loops.  This makes fuzzing
_much_ faster: one core on my laptop can now load about 30,000 save
files per second.
2023-01-12 22:21:45 +00:00
53a1faa0d7 Add a fuzzing harness for Puzzles
This just feeds save files into the loading code, but because of how
Puzzles is structured that actually exercises most of its parsers.
2023-01-12 22:21:34 +00:00