git bisect

Sat 7 Sep 2024

I was tracking down a bug in a big open-source project — let's call it Project Foo — and wanted to know exactly which commit had introduced the bug. I knew that the bug was present at HEAD, and that it was not present at 2e243b2bc. Checking out and trying every commit among the many in between would have been tedious, so I used Git's wonderful bisect subcommand. Along the way, I learned about magic number 125.

Rather than test every commit in the given range, git bisect uses binary search. On each run, it checks out a commit, runs a test script provided by the user, and uses the exit code to decide in which direction to move through commit history. Eventually, it narrows its search down to the first bad commit.

My test script (see below) builds Foo and runs it to see whether its output shows the bug. Its exit code is 0 if the bug appears, and 1 if it doesn't. That's all standard practice with git bisect.

The commands below start the search after telling git that the most recent commit was bad and that commit 2e243b2bc was good.

git bisect reset
git bisect start
git bisect bad
git bisect good 2e243b2bc
git bisect run test.sh

But git bisect has a workaround for that problem: return the magic number 125.

In Project Foo, there was a wrinkle. On some commits, it didn't even build, which meant that there was no way to test for the bug. But git bisect has a workaround for that problem: return the magic number 125. If the test script's exit code is 125, git bisect knows that that commit neither succeeded nor failed. It skips that commit, and continues searching.

This is all documented on git bisect's man page, but I didn't know that until I hit the problem myself. I'm impressed with this practical solution.

#!/bin/sh

set -e

BUGGY_OUTPUT="1 + 1 = 3"
TEST_OUTPUT_FILE=$(mktemp)

trap "rm -f $TEST_OUTPUT_FILE" 0 1 15
make clean
make foo || exit 125   # in case the build fails
./foo << EOF | tee $TEST_OUTPUT_FILE
1 + 1
EOF
if grep --fixed-strings --quiet "$BUGGY_OUTPUT" $TEST_OUTPUT_FILE; then
  echo Bug found.
  exit 1
fi
echo Bug not found.

P.S.

Daphne Preston-Kendal suggested that I mention the git bisect skip command, which can be used in the same way as exit status 125 when running git bisect manually.