Dealing with errors involves three distinct activities, and it helps to keep them apart. The first is knowing what can go wrong: the kinds of error, and the specific mistakes that cause most of them. The second is testing, which finds out whether a program is wrong by running it on chosen inputs and comparing the results with what should happen. The third is debugging, which finds out where and why it is wrong once a test has failed. A fourth tool, static analysis, checks code without running it at all, and it complements testing without replacing it. The section takes these in that order, starting from a classification of errors by when they become visible, since that determines which tool can catch them.

Three kinds of error#

Errors can be sorted by the moment they show themselves.¹ A syntax error breaks the rules of a language, such as a missing parenthesis or a misspelled keyword, so the program can't run at all: the whole text is checked against the language's rules before any statement executes, and nothing is displayed, not even output from lines before the mistake. A runtime error occurs while the program is running, when it attempts something impossible, such as reading an index that doesn't exist, and execution stops with an error message. A logic error is the hardest kind, because the program runs to completion and produces a wrong result with no message at all. A function meant to return the larger of two values that returns 3 for larger(3, 8) has a logic error, likely a comparison written backward, and a program that counts readings "at or below freezing" with c < 0 has a logic error that excludes the boundary value 0. Since nothing announces a logic error, only a test that compares the output with the right answer can reveal it.

Common mistakes#

Most errors come from a short list of mistakes, and most of those occur at the edges of loops and data.

An off-by-one error makes a loop run one time too many or too few, often from confusing < with <=.² With zero-based indexing, an array of size n has indexes 0 through n - 1, so its final index is n - 1. A loop over five scores written as for (int i = 0; i <= s.size[1]; i = i + 1) outputs all five values and then reaches s[5], which doesn't exist, and produces ERROR. A loop meant to run once for each of 6 seats but written for (int i = 1; i < 6; i = i + 1) runs only 5 times. Loops that look ahead need an earlier stop: a comparison of v[i] with v[i + 1] must use i < v.size[1] - 1, so that i + 1 is always a valid index. "Variable-focused iteration" lists the boundary checks worth making on every loop.

An infinite loop occurs when a loop's condition never becomes false. Often the loop variable isn't updated inside the loop's block at all, perhaps because the update line isn't indented under the loop, so check the indentation. Sometimes the variable is updated but in a way that never reaches the stopping point:

01 int x = 100
02 while (x != 0)
03   x = x - 15

Here x goes 100, 85, ..., 10, -5, and it skips past 0 without ever equaling it, so the loop never ends. A condition such as x > 0 would stop it. An update can also fail to change the value at all. A loop meant to halve n until it reaches 1, but whose update is n = n - n % 2, subtracts 0 whenever n is even, because 8 % 2 is 0, so n stays 8 forever. Tracing one pass by hand exposes this kind of mistake at once. Values of type float raise a subtler case: in this notation a float is treated as having infinite precision, so a loop that halves a positive float until it equals 0.0 never ends, though on real hardware rounding would eventually produce 0.0. Not every loop with an always-true condition is a mistake. A loop written while (TRUE) can end through break, which exits the loop immediately and continues with the first line after it, so a loop that reads input and breaks when the input is 0 ends there and goes on to its next line.

Runtime errors take a few common forms. An index out of bounds is one. Dividing by zero is another, including the easily missed case of averaging an empty array, or computing a percentage when the total is 0. Reading a map key that doesn't exist is a third, so a program should check that a key is present before reading it. Using a value of the wrong type is a fourth, as when == compares an int with a String, or when >= compares a number with a form field that arrives as text. Map keys must match exactly, including capital letters, so reading "Pencil" from a map that holds "pencil" is a missing key. In this notation, invalid or undefined values produce ERROR. Guarding an index means checking it before using it, as in if ((i >= 0) AND (i < v.size[1])) around the access.

Changing a list while looping over it combines several of these problems. Adding to a list while looping over it can make the loop run much longer than expected, or indefinitely, because a stopping condition that depends on the list's size keeps moving. A loop that appends a doubled copy of each value with i < v.size[1] as its condition never catches up with the end. The fix is to store the original size before the loop:

01 int n = v.size[1]
02 for (int i = 0; i < n; i = i + 1)
03   v.append(v[i] * 2)

Deleting while looping forward has the opposite problem. Each deletion shifts later values down one position, so the value that moves into position i is skipped when i increases, and a loop meant to remove every low score leaves one behind when two low scores are adjacent. Looping backward avoids this, since a deletion then shifts only values that have already been checked:

01 for (int i = v.size[1] - 1; i >= 0; i = i - 1)
02   if (v[i] < 60)
03     v.del(i)

Testing#

Knowing the common mistakes suggests where to look for them, which is the basis of choosing test cases. Good test cases include typical values, values at and just beyond each boundary, and unexpected input such as empty values. A function that returns "pass" for scores from 60 through 100 and "invalid" otherwise should be tested with 59, 60, 100, and 101, since off-by-one mistakes live exactly there. A form accepting ages 13 through 18 that has been tested with 15 still needs 12, 13, 18, 19, and a blank entry. Typical cases alone can pass while edge cases fail, and so can many random values, since random values seldom land exactly on a boundary. A function that starts a running maximum at 0 passes on ordinary lists and returns the wrong answer when every value is negative, so a list of negative values is also worth testing. A function that finds the middle value of a sorted array may work on every ordinary array and fail on an empty one, where it tries to read an index that doesn't exist. For a sort, the important edge cases are arrays with duplicates, with one value, and with none.

Testing becomes reliable when it is automatic. A unit test automatically checks that a small piece of code, usually one function, gives the expected output for a given input. Functions that take parameters and return values are the easiest to unit test, because their inputs and expected outputs are easy to state, while a function that reads keyboard input and prints its result is hard to check automatically. Units can each pass while failing in combination, as when a login function and an account function both work alone but the account function misreads what login passes it, so integration testing checks that units work together. And when a bug is fixed, adding a test for the input that revealed it creates a regression test, which keeps the bug from quietly returning in a later change. When a test fails, the fault may lie in the code or in the test itself, since the expected value written down in advance can also be wrong; a check that expects 18 for the area of a 4 by 5 rectangle fails even though the function is right.

Debugging#

When a test fails, debugging begins, and the difference between effective and ineffective debugging is mostly method. Systematic debugging means reproducing the problem with a specific input, isolating it to a specific location, forming a hypothesis about the cause, testing the hypothesis, changing one thing at a time, and retesting. Reproducing comes first, because a fix can't be checked against a problem that can't be made to happen. Isolating can be done efficiently by checking values in the middle of a function, which tells you which half contains the problem, much as binary search does.³ A hypothesis should lead to a direct test: if you suspect that a loop runs one time too few, printing the loop counter on each iteration shows exactly how many times it runs. Changing one thing at a time is what makes the process informative. A student who changes three lines and sees the bug disappear doesn't know which change mattered, or whether one of them introduced a new error, and a student who keeps changing several lines at once and rerunning has stopped learning anything from each run. A result that refutes a hypothesis is still useful, since it rules out one cause. For example, printing the counter shows that a loop does visit every position, and attention then moves to the loop's block. After a fix, rerun the earlier tests as well as the one that failed, because a change can break a case that used to pass: a leap-year condition changed to reject 1900 may also start rejecting 2000.

Careful tracing often finds bugs that hide at the end of the data. A program meant to report the longest run of identical values might update its record, best, only when a run ends with a different value. If the longest run is at the end of the array, it never ends that way, so run reaches 3 while best stays at 2.

Static analysis#

Testing and debugging both run the program. Static analysis instead examines source code without running it, and a linter is a tool that does this, flagging problems such as undefined variables, names that are declared but never used, and violations of style conventions. A linter would catch a variable rte that was never declared, perhaps a misspelling of rate, and a variable that is assigned but never read. What it can't catch are errors that depend on runtime values, such as dividing by a 0 that a user enters, or errors of meaning, such as a wrong formula for a tax amount, since a linter doesn't know what the program is supposed to compute. Static analysis and testing therefore find different kinds of problems, and a linter doesn't make testing unnecessary.

Several practices in this section, such as descriptive names and clear structure, also make errors less likely in the first place and easier to find when they occur. The next section turns to those qualities of code directly.

• syntax error: an error that breaks a language's rules, so the program can't run
• runtime error: an error during execution that stops the program
• logic error: an error that produces wrong results without any message
• off-by-one error: a loop that runs one time too many or too few
• infinite loop: a loop whose condition never becomes false
• break: a statement that exits a loop immediately
• test case: an input with its expected output, chosen to check a program
• edge case: an input at or beyond the limits of expected values, such as a boundary or an empty array
• unit test: an automatic check that one small piece of code gives the expected output
• integration testing: checking that units work correctly together
• regression test: a test that keeps a fixed bug from returning
• systematic debugging: reproducing, isolating, hypothesizing, changing one thing at a time, and retesting
• static analysis: examining source code without running it
• linter: a static analysis tool that flags undefined or unused names and style violations

Practice: practice questions for 0002.n

1. MWC1, Turtle: first programs, sequence, errors and bugs. ↩

2. MWC1, Iteration: lists, for loops, and range. ↩

3. MWC1, Typeface: decomposition and debugging in a problem set. ↩