CS253 Testing

Philosophy of testing

Some programmers don’t enjoy testing. They want to create, not test. ☹
- I don’t enjoy doing dishes, but that’s the price I pay for using a plate.
- Try to think of it as a step in perfecting your creation.
- Many big companies have QA departments, who do testing. Hooray!
  - It’s fun work.
  - Still, don’t send them trash. Everything runs more smoothly if you manage to do a minimum of testing.
Nearly all programmers have a mental block against testing. It’s like asking parents to find flaws in their children.
- This is why industry uses non-author testing.
- You just have to deal with it.

Sure, whoever programmed this is really smart (especially if they’re you). It’s probably correct.

Testing the typical cases is often more of a social thing than an engineering thing. For non-author testing, if you can present a typical, non-extreme, case that fails, the author will take it seriously. “Dude, it doesn’t even work for this case!”
If you don’t bother verifying the test cases that your teacher/programmer gave to you, then success must not be your goal.

Consider a program that’s a guessing game. You think of a number 1–100, and it has to guess the number. You tell it “too high”, “too low”, etc.
If this program works for 42, it’ll probably work for 56.
Test it for 1, 100, 0, and 101. Those are the edge cases. They’re at the boundary between “should work” and “shouldn’t work”. You want to ensure that a < isn’t accidentally a ≤.
Also, 50 & 51 might be edge cases, for a binary search.

Poor testers test manually. When the code changes, they test again.
- Well, some of the tests.
- If they remember what the tests are.
- And have the energy.
Good testers write an automated test suite (as simple as a shell script). They run the test suite after every code change.

    #! /bin/bash
    (
        ./hw2 <alpha
        ./hw2 alpha
        ./hw2 alpha beta
        …
    ) >&out
    diff out known-good-output