Some aspects of an experiment design cannot be pinned down before the experiment is run. I once developed a model of the time required to contain several simulated forest fires, fought in succession. My model said, irrespective of the size or rate of growth of the fires, they should be fought in order of their age, younger fires before older ones. To test this rather surprising prediction, I designed the following fully factorial experiment, an experiment in which all combinations of all levels of each independent variable are represented:
Set two fires in a simulate forest. Fire A should be set H hous after fire B, so fire A is the yougest fire.
Fire A should grow {slowly, quickly}
Fire B shoud grow {slowly, quickly}
Fire A should be {small, large}
Fire B should be {small, large}
Fire A should be fought {before B, after B}
The experiment thus had 
conditions. My hypothesis was
that in the 16 conditions in which fire A, the youngest fire, was
fought first, the time to contain both fires would be less than in the
other 16 conditions. It was clear before I ran the experiment that
the time required to fight the fires would vary considerably across
the conditions; for example, a pair of large, fast-growing fires would
take longer to contain than one large and one small, slow-growing
fire. Thus, it might be difficult to find evidence for the
theoretical advantage of fighting the youngest fire first. In
particular, I had no idea how to set the parameter H, the age
difference between the fires, although my model said that the
advantage of the youngest-first strategy ought to be proportional to
H. Because the models we test in experiments are incomplete or imprecise,
we often do not know how to set the values of independent
variables to best see effects on dependent variables. How slow should
``slow growth'' be? How large is a ``large'' fire?
In such cases, one runs pilot experiments to find informative settings of experiment parameters. Pilot experiments also debug experiment protocols. With human subjects, for instance, one worries about the effects of fatigue. Computer programs don't suffer fatigue, but one often finds the experimental procedure takes longer to run than anticipated. Or when the program is heavily instrumented, the procedure can generate unexpected quantities of data. Or when garbage collection is uncontrolled, run times become uninterpretable. Or pieces of code that implement the experimental procedure can be buggy; for example, because fires are reported by watchtowers, which sometimes fail, I found in a pilot experiment that roughly half of my ``youngest fire first'' trials actually fought the oldest fire first!
Often, the only way to detect spurious effects and sampling biases is to run a pilot experiment. Floor effects and ceiling effects, remember, arise when test problems are too difficult or too easy for both the control and treatment conditions. It is difficult to predict these effects, but a pilot experiment can disclose them. Similarly, order effects and sampling biases are due to problems with the experiment protocol, and can be discovered in pilot data.