The books by Campbell and Stanley and Cook and Campbell are considered classic in the field of experimental design. The following is summary of their books with insertion of our examples.
Problem and Background Experimental method and essay-writing Campbell and Stanley point out that adherence to experimentation dominated the field of education through the s Thorndike era but that this gave way to great pessimism and rejection by Threats to internal validity maturation late s. However, it should be noted that a departure from experimentation to essay writing Thorndike to Gestalt Psychology occurred most often by people already adept at the experimental tradition.
Therefore we must be aware of the past so that we avoid total rejection of any method, and instead take a serious look at the effectiveness and applicability of current and past methods without making false assumptions. Replication Multiple experimentation is more typical of science than a once and for all definitive experiment!
Experiments really need replication and cross-validation at various times and conditions before the results can be theoretically interpreted with confidence. Cumulative wisdom An interesting point made is that experiments which produce opposing theories against each other probably will not have clear cut outcomes--that in fact both researchers have observed something valid which represents the truth.
Adopting experimentation in education should not imply advocating a position incompatible with traditional wisdom, rather experimentation may be seen as a process of refining this wisdom. Therefore these areas, cumulative wisdom and science, need not be opposing forces.
Factors Jeopardizing Internal and External Validity Please note that validity discussed here is in the context of experimental design, not in the context of measurement.
Factors which jeopardize internal validity History --the specific events which occur between the first and second measurement. Factors which jeopardize external validity Reactive or interaction effect of testing --a Threats to internal validity maturation might increase or decrease a subject's sensitivity or responsiveness to the experimental variable.
X --Treatment O --Observation or measurement R --Random assignment The three experimental designs discussed in this section are: A group is introduced to a treatment or condition and then observed for changes which are attributed to the treatment X O The Problems with this design are: A total lack of control.
Also, it is of very little scientific value as securing scientific evidence to make a comparison, and recording differences or contrasts. O 1 X O 2 However, there exists threats to the validity of the above assertion: History --between O 1 and O 2 many events may have occurred apart from X to produce the differences in outcomes.
The longer the time lapse between O 1 and O 2the more likely history becomes a threat. X O 1 O 2 Threats to validity include: Selection --groups selected may actually be disparate prior to any treatment.
Three True Experimental Designs The next three designs discussed are the most strongly recommended designs: An explanation of how this design controls for these threats is below. History --this is controlled in that the general history events which may have contributed to the O 1 and O 2 effects would also produce the O 3 and O 4 effects.
This is true only if the experiment is run in a specific manner--meaning that you may not test the treatment and control groups at different Threats to internal validity maturation and in vastly different settings as these Threats to internal validity maturation may effect the results.
Rather, you must test simultaneously the control and experimental groups.
Intrasession history must also be taken into consideration. For example if the groups truly are run simultaneously, then there must be different experimenters involved, and the differences between the experimenters may contribute to effects. A solution to history in this case is the randomization of experimental occasions--balanced in terms of experimenter, time of day, week and etc. The factors described so far effect internal validity.
These factors could produce changes which may be interpreted as the result of the treatment. These are called main effects which have been controlled in this design giving it internal validity.
However, in this design, there are threats to external validity also called interaction effects because they involve the treatment and some other variable the interaction of which cause the threat to validity. It is important to note here that external validity or generalizability always turns out to involve extrapolation into a realm not represented in one's sample. In contrast, internal validity are solvable within the limits of the logic of probability statistics.
This means that we can control for internal validity based on probability statistics within the experiment conducted, however, external validity or generalizability can not logically occur because we can't logically extrapolate to different conditions.
Hume's truism that induction or generalization is never fully justified logically. Interaction of testing and X --because the interaction between taking a pretest and the treatment itself may effect the results of the experimental group, it is desirable to use a design which does not use a pretest.
Research should be conducted in schools in this manner--ideas for research should originate with teachers or other school personnel. The designs for this research should be worked out with someone expert at research methodology, and the research itself carried out by those who came up with the research idea.