Pseudoreplication
Encyclopedia
Hurlbert
defined pseudoreplication as the use of inferential statistics to test for treatment effects with data from experiments where either treatments are not replicated (though samples may be) or replicates are not statistically independent.
The error described by Hurlbert arises when an F-ratio in an analysis of variance (ANOVA) table is formed with respect to the residual mean square rather than with respect to the among unit mean square. The misformed F-ratio fails to account for among unit effects when declaring statistical significance. When unit number is small (e.g four tank units, two tanks treated, two not treated, many measurements per tank) the misformed F-ratio is vulnerable to unit (tank) effects, and can result in a statistically significant treatment mean square when there are no treatment effects. The error arises frequently from the default setting in many statistical packages, which is to form the F-ratio relative to the residual mean square. The error is avoided by forming the F-ratio relative to the next lower random factor in the ANOVA table (tanks in the example above), rather than the lowest level (residual mean square in the example above).
increases the precision of an estimate, while randomization addresses the broader applicability of a sample to a population. Replication must be appropriate: replication at the experimental unit
level must be considered, in addition to replication within units.
and the related ANOVA family of tests) rely on adequate replication to estimate statistical confidence
. Tests based on the chisquare, t, and F- distributions assume homogeneous, normal, and independent errors.
defined pseudoreplication as the use of inferential statistics to test for treatment effects with data from experiments where either treatments are not replicated (though samples may be) or replicates are not statistically independent.
The error described by Hurlbert arises when an F-ratio in an analysis of variance (ANOVA) table is formed with respect to the residual mean square rather than with respect to the among unit mean square. The misformed F-ratio fails to account for among unit effects when declaring statistical significance. When unit number is small (e.g four tank units, two tanks treated, two not treated, many measurements per tank) the misformed F-ratio is vulnerable to unit (tank) effects, and can result in a statistically significant treatment mean square when there are no treatment effects. The error arises frequently from the default setting in many statistical packages, which is to form the F-ratio relative to the residual mean square. The error is avoided by forming the F-ratio relative to the next lower random factor in the ANOVA table (tanks in the example above), rather than the lowest level (residual mean square in the example above).
Replication
ReplicationReplication (statistics)
In engineering, science, and statistics, replication is the repetition of an experimental condition so that the variability associated with the phenomenon can be estimated. ASTM, in standard E1847, defines replication as "the repetition of the set of all the treatment combinations to be compared in...
increases the precision of an estimate, while randomization addresses the broader applicability of a sample to a population. Replication must be appropriate: replication at the experimental unit
Statistical unit
A unit in a statistical analysis refers to one member of a set of entities being studied. It is the material source for the mathematical abstraction of a "random variable"...
level must be considered, in addition to replication within units.
Statistics and replication
Statistical tests (e.g. t-testStudent's t-test
A t-test is any statistical hypothesis test in which the test statistic follows a Student's t distribution if the null hypothesis is supported. It is most commonly applied when the test statistic would follow a normal distribution if the value of a scaling term in the test statistic were known...
and the related ANOVA family of tests) rely on adequate replication to estimate statistical confidence
Confidence interval
In statistics, a confidence interval is a particular kind of interval estimate of a population parameter and is used to indicate the reliability of an estimate. It is an observed interval , in principle different from sample to sample, that frequently includes the parameter of interest, if the...
. Tests based on the chisquare, t, and F- distributions assume homogeneous, normal, and independent errors.
Types
Hurlbert described several sources of pseudoreplication.- Simple pseudoreplication occurs when there is one experimental unit per treatment, as in Hurlbert (1984) Figure 5a. Treatment effects cannot be separated statistically from variability of experimental units.
- Temporal pseudoreplication occurs when experimental units differ enough in time that temporal effects are likely, and limited replication of experimental units precludes statistical separation of treatment effects from variability among experimental units.
- Sacrificial pseudoreplication occurs when means within a treatment are used in an analysis, and the means are tested over the within unit variance. In the example shown (Figure 5b) the erroneous F-ratio will have 1 df in the numerator (treatment) and 4 df in the denominator (2-1 = 1 df for each experimental unit). The correct F-ratio will have 1 df in the numerator (treatment) and 2 df in the denominator (2-1 = 1 df for each treatment). This F-ratio controls for effects of experimental units but with 2 df in the denominator it will have little power to detect treatment differences.
- Implicit pseudoreplication occurs when standard errors (or confidence limits) are estimated within experimental units. As with other sources of pseudoreplication, treatment effects cannot be statistically separated from effects due to variation among experimental units.