Experimental design

Outline

This course identifies major problems in experimental and observational research and provides designs that avoid or mitigate those problems. After completing the course, the participants will be aware of common problems in experimental and observational research. They will also have a set of tools with which to avoid or remedy such problems. The order of module presentation may be different on the day of the course and Dr. Davis will, of course, answer any specific questions from course participants.

1. Confounding of factors

Problem
Two variables co‑vary so their effects cannot be separated.

Ecological example

• Species diversity correlates with species taxonomy. Thus diversity effects are confounded with taxonomy.
• Along the edge of woodland the amount of shade and amount of moisture co‑vary. Thus it’s impossible to isolate the effect of light alone on seedling survival.

Biochemical example

• Enzyme activity appears higher in one treatment, but pH differs systematically between treatments, so pH and treatment are confounded.
• Extraction solvent and extraction time vary together, making it unclear which factor drives yield.

Designs that mitigate

• Randomized controlled designs to break systematic associations.
• Factorial designs to estimate independent and interaction effects.
• Blocking (e.g., soil type, batch, plate) to remove nuisance variation.

2. Multi-collinearity among predictors

Problem
Predictors are correlated, inflating variance and destabilizing inference.

Ecological example

• Soil nitrogen, phosphorus, and organic matter are strongly correlated across plots.
• Temperature and elevation co‑vary in mountain transects.

Biochemical example

• In metabolomics, several metabolites in the same pathway rise and fall together, creating collinearity in regression models.
• In enzyme kinetics, substrate concentration and ionic strength may be inadvertently correlated across assay conditions.

Designs that mitigate

• Orthogonal factorials to ensure independent variation.
• Response‑surface designs (Box–Behnken, central composite) for clean quadratic estimation.
• Balanced sampling across environmental gradients.

3. Spatial or temporal autocorrelation

Problem
Observations that are closer in space or in time are more similar. The observations are therefore not independent as required by many analyses.

Ecological example

• Herbivore damage on plants is spatially clustered due to insect movement.
• Soil microbial communities show spatial structure at metre scales.
• Repeated surveys of the same transect show temporal autocorrelation.

Biochemical example

• Plate‑based assays show row/column effects (temperature gradients, evaporation).
• Batch effects in sequencing runs create temporal autocorrelation in read counts.

Designs that mitigate

• Spatially balanced sampling (Generalized Random‑Tessellation Stratified sampling, Latin hypercube).
• Blocking by spatial strata (e.g., watershed, plot, transect).
• Randomization restrictions (whole‑plot/strip‑plot).
• Replication across independent sites or batches.

4. Insufficient sample size / low power

Problem
Effects are undetectable or unstable.

Ecological example

• Rare species occupancy models fail because detection probability is low and sampling effort insufficient.
• Trait–environment regressions have too few species per environmental gradient.

Biochemical example

• Enzyme inhibition curves with too few substrate concentrations cannot estimate or reliably.
• RNA‑seq experiments with low replication inflate false positives.

Designs that mitigate

• Power‑based sample‑size planning using expected variance.
• Balanced designs to maximize power for fixed N.
• Repeated‑measures designs (e.g., same plant measured over time).
• Sequential designs to expand if needed.

5. Uncontrolled heterogeneity

Problem
Background variation obscures treatment effects.

Ecological example

• Soil heterogeneity within plots masks fertilizer effects.
• Variation in canopy cover obscures understory plant responses.

Biochemical example

• Variation in protein concentration across extracts masks treatment differences.
• Cell‑culture experiments suffer from flask‑to‑flask heterogeneity.

Designs that mitigate

• Blocking (soil type, canopy class, batch, plate).
• Split‑plot designs for hard‑to‑change factors (e.g., irrigation regime).
• Matched‑pairs designs (pair similar plants, soils, or samples).

6. Non‑independence due to repeated measurements

Problem
Multiple measurements on the same unit treated as independent inflate degrees of freedom.

Ecological example

• Measuring leaf traits on multiple leaves from the same plant.
• Multiple samples from the same leaf.
• Repeated surveys of the same nest, burrow, or colony.
• Multiple soil cores from the same plot.

Biochemical example

• Multiple technical replicates from the same extract treated as independent.
• Time‑course measurements of enzyme activity treated as separate samples.
• Sequences of NMR spectrums from the same machine.

Designs that mitigate

• Repeated‑measures designs with appropriate random effects.
• Nested designs
• Crossover designs (e.g., plants receiving treatments in different orders).
• Cluster‑randomized designs (e.g., randomize at plot or colony level).

7. Non‑linear responses missed by simple designs

Problem
Linear designs cannot detect curvature, thresholds, or optima.

Ecological example

• Nutrient addition shows saturating growth response.
• Temperature effects on metabolic rate follow a Q10 curve, not a line.
• Herbivore density effects on plant biomass show thresholds.

Biochemical example

• Enzyme kinetics are inherently non‑linear (Michaelis–Menten).
• Dose–response curves for toxins or inhibitors are sigmoidal.
• Fluorescence assays saturate at high concentrations.

Designs that mitigate

• Response‑surface designs to map curvature.
• Three‑level factorials for quadratic terms.
• Optimal designs (D‑optimal, I‑optimal) for non‑linear models.

(8. Aliasing of effects in fractional factorials

Problem
In fractional factorials, some effects are confounded with others.

Ecological example

• In a screening experiment manipulating light, nutrients, and herbivory, a fractional factorial may alias nutrient × herbivory with light × nutrients.
• In mesocosm studies, water temperature and dissolved oxygen manipulations may be aliased if not fully crossed.

Biochemical example

• Screening multiple buffer components (pH, salt, cofactor, reducing agent) with a fractional factorial can alias main effects with interactions.
• In protein‑expression optimization, inducer concentration and temperature may be aliased.

Designs that mitigate

• Higher‑resolution fractional factorials (Resolution V).
• Foldover designs to break aliasing.
• Sequential augmentation to de‑alias key effects.)