Understanding how multiple factors influence outcomes is crucial in experimental design. When you have more than one factor, you need to distinguish between main effects and interaction effects. The main effect of a factor is the average impact that factor has on the outcome, regardless of the levels of other factors. For example, if you are testing two website layouts (Factor A: Layout 1 vs Layout 2) and two button colors (Factor B: Blue vs Red), the main effect of layout is how changing the layout affects user clicks, averaged across both button colors.

Interaction effects occur when the effect of one factor depends on the level of another factor. Using the website example, suppose Layout 1 works best with a blue button, but Layout 2 works best with a red button. Here, the impact of layout changes depending on button color — this is an interaction effect. Recognizing interactions is essential because it helps you avoid misleading conclusions that might arise if you only consider main effects.

To make interaction effects more concrete, consider the following table showing average user clicks for each combination of two factors, A (Layout) and B (Button Color):

In this table, you can see that Layout 1 performs better with a red button, while Layout 2 performs better with a blue button. The outcome for each combination is not simply the sum of the main effects of A and B; instead, the effect of one factor depends on the level of the other. This pattern highlights the presence of an interaction effect.

You can use a simple formula to capture the intuition behind interaction effects. For two factors, A and B, the interaction effect can be calculated as the difference of differences:

If this value is not zero, it means there is an interaction: the effect of one factor changes depending on the level of the other factor.