Effect size is a measure of the magnitude of a finding of statistical significance. As we discussed in the summary of statistical significance, not all statistically significant findings are important or meaningful.
For example, in our research we found that IT buyers and non-IT buyers reliably rated the helpfulness of their interactions with potential suppliers differently. On a five point scale, non-IT buyers would be expected to rate seller interactions between 4.184 to 4.286 while IT buyers would be expected to rate their experience between 4.307 and 4.483. The top of the range for non-IT buyers is just .021 points below the bottom of the range for IT buyers. It’s a statistically reliable difference, but not a meaningful one. This would be deemed a small effect size.
Researchers measure effect size in a variety of ways, depending on the nature of the data and the statistical test used. For the most common types of data and tests, there are consensus guidelines for deciding whether an effect is small, medium, or large. These can be found in a variety of online resources.
In some cases, the determination of whether a finding is meaningful or not is best made by subject matter experts.
