An ANOVA tells us whether a statistically reliable difference exists between groups (e.g., Is the conversion rate difference we observe between East, Central, and South regions statistically reliable?), but it does not tell us between which groups the difference exists. This is where post-hoc analysis comes in. The post-hoc analysis tests each pair of groups (e.g., East v Central, East v West, West v Central) to see if they are reliably different from one another. Let’s consider an example.
In our own B2B Buying Experience research, we surveyed buyers from six industries and studied their interactions with sellers. The result of our ANOVA told us that there is indeed a statistically reliable difference in the number of interactions buyers have with sellers across industries.
Through a post-hoc test we found that buying group members in Manufacturing interacted less (12.8 times) than those in Professional Services (16.7), Business Services (16.2), Tech (16.9), and Financial Services (16.9). Differences between the other industries did not meet the p<.05 standard for statistical significance (reliability).
Thus, our ANOVA test told us that there was at least one reliable difference across industries, while our post-hoc analysis helped us understand which groups differed from which others.
