What are some examples when practical significance would outweigh statistical significance?
1 Answer
Any time the statistical significance is considered too small or marginal to justify the risk of a potential negative outcome.
Explanation:
It depends on your risk tolerance, which is different for everyone. It also requires that you understand that "significance" itself is relative to the test parameters and sample sizes.
In ALL cases, statistics are only a tool to assist people in making better decisions. They are neither absolute nor infallible. They also include their own set of assumptions that may not be accurate.
Generally, the "judgement call" that goes against an evaluation of "statistical significance" is based on a personal or financial risk tolerance.
A forecast for sunny skies may be statistically significant for a particular set of criteria, but IF it rains, the outcome could be perceived as more disastrous for an individual.
Similarly, a calculated significance of one product line performance over another may be too marginal to overcome the owner's personal preference or fear of financial loss.
Finally, a "significant difference" in poll readings may again be subject to future changes or personal perceptions of future outcomes. In that case, any decision based on the poll results would be either deferred for more data, or rejected entirely for other "practical" reasons of the individuals.
