Confounding Events
Tip: If there have been extreme external events prior to or during the study period, check for possible impact of those events in the analysis.
Tip: Explore possible alternative explanations for both positive and negative findings.
Rationale: Unexpected outcomes from such natural disasters such as 2005's Hurricane Katrina and 2012's Superstorm Sandy closed schools and in many cases meant students needed extensions of deadlines to complete their work.1 While these are extreme events, there may be other events that may influence the data and skew the results so they could not be accurately applied to a program under more normal circumstances. For example, in difficult economic times, staying employed with no promotion could be a positive outcome for a training program.
1 Pappas, S. (October 31, 2012). Sandy wiped out NYU lab mice, dealing blow to medical research. LiveScience.