3 years ago
Machine Learning Tip: Nested Cross Validation – When (Simple) Cross Validation Isn’t Enough

 Several scientific disciplines have been rocked by a crisis of reproducibility in recent years [1]. Not long ago, Bayer researchers found that they were only able to replicate 25% of the important pharmaceutical papers they examined [2], and an MIT report on Machine Learning papers found similar results. Some fields have begun to emerge from their crises, but other fields, such as psychology, may have not yet hit bottom [3] [4]. We might imagine that this is because many scientists are good at science but not so adept with statistics. We might even imagine that we Analytics practitioners should have fewer problems because we