Do you know what p-hacking is? John Oliver – of HBO’s “Last Week Tonight” and formerly of “The Daily Show with Jon Stewart” – does.
It’s a tricky, potent analytical pitfall that’s gaining increased, deserving attention – across fields of science and even within Predictive Analytics Times articles and Predictive Analytics World sessions.
“An orange used car is least likely to be a lemon,” for example. That’s what was claimed by The Seattle Times, The Huffington Post, The New York Times, NPR, and The Wall Street Journal. But this discovery has since been debunked as inconclusive.
As data gets bigger, so does a common pitfall in the application of standard stats: Testing many predictors means taking many small risks of being fooled by randomness, adding up to one big risk.
John Elder calls this issue vast search.
It is also known as many other things, including multiple hypothesis testing, over-search, the look-elsewhere effect, the garden of forking paths, cherry-picking findings, and significance chasing.
Perhaps the catchiest name is p-hacking – and, believe it or not, John Oliver even used that particular term in his recent coverage of this hairy topic:
I was pleased to see this important topic covered by a relatively mainstream outlet.
It also so happens to be the topic of my Predictive Analytics World keynote this year – which I’m only set to deliver once more in 2016, at PAW New York (October 23-27):
Keynote title: Weird Science – How to Know Your Predictive Discovery Is Not BS
Speaker: Eric Siegel, Founder, Predictive Analytics World
Event: Predictive Analytics World for Business, New York
In this keynote, I will cover the issue and provide guidance on tapping data’s potential without drawing false conclusions. Click here for more info
Also check out these two articles on the topic:
Write-up describing my keynote: How Stubby Datasets Can Lead to Predictive Analytics Snafus
Another article by me: Breakthrough – How to Avert Analytics’ Most Treacherous Pitfall
Eric Siegel, Ph.D., founder of the Predictive Analytics World conference series and executive editor of The Predictive Analytics Times, makes the how and why of predictive analytics understandable and captivating. He is the author of the award-winning Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie, or Die, a former Columbia University professor who used to sing to his students, and a renowned speaker, educator, and leader in the field. Eric has appeared on Al Jazeera America, Bloomberg TV and Radio, Business News Network (Canada), Fox News, Israel National Radio, NPR Marketplace, Radio National (Australia), and TheStreet. He and his book have been featured in Businessweek, CBS MoneyWatch, The Financial Times, Forbes, Forrester, Fortune, Harvard Business Review, The Huffington Post, The New York Review of Books, Newsweek, The Seattle Post-Intelligencer, The Wall Street Journal, The Washington Post, and WSJ MarketWatch.
 |
 |
 |
