Machine Learning Times
Machine Learning Times
Elon Musk Predicts Artificial General Intelligence In 2 Years. Here’s Why That’s Hype
 Originally published in Forbes, April 10, 2024 When OpenAI’s...
Survey: Machine Learning Projects Still Routinely Fail to Deploy
 Originally published in KDnuggets. Eric Siegel highlights the chronic...
Three Best Practices for Unilever’s Global Analytics Initiatives
    This article from Morgan Vawter, Global Vice...
Getting Machine Learning Projects from Idea to Execution
 Originally published in Harvard Business Review Machine learning might...

8 years ago
HBO Teaches You How to Avoid Bad Science


Do you know what p-hacking is? John Oliver – of HBO’s “Last Week Tonight” and formerly of “The Daily Show with Jon Stewart” – does.

It’s a tricky, potent analytical pitfall that’s gaining increased, deserving attention – across fields of science and even within Predictive Analytics Times articles and Predictive Analytics World sessions.

“An orange used car is least likely to be a lemon,” for example. That’s what was claimed by The Seattle Times, The Huffington Post, The New York Times, NPR, and The Wall Street Journal. But this discovery has since been debunked as inconclusive.

As data gets bigger, so does a common pitfall in the application of standard stats: Testing many predictors means taking many small risks of being fooled by randomness, adding up to one big risk.

John Elder calls this issue vast search.

It is also known as many other things, including multiple hypothesis testing, over-search, the look-elsewhere effect, the garden of forking paths, cherry-picking findings, and significance chasing.

Perhaps the catchiest name is p-hacking – and, believe it or not, John Oliver even used that particular term in his recent coverage of this hairy topic:

John Oliver covers p-hacking on “Last Week Tonight.”

I was pleased to see this important topic covered by a relatively mainstream outlet.

It also so happens to be the topic of my Predictive Analytics World keynote this year – which I’m only set to deliver once more in 2016, at PAW New York (October 23-27):

Keynote title: Weird Science – How to Know Your Predictive Discovery Is Not BS
Speaker: Eric Siegel, Founder, Predictive Analytics World
Event: Predictive Analytics World for Business, New York

In this keynote, I will cover the issue and provide guidance on tapping data’s potential without drawing false conclusions. Click here for more info

Also check out these two articles on the topic:

Write-up describing my keynote: How Stubby Datasets Can Lead to Predictive Analytics Snafus

Another article by me: Breakthrough – How to Avert Analytics’ Most Treacherous Pitfall

Author bio:

Eric Siegel, Ph.D., founder of the Predictive Analytics World conference series and executive editor of The Predictive Analytics Times, makes the how and why of predictive analytics understandable and captivating. He is the author of the award-winning Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie, or Die, a former Columbia University professor who used to sing to his students, and a renowned speaker, educator, and leader in the field. Eric has appeared on Al Jazeera America, Bloomberg TV and Radio, Business News Network (Canada), Fox News, Israel National Radio, NPR Marketplace, Radio National (Australia), and TheStreet. He and his book have been featured in Businessweek, CBS MoneyWatch, The Financial Times, Forbes, Forrester, Fortune, Harvard Business Review, The Huffington Post, The New York Review of Books, Newsweek, The Seattle Post-Intelligencer, The Wall Street Journal, The Washington Post, and WSJ MarketWatch.

Leave a Reply