{"id":5097,"date":"2015-03-23T17:27:36","date_gmt":"2015-03-23T17:27:36","guid":{"rendered":"http:\/\/www.predictiveanalyticsworld.com\/machinelearningtimes\/?p=5097"},"modified":"2015-03-23T17:27:50","modified_gmt":"2015-03-23T17:27:50","slug":"failing-optimally-data-sciences-measurement-problem-032215","status":"publish","type":"post","link":"https:\/\/www.predictiveanalyticsworld.com\/machinelearningtimes\/failing-optimally-data-sciences-measurement-problem-032215\/5097\/","title":{"rendered":"Failing Optimally \u2013 Data Science\u2019s Measurement Problem"},"content":{"rendered":"Data science has a measurement problem. Simple metrics may not address complex situations. But complex metrics present myriad problems. What is the best way to quantify the quality of your classification algorithm? As we strive for better algorithms, we often fail to think critically about what it means for predictions to be &#8220;good&#8221;. Evaluation methodology and decision theory are tightly linked. Often a classifier outputs numerical scores but a performance metric takes binary predictions. In these cases, any such metric specifies an optimal decision rule to convert scores to predictions. Thus in many settings, it would seem data <a href=\"https:\/\/www.predictiveanalyticsworld.com\/machinelearningtimes\/failing-optimally-data-sciences-measurement-problem-032215\/5097\/\" class=\"more-link\">(more&hellip;)<\/a>","protected":false},"excerpt":{"rendered":"<p>Data science has a measurement problem. Simple metrics may not address complex situations. But complex metrics present myriad problems. What is the best way to quantify the quality of your classification algorithm? As we strive for better algorithms, we often fail to think critically about what it means for predictions to be &#8220;good&#8221;. Evaluation methodology [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":"","_links_to":"","_links_to_target":""},"categories":[11,48],"tags":[],"class_list":["post-5097","post","type-post","status-publish","format-standard","hentry","category-industry-news","category-left-hand"],"_links":{"self":[{"href":"https:\/\/www.predictiveanalyticsworld.com\/machinelearningtimes\/wp-json\/wp\/v2\/posts\/5097","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.predictiveanalyticsworld.com\/machinelearningtimes\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.predictiveanalyticsworld.com\/machinelearningtimes\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.predictiveanalyticsworld.com\/machinelearningtimes\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.predictiveanalyticsworld.com\/machinelearningtimes\/wp-json\/wp\/v2\/comments?post=5097"}],"version-history":[{"count":1,"href":"https:\/\/www.predictiveanalyticsworld.com\/machinelearningtimes\/wp-json\/wp\/v2\/posts\/5097\/revisions"}],"predecessor-version":[{"id":5098,"href":"https:\/\/www.predictiveanalyticsworld.com\/machinelearningtimes\/wp-json\/wp\/v2\/posts\/5097\/revisions\/5098"}],"wp:attachment":[{"href":"https:\/\/www.predictiveanalyticsworld.com\/machinelearningtimes\/wp-json\/wp\/v2\/media?parent=5097"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.predictiveanalyticsworld.com\/machinelearningtimes\/wp-json\/wp\/v2\/categories?post=5097"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.predictiveanalyticsworld.com\/machinelearningtimes\/wp-json\/wp\/v2\/tags?post=5097"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}