Machine Learning Times
Machine Learning Times
EXCLUSIVE HIGHLIGHTS
Guidebook to the Future of Data Science: How to Navigate the Increasingly Symbiotic Dynamic Between Executives and Universities
 Book Review of Closing the Analytics Talent Gap: An...
Guilty or Not Guilty: Weight of Evidence
 You have been invited to serve as a juror...
How Machine Learning Works for Social Good
  Originally published in KDnuggets, Nov 2020. This article...
Diversity and Collaborative Problem Solving to Address Wicked Data Ethics Problems
 The complexity of the ethical issues facing professionals who...
SHARE THIS:

4 weeks ago
Machine Learning Models are Missing Contracts

 

Why pretrained machine learning models are often unusable and irreproducible — and what we can do about it.

Introduction

A useful approach to designing software is through contracts. For every function in your codebase, you start by writing its contract: clearly specifying what inputs are expected and valid for that function (the precondition), and what the function will do (the postcondition) when provided an appropriate input. This is often explicitly stated in the docstring of a function. Consider this example from the math module in Python (implemented in C):

/* Approximate square root of a large 64-bit integer.
Given `n` satisfying `2**62 <= n < 2**64`, return `a`
satisfying `(a - 1)**2 < n < (a + 1)**2`. */

static uint64_t
_approximate_isqrt(uint64_t n)
{
uint32_t u = 1U + (n >> 62);
u = (u << 1) + (n >> 59) / u;
u = (u << 3) + (n >> 53) / u;
u = (u << 7) + (n >> 41) / u;
return (u << 15) + (n >> 17) / u;
}

The contract in the docstring has two parts:

  • Precondition: input should be an integer between 2⁶² and 2⁶⁴
  • Postcondition: output is an integer within 1 of the square root of the input

The contract is powerful because when the code is published, other developers do not need to test the function themselves, nor consider its internal implementation. They can read off the range of valid inputs for the function and start using it immediately. Conversely, they operate knowing that if the precondition is not satisfied, then neither is the postcondition guaranteed.

Nowadays, pretrained machine learning models are increasingly being deployed as functions and APIs. They are part of companies’ internal codebases [1], released externally for use through APIs [2], and, in research, pretrained models are published as part of the review and reproducibility processes [3].

To continue reading this article, click here.