Machine Learning Times
Machine Learning Times
EXCLUSIVE HIGHLIGHTS
Podcast: Four Things the Machine Learning Industry Must Learn from Self-Driving Cars
    Welcome to the next episode of The Machine...
A Refresher on Continuous Versus Discrete Input Variables
 How many times have I heard that the most...
Podcast: Why Deep Learning Could Expedite the Next AI Winter
  Welcome to the next episode of The Machine Learning...
PAW Preview Video: Evan Wimpey, Director of Strategic Analytics at Elder Research
 In anticipation of his upcoming presentation at Deep Learning...
SHARE THIS:

1 year ago
How Image Search Works at Dropbox

 
Originally posted in Dropbox.tech, May 11, 2021

Photos are among the most common types of files in Dropbox, but searching for them by filename is even less productive than it is for text-based files.  When you’re looking for that photo from a picnic a few years ago, you surely don’t remember that the filename set by your camera was 2017-07-04 12.37.54.jpg.

Instead, you look at individual photos, or thumbnails of them, and try to identify objects or aspects that match what you’re searching for—whether that’s to recover a photo you’ve stored, or perhaps discover the perfect shot for a new campaign in your company’s archives.  Wouldn’t it be great if Dropbox could pore through all those images for you instead, and call out those which best match a few descriptive words that you dictated? That’s pretty much what our image search does.

In this post we’ll describe the core idea behind our image content search method, based on techniques from machine learning, then discuss how we built a performant implementation on Dropbox’s existing search infrastructure.

1. Our approach

Here’s a simple way to state the image search problem: find a relevance function that takes a (text) query q and an image j, and returns a relevance score s indicating how well the image matches the query.

s = f(q, j)

Given this function, when a user does a search we run it on all their images and return those that produce a score above a threshold, sorted by their scores.  We build this function using two key developments in machine learning: accurate image classification and word vectors.

To continue reading this article, click here.

Leave a Reply