ML for Ranking System in Search System: Comprehensive Overview

Learning to Rank

Ching (Chingis)

--

What Does Search Ranking Look Like?

The simplest Search System

This is an overview or simplified figure of what a typical search system might look like. First, a user provides a query, and a candidate set is built by some light and fast search methods, such as BM25, since the first layer of a search system might be dealing with millions of products which is a computationally costly task for DL. Next, top-N candidates a ‘ranked’ using Deep Learning techniques to refine the order of candidates. Ranking is a huge research area in the field and it might even consist of multiple stages of ranking, where each stage might have its own purpose regarding sorting the candidates.

What Is Reranking?

ML Ranking

In other words, the Ranking model (also ML model) is a mapping function that re-ranks or sorts a given list of documents based on their relevancy given some context/query. This task in Machine learning is referred to as Learning-to-Rank.

Learning to Rank (LTR)

  • Function F() can sort objects according to their degrees of relevance, preference, or importance given context/query.
  • Key component for search and recommendations.

Problem Definition

Suppose there’s a document set D = (d1, d2, d3, …) where each document di has relevance ri given some query q. The goal is to learn the parameters 0 (theta) for the ranker f_0 (theta) that would produce the desirable/optimal ranking R over a document set D:

  • R = (r1, r2, r3, …) where R1 ≥ R2 ≥ R3 ≥ …

where documents are ordered by their scores (descending):

--

--

Ching (Chingis)

I am a passionate student. I enjoy studying and sharing my knowledge. Follow me/Connect with me and join my journey.