Overview

  • Recommendation Systems (RecSys) has been a pivotal field in Machine Learning and has been an active ingredient in products offerings from the who’s who of tech (think Google, Facebook, Amazon, Netflix, etc.).
    • RecSys is an integral part of industries right from food delivery to sales driven by the recommendations coming from these models. Not only do they help businesses grow, but they also help individuals discover things they like that they may not have encountered before.
    • The following figure shows results from pilot customer A/B experiments, showing improvements to business metrics (such as revenue, conversions, click-through rate, etc) compared to their previous recommendation systems.

  • Recommendations are ML models that determine a users preference based on the users past history, or by looking at other similar users preferences.
  • These systems have documents (entities a system recommends, like movie, videos etc), queries (the information we need to make recommendation, like user info, location etc.), and embeddings ( a mapping from a set (of queries or documents to recommend) to a vector space called embedding space)).
  • Here is a common architecture for a recommendation systems model:
    • Candidate Generation: This is where the model starts from. It has a huge corpus and generates a smaller subset of candidates.
      • For e.g., Netflix has thousands of movies, but it has to pick which ones to show you on your home page!
      • The model needs to evaluate the queries at a rapid speed on a large corpus and keep in mind, a model may provide multiple candidate generators which each having different candidate subsets.
    • Scoring: This is the next step in our architecture. Here, we simply rank the candidates in order to select a set of documents.
      • Here the model runs on a smaller subset of documents, so we can aim for higher precision in terms of ranking.
    • Re-ranking: The final step in our model is to re-rank the recommendations.
      • For e.g., If the user explicitly dislikes some movies, Netflix will need to move those out of the main page.
  • Many large scale recommendation systems are implemented in two steps: retrieval and ranking.
  • “Consider matrix factorization, where the model learns a user and an item embedding (in the case of collaborative filtering). If the embedding dimension is 100, you have 100 million users and 10 million items.
  • The total number of embeddings is 110 million. Each embedding has 100 learnable parameters. Hence the model has 110*100 million or ~11 billion parameters. However, to compute scores for a user, you need to access just one of the 100 million user embeddings at a time. This particular user embedding is used along with all the item embeddings to score all the items. Hence, recommendation models are memory intensive but compute light.” source

Retrieval

  • Retrieving more items will result in better results but will slow computations for recommendations.
  • We should carry out offline experiments to see if retrieving additional items results in more relevant recommendations.

Ranking

  • Ranking is a core task in recommender systems, which aims at providing an ordered list of items to users.
  • A ranking function is learned from the labeled dataset to optimize the global performance, which produces a ranking score for each individual item.
  • The quality of the ranked list given by a ranking algorithm has a great impact on users’ satisfaction as well as the revenue of the recommender systems.

  • Lastly, we’ll do a code deep-dive of a Music Recommendation model build using PySpark so be sure to check that out!

References