Basic knowledge

https://developers.google.com/machine-learning/crash-course/embeddings/video-lecture

A good one : explain embedding and recommendation system

https://developers.google.com/machine-learning/recommendation/dnn/re-ranking

Similarity Measures

• cosine
• dot product
• Euclidean distance

what is Matrix factorization

Matrix factorization is a mathematical technique used in linear algebra and data analysis to decompose a matrix into the product of two or more smaller matrices. The primary goal of matrix factorization is to represent the original matrix in a more compact form, capturing the underlying patterns or latent factors within the data. This technique is widely used in various fields, including recommendation systems, data compression, and dimensionality reduction. Here are some key aspects of matrix factorization:

1. Basic Idea:
   • Given an original matrix, typically a data matrix, matrix factorization seeks to find two or more matrices whose product approximates the original matrix as closely as possible.
2. Matrix Decomposition:
   • Matrix factorization decomposes the original matrix (usually denoted as A) into two or more matrices, such as A = B * C, where B and C are the factorized matrices.
3. Applications:
   • Recommendation Systems: Matrix factorization is commonly used in collaborative filtering for recommendations. It decomposes a user-item interaction matrix into user and item matrices to predict missing values (user-item ratings) and make personalized recommendations.
   • Dimensionality Reduction: Matrix factorization can be used for reducing the dimensionality of data. For example, in Principal Component Analysis (PCA), it decomposes a data covariance matrix into eigenvectors and eigenvalues.
   • Data Compression: In image compression, matrix factorization can be used to represent images more efficiently by decomposing them into a combination of basis images or patterns.
4. Latent Factors:
   • The factorized matrices often represent latent factors or features that are not explicitly present in the original data. In recommendation systems, these factors can represent user preferences and item characteristics.
5. Optimization:
   • Matrix factorization is typically performed using optimization algorithms that minimize the difference between the original matrix and the product of the factorized matrices. Common optimization methods include gradient descent, alternating least squares (ALS), and singular value decomposition (SVD).
6. Regularization:
   • To prevent overfitting and improve generalization, regularization terms are often added to the optimization objective. Regularization can help control the complexity of the factorized matrices.
7. Non-Negative Matrix Factorization (NMF):
   • In some cases, matrix factorization is constrained to ensure that the factorized matrices contain only non-negative values. NMF is commonly used for data decomposition and topic modeling.
8. Sparse Matrix Factorization:
   • When dealing with sparse data matrices (many missing values), specialized techniques are used to perform matrix factorization efficiently while handling sparsity.

Matrix factorization is a versatile technique that has found applications in various domains, and it forms the basis for many advanced machine learning and data analysis methods. Depending on the specific problem, different factorization algorithms and constraints may be used to obtain meaningful representations of the data.

https://colab.research.google.com/github/google/eng-edu/blob/main/ml/recommendation-systems/recommendation-systems.ipynb?utm_source=ss-recommendation-systems&utm_campaign=colab-external&utm_medium=referral&utm_content=recommendation-systems#scrollTo=k0IFBGCx8_im

Design a recommendation system

Designing a recommendation system involves several steps and considerations. Below is a high-level overview of the process:

1. Define the Objective:
   • Clearly define the goal of your recommendation system. Are you building a movie recommendation system, a product recommendation system for an e-commerce website, or something else? Understanding the objective is crucial for the design process.
2. Data Collection and Preprocessing:
   • Gather the data you need for building recommendations. This may include user data, item data, user-item interaction data (such as ratings, reviews, or purchase history), and contextual information (if relevant). Clean and preprocess the data to handle missing values and outliers.
3. Choose a Recommendation Approach:
   • There are several recommendation approaches to consider:
     • Collaborative Filtering: This method makes recommendations based on user behavior and preferences. It can be user-based (finding users with similar preferences) or item-based (finding items similar to those the user has interacted with).
     • Content-Based Filtering: This approach recommends items based on their attributes and user profiles. It relies on feature extraction and matching user preferences to item features.
     • Hybrid Approaches: Combining collaborative and content-based filtering techniques often leads to more accurate recommendations.
     • Matrix Factorization: Techniques like Singular Value Decomposition (SVD) and matrix factorization can be used to uncover latent factors in the data for recommendations.
     • Deep Learning: Neural networks, including models like Neural Collaborative Filtering (NCF) and Recurrent Neural Networks (RNNs), can be employed for recommendation tasks.
4. Feature Engineering:
   • If you’re using content-based filtering or deep learning, you’ll need to engineer relevant features from your data. This might involve natural language processing (NLP) for text data, image analysis for images, or other techniques based on your data types.
5. Split Data for Evaluation:
   • Split your data into training, validation, and test sets. This allows you to train and evaluate your recommendation system’s performance.
6. Model Training:
   • Train the chosen recommendation model(s) on your training data. Depending on the complexity of your model, this step might require significant computational resources.
7. Hyperparameter Tuning:
   • Optimize the hyperparameters of your recommendation model(s) to achieve the best performance on the validation set. This may involve techniques like grid search or Bayesian optimization.
8. Evaluation Metrics:
   • Choose appropriate evaluation metrics for your recommendation system. Common metrics include Mean Absolute Error (MAE), Root Mean Square Error (RMSE), Precision, Recall, F1-score, and others. The choice of metrics depends on your specific use case.
9. Deployment and Integration:
   • Deploy your recommendation system in a production environment. Integrate it with your application or platform so that it can provide real-time recommendations to users.
10. Monitoring and Maintenance:
    • Continuously monitor the performance of your recommendation system in production. Collect user feedback and usage data to refine the recommendations over time. Periodically retrain the model with new data.
11. Privacy and Ethics:
    • Consider privacy and ethical concerns when designing your recommendation system, especially if you’re dealing with sensitive user data. Implement user consent mechanisms and anonymize data where necessary.
12. Scale and Scalability:
    • Ensure that your recommendation system can handle growing datasets and user bases. Consider scalability and resource constraints when deploying and maintaining the system.
13. A/B Testing:
    • Implement A/B testing to evaluate the impact of recommendations on user engagement, conversion rates, or other key performance indicators. This allows you to fine-tune the recommendation algorithms and strategies.
14. Feedback Loops:
    • Implement feedback loops that incorporate user interactions and feedback into the recommendation model. This can lead to more personalized and accurate recommendations over time.

Remember that the design of a recommendation system is an iterative process, and it may require ongoing refinement and adaptation to changing user preferences and business goals.

On Matrix Factorization Vs. Softmax

DNN models solve many limitations of Matrix Factorization, but are typically more expensive to train and query. The table below summarizes some of the important differences between the two models.

In summary:

• Matrix factorization is usually the better choice for large corpora. It is easier to scale, cheaper to query, and less prone to folding.
• DNN models can better capture personalized preferences, but are harder to train and more expensive to query. DNN models are preferable to matrix factorization for scoring because DNN models can use more features to better capture relevance. Also, it is usually acceptable for DNN models to fold, since you mostly care about ranking a pre-filtered set of candidates assumed to be relevant.

References

https://www.nvidia.com/en-us/glossary/data-science/recommendation-system/#:~:text=A%20recommendation%20system%20is%20an,demographic%20information%2C%20and%20other%20factors.c

https://developers.google.com/machine-learning/recommendation/overview/types