Similarity-Based Explanations meet Matrix Factorization via Structure-Preserving Embeddings

Document Title

Similarity-Based Explanations Meet Matrix Factorization via Structure-Preserving Embeddings

Document Information

• Topic Area: Combining model interpretability and matrix factorization methods in recommendation systems
• Keywords: Recommendation systems, Matrix factorization, Model initialization, Similarity-based explanations, Structure preservation, Algorithm transparency

Research Background and Problem Statement

• Problem or Challenge:

  • Matrix Factorization (MF) methods, while being one of the mainstream models in recommendation systems, lack interpretability in their embedding vectors, which may reduce user trust in the recommendation results.
  • Many current explanation methods (e.g., KNN-based user/item collaborative filtering) are relatively simple, providing "similarity-based" explanations but often underperform compared to MF.
  • Existing studies often rely on explicit metadata or post-structured models for explanations, adding unnecessary complexity or computational costs.

• Importance:

  • Interpretability enhances user trust, transparency, and acceptance of recommendation results.
  • Facilitates better interaction between complex ML/AI algorithms and users.

• Research Motivation and Related Work:

  • Few studies focus on improving the similarity-based interpretability of MF models without significantly sacrificing performance.
  • The authors propose leveraging dimensionality reduction methods (e.g., PCA) to preserve data structure and enhance the interpretability of MF models.

Solution

• Core Method/Approach:

  • Propose a matrix factorization method combined with structure-preserving embeddings, where PCA is used to initialize embeddings to retain the structural characteristics of the original rating matrix.
  • Explore the benefits of structure-preserving embeddings in achieving KNN-style explanations while maintaining prediction accuracy.

• Innovations:

  • Unlike traditional MF methods, the proposed approach retains the similarity between users/items in the original rating matrix.
  • Introduce a regularization strategy that balances prediction performance and interpretability by freezing certain PCA-initialized embedding factors.
  • Improve the local optimal solutions of MF models, enhancing model performance.

• Implementation Steps and Key Techniques:

  1. Embedding Initialization:
     • Apply PCA to the rating matrix for dimensionality reduction, obtaining low-dimensional embeddings for users and items.
     • Freeze certain principal factors from PCA that preserve data structure, reducing training parameters to strengthen interpretability.
  2. Regularization Strategy:
     • Control the number of frozen principal factors to balance interpretability and prediction accuracy.
  3. Algorithm Implementation:
     • Optimize embeddings using stochastic gradient descent (SGD).
     • Provide multiple model variants: fully trained embeddings, freezing single embedding, or partial dimensions of both embeddings.
  4. Evaluation Scheme:
     • Use Mantel tests to measure whether embeddings preserve the distance relationships of the original matrix.
     • Measure interpretability by assessing neighborhood overlap compared to KNN recommendations.

Research Outcomes

• Specific Results:

  • After applying structure-preserving embeddings, traditional MF models showed significantly improved similarity-based interpretability, while prediction performance remained stable or improved.
  • Experiments on four datasets, including Movielens and Amazon, demonstrated significant improvements in neighborhood overlap metrics, with RMSE predictions remaining stable or better than baseline models.

• Advantages Over Existing Solutions:

  • Improved Interpretability: Relationships between similar users and items in the original rating matrix are better preserved in the embedding space.
  • Uncompromised Prediction Ability: The method controls embedding training dimensions while achieving better local optimal solutions than traditional MF.
  • Flexibility: Offers multiple model variants to adapt to different application scenarios (e.g., user or item explanations).

• Experimental or Evaluation Results:

  • On Movielens 100k and 1M datasets, the method's RMSE was comparable to complex deep learning-based methods (e.g., within 0.01 of GHRS and GLocal-K).
  • Across all experimental datasets, regardless of the proportion of frozen PCA embeddings, interpretability significantly outperformed standard MF methods.
  • Analysis of loss function optimization paths revealed that models initialized with PCA were more likely to find local optimal solutions.

• Limitations and Future Directions:

  • The method has only been tested on rating prediction tasks; future plans include extending it to recommendation ranking tasks.
  • Future work will explore nonlinear dimensionality reduction methods (e.g., manifold learning) to further optimize embedding initialization.
  • User studies will be conducted to evaluate the impact of enhanced explanations on user experience.