Machine Learning Department
School of Computer Science, Carnegie Mellon University
Relational Learning via
Collective Matrix Factorization
Ajit P. Singh, Geoffrey J. Gordon
Keywords: Matrix factorization, relational learning, stochastic
Relational learning is concerned with predicting unknown values of a relation,
given a database of entities and observed relations among entities. An
example of relational learning is movie rating prediction, where
entities could include users, movies, genres, and actors. Relations would
then encode users's ratings of movies, movies' genres, and actors' roles
in movies. A common prediction technique given one pairwise relation, for
example a #users x #movies ratings matrix, is low-rank matrix factorization.
In domains with multiple relations, represented as multiple matrices, we may
improve predictive accuracy by exploiting information
from one relation while predicting another. To this end, we propose a
collective matrix factorization model: we simultaneously factor several
matrices, sharing parameters among factors when an entity participates in
multiple relations. Each relation can have a different value type and error
distribution; so, we allow nonlinear relationships between the parameters
and outputs, using Bregman divergences to measure error. We extend
standard alternating projection algorithms to our model, and derive an
efficient Newton update for the projection. Furthermore, we propose
stochastic optimization methods to deal with large, sparse matrices.
Our model generalizes several existing matrix factorization methods, and
therefore yields new large-scale optimization algorithms for these problems.
Our model can handle any pairwise relational schema and a wide variety of
error models. We demonstrate its efficiency, as well as the benefit of
sharing parameters among relations.