There are multiple techniques are
being used in modern recommendation systems. Different types of algorithms are
also used in implementing those systems. When implementing recommendation
systems data scientists and developers have used several approaches such as
Knowledge-Based Recommendation systems, Demographic Recommendation Systems,
Collaborative Filtering, Content-Based Filtering and Hybrid Recommendation
systems. Among those approaches last three are very famous in modern world.
Knowledge-based recommender systems
incorporate inferences about users’ preferences and needs to make the
recommendation. The knowledge may sometimes contain explicit information about
how certain characteristics of a product meets user’s requirements sh1. In other words, such systems are based on the
knowledge about user preferences, item groupings, and recommendation criteria
and are applied in situations where other approaches (e.g. collaborative or
content-based filtering) cannot be employed.
This type of recommender systems employs demographic profile
of users/items for making recommendations. Collecting the accurate data in
large-scale is one of the issues in the mentioned recommender systems where
data are mainly collected and integrated manually from the users or from the
product catalogues. Sometimes, such systems also employ a pre-generated
demographic clustering of users/items sh6.
Collaborative filtering is based on the nature of humans, that is people
share ideas and opinions and explicitly or implicitly influence each other’s
decisions. In collaborative filtering, the opinions of other (similar) people
are evaluated to infer the required information.
For this purpose, the interests
of users are predicted through collecting related information from large number
of similar users. This method is based on the assumption that those people who
agreed in the past will more likely continue to agree in the future. In
real-life problems (e.g. e-commerce), systems that are based on the
collaborative filtering method should process a huge amount of information
which apart from being resource-consuming, it might also harm the systems’
responsiveness. However, since collaborative filtering method has been in the
core of researchers’ attention during the last decade, it is now considered as
one of the prominent personalization techniques in recommendation.
As one of the fundamental approaches to recommendation, collaborative
filtering has several forms. In user-based collaborative filtering, the
recommendation is made to a user through finding other similar users and
suggesting the things that they like sh8, sh9.
In another approach, named as item-based collaborative filtering, the system
finds and recommends similar items to those the user has already bought or
liked. Hence, item-based algorithms can be considered as the transpose of the
To find the similarity between users, a user-item preference matrix
should be created. Such matrix can be of huge size in complex problems. Matrix
factorization methods take the user-item matrix and decompose it into a more
compact representation which is used to predict the preference of the items
that a given user has not seen yet or to prioritize among a set of seen items.
One of the common techniques in matrix factorization is called singular value
decomposition or SVD. Gradient descent is mainly
used for factorizing the matrix in a computationally efficient way; however, it
does not contain all the properties of proper singular value decomposition sh10. Other common techniques include factor
analysis sh11 and eigenvalue decomposition sh12. After creating the preference matrix,
collaborative filtering method calculates a similarity measure between the
given user’s profile and the other users. Various distance-based or
correlation-based metrics can be used to calculate the profile similarity
measure, e.g. Euclidean distance, Pearson correlation and cosine similarity to
name the few. The final output will be a single item or a list of items (top n
recommended items) which the given user will most likely be interested sh13.
Collaborative filtering techniques suffer from two main problems, namely
sparsity and the first-rater problem. Sparsity problem arises as most users
only rate a very small portion of the available items, making it very difficult
to find similar users. The second problem is caused by the fact that an item
cannot be recommended unless it has received a user’s rating. A special case
would be newly added items which will be biased by such recommendation method sh14. Despite the mentioned drawbacks, collaborative
filtering can provide the user with fitting recommendations which do not share
almost anything with the items previously rated by the user. Thus, this method
has been proven successful in a wide variety of domains and applications.
Content-based filtering is another common approach for designing
recommender systems which are based on items’ descriptions and users’
preferences. Such systems mainly use a set of keywords to describe the items
while in parallel a profile is built for users indicating the type of items
they prefer. Hence, content-based filtering recommends items to a given user
similar to the ones the user liked in the past. Content-based methods have been
proven to perform well in text-intensive domains/applications where there is
enough content associated with the items. However, this approach requires
significant knowledge engineering efforts to provide/collect the needed
metadata about the items sh13. In
content-based systems, users’ profiles are made independently from each other.
For example, even if two categories of items are frequently agreed together by
users, the system will never recommend items from a category if the user has
not rated any items from that category. Therefore, a critical issue with
content-based filtering is their ability to learn user preferences from users’
actions and behavior.
In hybrid recommender systems, two
or more different recommender algorithms/techniques/approaches are combined,
creating a composite recommender. The main purpose in applications is to use
hybrid recommenders (of various types) which outperform the individual
algorithms. One of the most important advantages of the hybrid recommender
systems is the inclusion of algorithms that cover different aspects of the data
and the subject problem in order to produce improved recommendations. For
example, as seen before, one of the limitations of the item-based recommender
systems is when there is no rating for an item. A hybrid recommender system can
use text similarity measures to relate the new item with the existing items and
increase the accuracy of the collaborative filtering method as users start
rating the item.
Combining collaborative and
content-based filtering methods in a hybrid framework has been considered by
researchers recently where various approaches have been applied. Some examples
are: using collaborative and content-based filtering separately and then
combining their predictions, using collaborative filtering as the basis and
adding content-based capabilities to the system, or incorporating both
approaches in one model sh15. Netflix, an
American provider of Internet streaming media, is a famous example of the
success of hybrid systems. Netflix analyzes the searching and watching preferences
and habits of similar users (collaborative filtering) and recommends movies
which have similar features as the ones the user has already highly rated
There exist a variety of studies
combining various techniques to design a hybrid recommender system for
different applications. For example, Melville and his colleagues proposed a
hybrid system, named content-boosted collaborative filtering sh14. They first employ content-based predictions
and convert a sparse user rating matrix to a full user rating matrix.
Collaborative filtering is then applied on the created matrix to make the
recommendations. Their results show that the performance of the content-boosted
collaborative filtering overcomes the recommender systems which use either
content-based or collaborative filtering approaches.
2.4.2 Common Algorithms in Recommendation
In this section, number of common
algorithms that are widely used in recommender systems are discussed. Among
those k-Nearest-Neighbors and Matrix Factorization are ones of the most common
the methods which have been used in both academia and industry.
Association rules are a well-known
technique in data mining, e.g. the Apriori algorithm. Such techniques learn
from the data and extract rules which predict the occurrence of an item
based on the other items’ occurrences. There are also number of studies which
employed association rules in the context of recommender systems. However,
association rules need to be adopted according to the application of the
recommender system. That is in recommender systems any item can be recommended,
thus the association rules should be able to capture the associations among any
items, even if the support is small. Here, it becomes challenging since setting
a small threshold for the support can lead to a large set of associations!
There exist some heuristic approaches, such as adaptive support sh16 and sliding windows, to overcome the mentioned
In Bayesian network classifiers all
features are considered as random continuous or discrete variables. Bayes
theorem and conditional probabilities are used in Bayesian classifiers to
classify the given data through maximizing the posterior probability of the
items’ class. In recommenders’ context, ratings can be considered as classes
and the Bayesian classifier can be applied on the real-valued ratings. It is
assumed that given a class (e.g. rating), users (or items) are independent, and
thus the probability of the class is calculated. However, it is obvious that
such assumption is not satisfied in collaborative filtering where it is assumed
that users and/or items are related. For this reason, Bayesian classifiers are
mostly coupled with another algorithm in recommender systems. For example,
coupled a Bayesian model with a k-Nearest-Neighbors algorithm sh17.
Neural Networks (NN) are widely
used in computer science in a variety of applications. Despite the large
coverage of NN algorithms in machine learning, they are not so common in
recommender systems due to several reasons such as their low convergence speed
in high dimensional problems, or the fact that most of the time there is no
need to employ NN as complex non-linear classifiers in recommenders. Another
drawback is about the black box effect of NN algorithms such that the output of
the system might be hard to interpret. However, some instances of NN algorithms
were successfully used in recommender systems, e.g. fast matrix factorization
K-Nearest-Neighbor (KNN) based
algorithms, also called memory-based algorithms, are widely used in the context
of recommender systems. They can be considered as a generalization to the
association rules as they go over all the items and/or users in the corpus. One
of the serious limitations KNN approaches is their lack of scalability.
Moreover, they might be time consuming in large-scale real-life applications as
the time needed for building the model is quadratic, i.e. a function of squared
number of objects in the corpus.
It was in Netflix challenge when
matrix factorization techniques became very popular. Matrix factorization
algorithms are not only fast and accurate but also relatively easy to
implement. However, they might be difficult to be adopted for item-item recommendations.
In general, such techniques transform a given matrix into typically three
simpler matrices. In recommender systems, matrix factorization techniques
should deal with the missing values problem. The first matrix factorization
approaches handled the missing value problem through replacing the missing
values of the rating matrix, which was not an effective way since it resulted
in large dense matrices. Another more effective approach is to use parameters
and regularization sh18.
There exists a wide range of
machine learning algorithms and techniques that are being used in data mining
projects and recommender systems. For example, Latent Semantic Analysis is
closely related to matrix factorization technique and is used for document representation.
Other techniques that are worth mentioning include: Markov Decision Process,
Boltzmann machines, and random walk systems sh19.