View on GitHub

Tutorial on Replication of Recommender Systems Research

2017 ACM RecSys Summer School

This is a supplementary website for the tutorial on Replication of Recommender Systems Research held at the ACM RecSys Summer School.

Preparation

Before attending the tuturial, make sure to prepare the following:

  1. Download and familiarize yourself with Lenskit, Librec, or RankSys.
  2. Download and extract the Movielens 100k dataset.

During the hand-on session, you will be expected to be able to execute and evaluate a recommendation using your selected tool.

Slidedeck

Replication of Recommender Systems Research from Alan Said

Code examples

Check out the following github repository rsss2017. The repository contains a Maven project with two classes ControlledEvaluation.java and RankSysEvaluation.java.

  1. ControlledEvaluation performs a transparent, standalone, evaluation of RankSys, LensKit, and Mahout using RiVal. The evaluation metrics presented are fully comparable as the data splitting, candidate item generation, and metric calculation is done identically for all three cases.
  2. RankSysEvaluation performs the same type of evaluation, using RankSys’s internal evaluation methods. The evaluation metrics are only comparable with other evaluation made within RankSys with the same settings.

To build the project, do the following:

cd rsss2017/project
mvn clean install

The code consists of two classes: RankSysEvaluation.java one which runs RankSys’ knn recommender using Movielens100k and evaluates the generated recommendations using RankSys internal evaluations methods, and ControlledEvaluation.java which runs RankSys and Mahout and evaluates the generated recommendations using RiVal.

To execute the RankSysEvaluation class, execute the following commands:

mvn exec:java -Dexec.mainClass="net.recommenders.rsss2017.RankSysEvaluation"

To execute the ControlledEvaluation class, execute the following commands:

mvn exec:java -Dexec.mainClass="net.recommenders.rsss2017.ControlledEvaluation"

Instructors

The tutorial is given by Alejandro Bellogín and Alan Said.