0
$\begingroup$

I recently started to learn about machine learning. I have created a feed forward neural network (ffnn) and a recurrent neural network (rnn) to predict user ratings of movies. I am using a subset of 2000 users and their ratings of the "Netflix Prize" dataset.

The ffnn as well as the rrn have an accuracy of ~40% - 45% on the test set evaluation. This seems to be very low and I expected to get at least somewhere near 60% - 70% accuracy. I tried different network configurations (dimensions, layers, optimizers, etc.) but nothing changed the accuracy significantly (only 1% - 3% max).

Both models are constructed in a sense of supervised learning. The ffnn uses embeddings of users+movies for the input and ratings for the output. For the rrn I am using one hot encoded movie vectors as input and one hot encoded ratings vectors as the output.

For the implementation I am using Keras in Python.

The ffnn is constructed like this:

dimension = 120
model_users = Sequential()
model_users.add(Embedding(len(np.unique(users)), dimension))
model_users.add(Reshape((dimension,)))

model_movies = Sequential()
model_movies.add(Embedding(len(np.unique(movies)), dimension, input_length=1))
model_movies.add(Reshape((dimension,)))

model = Sequential()
model.add(Merge([model_users, model_movies], mode = 'concat'))
model.add(Dropout(0.1))
model.add(Dense(100, activation = 'relu'))
model.add(Dropout(0.1))
model.add(Dense(500, activation = 'sigmoid'))
model.add(Dropout(0.1))
model.add(Dense(dimension, activation = 'linear'))
model.add(Dropout(0.1))
model.add(Dense(5, activation = 'softmax'))
model.compile(loss = 'categorical_crossentropy', optimizer = 'adam', metrics=['accuracy'])
print(model.summary())

The rnn model is constructed like this:

dimensions = len(movies_unique)

model = Sequential()
model.add(Masking(mask_value = 0, input_shape = (dimensions, dimensions)))
model.add(LSTM(32, return_sequences = True))
model.add(TimeDistributed(Dense(len(ratings_unique), activation = 'relu')))
model.add(Activation('softmax'))
model.compile(loss = 'mse', optimizer = 'adam', metrics = ['accuracy'])
print(model.summary())

How can I further improve the accuracy to get beyond ~45%? I might be missing something fundamental here, so any help is appreciated! :)

Best, Nico

$\endgroup$
  • $\begingroup$ I don't understand the input encoding or output encoding. How do you deal with missing data? (e.g., user A has rated movie A but not movie B; and user B has rated movie B but not movie A?) How is that encoded in your input encoding? We're not a coding site, so please don't force us to understand Keras code to be able to understand what you're asking or what you're doing. Thanks! $\endgroup$ – D.W. Feb 21 '18 at 17:19
  • $\begingroup$ Predicting movie ratings is hard. Why do you expect 60-70% accuracy? $\endgroup$ – D.W. Feb 21 '18 at 17:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.