I recently started to learn about machine learning. I have created a feed forward neural network (ffnn) and a recurrent neural network (rnn) to predict user ratings of movies. I am using a subset of 2000 users and their ratings of the "Netflix Prize" dataset.
The ffnn as well as the rrn have an accuracy of ~40% - 45% on the test set evaluation. This seems to be very low and I expected to get at least somewhere near 60% - 70% accuracy. I tried different network configurations (dimensions, layers, optimizers, etc.) but nothing changed the accuracy significantly (only 1% - 3% max).
Both models are constructed in a sense of supervised learning. The ffnn uses embeddings of users+movies for the input and ratings for the output. For the rrn I am using one hot encoded movie vectors as input and one hot encoded ratings vectors as the output.
For the implementation I am using Keras in Python.
The ffnn is constructed like this:
dimension = 120 model_users = Sequential() model_users.add(Embedding(len(np.unique(users)), dimension)) model_users.add(Reshape((dimension,))) model_movies = Sequential() model_movies.add(Embedding(len(np.unique(movies)), dimension, input_length=1)) model_movies.add(Reshape((dimension,))) model = Sequential() model.add(Merge([model_users, model_movies], mode = 'concat')) model.add(Dropout(0.1)) model.add(Dense(100, activation = 'relu')) model.add(Dropout(0.1)) model.add(Dense(500, activation = 'sigmoid')) model.add(Dropout(0.1)) model.add(Dense(dimension, activation = 'linear')) model.add(Dropout(0.1)) model.add(Dense(5, activation = 'softmax')) model.compile(loss = 'categorical_crossentropy', optimizer = 'adam', metrics=['accuracy']) print(model.summary())
The rnn model is constructed like this:
dimensions = len(movies_unique) model = Sequential() model.add(Masking(mask_value = 0, input_shape = (dimensions, dimensions))) model.add(LSTM(32, return_sequences = True)) model.add(TimeDistributed(Dense(len(ratings_unique), activation = 'relu'))) model.add(Activation('softmax')) model.compile(loss = 'mse', optimizer = 'adam', metrics = ['accuracy']) print(model.summary())
How can I further improve the accuracy to get beyond ~45%? I might be missing something fundamental here, so any help is appreciated! :)