I've set up an RNN LSTM network in Java using DL4J as the library.
I currently have 500 examples of positive text, and 500 examples of negative text.
When I fitness the training data by first training all the negatives and then all the positives, my predictions only favor high positive responses even for things that would be considered negative in my training examples.
And if I reverse it and train positive first and then train negative last everything is favored as a negative in high 80 - 90%.
However, when I train in an oscillating pattern such as: Positive, Negative, Positive, Negative and so on, the predictions become accurate again and similar examples of negative text are picked up relatively accurately and vice versa for positive text.
I've only just started studying machine learning in general but I can't quite find any resources explaining why the sequence of training impacted my model so hugely. Is this normally how an LSTM network should behave or is oscillating the training data the correct approach?
Would the classification of a messages negative intent be something an LSTM RNN network should be used for or should I consider another network?