I'm trying to train a neural network using this sort of data (for a homework): Number of Features : 42, Target data : 0 or 1, Number of Samples : 111 Individuals ( 69 Cases + 42 Controls )

However i'm facing an over-learning issue (the learning curve approaches the goal set, however the validation curve shows no improvement): the learning stops after less than 15 iterations because of a great number of validation fails: no improvement in the quality of validation tests. I managed to have great results by using 444 (4*111) examples in my training as I repeated the initial data four times. but i'm not sure if this is clean. In other words, is it alright to repeat existing examples to have more values for training ?

Thanks in advance.

  • 1
    $\begingroup$ I'm not sure this question is answerable as it stands. What is a "validation fail"? I'm not familiar with what you mean by "learning performance" or "regression = 1"; would someone who works in this field know what that means? Also please state a question explicitly. It sounds like you are trying to pack two different questions into one ('how do I make learning not stop?' and 'is it OK to repeat training data points?') -- please stick to one question per question. I suggest that you edit your question to fix all of these issues. $\endgroup$ – D.W. May 6 '15 at 18:03
  • $\begingroup$ Once you've fixed those issues... this question might be more likely to get useful answers on Stats.SE. Don't re-post/cross-post it there, but if you'd prefer to see your question over there, feel free to flag it for moderator attention and ask the moderators to migrate it -- but I encourage you to address my comments above first. $\endgroup$ – D.W. May 6 '15 at 18:04
  • $\begingroup$ Well, i use Matlab's Neural Network Toolbox, so I guess that people who use it for make NN are familiar with those terms. Anway, what I mean when I say validation fail is that the output that the NN predicted after his learning is not the one that he should have predicted. $\endgroup$ – ryuzakinho May 6 '15 at 21:56
  • $\begingroup$ If your question is how to use a specific piece of software (e.g., it is specific to Matlab), you should mention the software package in the question and ideally define the meaning of those terms. If the language is generic and applies to all uses of neural networks regardless of software package, then this is not an issue. It looks to me like your question is about how to train neural networks in general, at the conceptual question, rather than anything specific to Matlab, so using generally-understood language increases the chances someone will be able to help you. $\endgroup$ – D.W. May 7 '15 at 0:08
  • $\begingroup$ I modified my question a little bit. I hope it is good enough now. $\endgroup$ – ryuzakinho May 8 '15 at 11:16

Yes, it is alright to repeat training data, as long as each one is repeated an equal number of times: this does not violate assumptions. That said, it smells to me like a kludge. If you have no theoretical understanding of why the approach was failing without that adjustment, then just randomly tweaking things (like repeating points) in hopes that it will happen to succeed does not seem like the right approach.

Be careful that you don't do cross-validation on the result after repeating samples. You need to separate the test set from the training set before repeating. If you repeat samples, then do cross-validation, there is a high likelihood that most or all samples in the test set will also appear in the training set (due to the repetition) and thus you'll be over-fitting -- the cross-validation results won't be meaningful in that case.

  • $\begingroup$ Thanks for the answer, In my case I beleive it was the lack of data that caused the problem. In general,what are the reasons of over-learning ? $\endgroup$ – ryuzakinho May 9 '15 at 14:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.