If I split my data properly into 75% train, 15% test, and 15% validation, and there are over 100,000 samples, is it appropriate for me to train 100s of neural networks then select only a couple based on their testing set grades for publication in a research article?
The question is for neural network training in which the random initialization of weights can drastically affect test grades. Network parameters like the number of neurons, number of layers, etc all can change how the testing grade is. These are all things that are easy to modify, rather than data preparation which can take a long time.
Do I need to resplit my data every time I train a new network? What strategy do most NN theorists use?