0
$\begingroup$

As the title suggests, I am curious whether Linear Discriminant Method can be performed on a dataset as pre-processing before putting the reduced-dimensional data as input for a neural network for creating the mapping function/ surrogate model.

Thanks!

$\endgroup$
3
  • $\begingroup$ What are your thoughts? What would the input to the neural network be? $\endgroup$ – D.W. Mar 21 at 2:33
  • $\begingroup$ Hi D.W. I believe the input to the neural network would be a smaller dataset consisting of less features based on LDA (linear discriminant analysis). I'm just unsure whether I would be doing the classification twice here (once with LDA as it considers output labels and again with the neural network afterwards) $\endgroup$ – Jeffery Mar 21 at 16:30
  • $\begingroup$ I suggest spending more time studying LDA. LDA doesn't output fewer features. $\endgroup$ – D.W. Mar 21 at 17:14
0
$\begingroup$

This is not a useful method. LDA is basically a two-class classifier, which works under certain assumptions on the distribution of the data. If those assumptions hold, LDA is optimal and there is no reason to use a neural network. If those assumptions don't hold, LDA might not be a useful tool. Even if you do use LDA, LDA maps each data point to one of two classes, so there is no information left for the neural network to do any further processing.

In short: LDA is not useful as a preprocessing step before a classifier; you either use LDA as the classifier itself, or you don't use LDA at all.

$\endgroup$
5
  • $\begingroup$ This is what I suspected. Thanks for the clarification. Would performing PCA on the input to reduce dimensionality before putting it in a neural network make more sense then? I'm just curious about methods for reducing input dimensionality for surrogate methods like NNs and SVM. $\endgroup$ – Jeffery Mar 21 at 17:16
  • $\begingroup$ @Jeffery, I think that's already been answered at: cs.stackexchange.com/users/132922/awillia91. If it doesn't make sense to use LDA, then almost anything else will make more sense. :-) In practice experience with using PCA before neural networks has not been too good from what I can tell. $\endgroup$ – D.W. Mar 21 at 17:19
  • $\begingroup$ I really appreciate your comments D.W. One more thing. According to this study, it seems LDA can be used with NN's successfully. towardsdatascience.com/… $\endgroup$ – Jeffery Mar 21 at 17:20
  • $\begingroup$ @Jeffery, if you wanted to know about that article, it would have helped to give that context in the question. Anyway, what that article is doing - classifying with LDA, and then using a neural network on the classification decision with LDA - is silly and makes no sense. I see no reason to do that, and indeed, in their experiments, it seems to perform the same or worse as just using LDA, which is exactly what I'd expect. Their conclusion that LDA is great for this seems bogus to me. People write all sorts of things on the Internet; not all of it is at the same level of quality. $\endgroup$ – D.W. Mar 21 at 17:23
  • $\begingroup$ My apologies D.W. The LDA is limited to linear mapping between input and output labels isnt it? Then shouldnt combining it with NN for problems that have nonlinear behaviour be a bit more useful? Just a thought. $\endgroup$ – Jeffery Mar 21 at 17:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.