I work on the topics of sparse coding (SC) and dictionary learning (DL), in which classification is not the main goal, but the discriminative power of the representation is one of the concerns. Therefore, i use classification accuracy in order to show how my method is more discriminative than other SC or DL approaches.

Also, i use typical datasets in the literature of SC which are image datasets such as Ex.YALE, Caltech101, AR-face and similars.

But so often i'm criticized (even harshly!) by the reviewers (at least one in each submission) on why i didn't compare to deep-networks or why my results are not better than the reported CNN-method in paperX.

Or even i got feedbacks like "The datasets are small sized, why not using ImageNet and similars?!"

The thing is that my method is supposed to be compared to its family of methods (SC-based baselines) and the point of my work is to improve the model/algorithm's performance in this domain and not compared to any top-classifier on any big-data on the earth! :((

  • 1
    $\begingroup$ I don't understand your question. Why wouldn't you want to compare to any other method that can achieve the same thing and, especially, to the best methods? As it is, it sounds like the reviewers are saying, "Your paper about using horses for transporting goods only compares with donkeys and men carrying sacks. You need to compare with trucks and trains, too" and you're saying "But obviously horses should only be compared to donkeys and men carrying sacks." $\endgroup$ Aug 17, 2018 at 13:19
  • 3
    $\begingroup$ I can make a guess: either you are not making the comparison but you should, or you are not clearly explaining why it's ok not to make the comparison. $\endgroup$
    – Juho
    Aug 17, 2018 at 15:28
  • $\begingroup$ @DavidRicherby: Nice example, but the way i see it is like i try to enhance "human" performance in "Marathon running" and i evaluate it by the time it takes to get from A to B. Then the one reviewers confidently says "Nowadays who needs the human to run long distances? Why not compare your performance to the recent BMW models?! and instead of a short distance of A to B why didn't you try to get from Berlin to Paris?!". I think that reviewer does not have enough expertise in the domain of sparse coding/dictionary learning. $\endgroup$
    – Bob
    Aug 20, 2018 at 8:52
  • $\begingroup$ @Juho: I think i need to focus more on other benefits and characteristics of sparse coding/dictionary learning than just the classification accuracy. Then the reviewers can better understand the reason behind the selected domain of comparisons. $\endgroup$
    – Bob
    Aug 20, 2018 at 9:12
  • $\begingroup$ @Bob Obviously, one can't get too caught up in analogies. But marathon running is a sport: it's goal is to solve a problem within specific, artificial constraints. In the real world, people are not constrained to using your method: they want the best method for solving their problem. It's perfectly reasonable if your method is good for "marathon distances" but other methods beat it for "Berlin to Paris". But if your method is beaten "at all distances" then you have a lot of work to do to convince people to care about it. $\endgroup$ Aug 20, 2018 at 10:05


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.