I had review feedbacks regarding my recent publications and reviewers argued that the improvements are not significant over the baselines.
I tried my classification algorithm on some image datasets such as Extended Yale, and in the literature (other papers) i noticed the same amount of improvements in the results was assumed good enough (e.g. only 0.20 % improvements).
So, how should i convince the reviewers that my results are good enough according to the previous works?
Odds are, you can't. That's a judgement call, and it's notoriously hard to get other people to change their judgement. The path is to find new evidence or facts they might not be aware of; assemble a persuasive argument; and see what happens.
But in this case, without knowing anything about the domain, I suspect the reviewers might be right. Why would a difference in accuracy of 0.20% make a practical difference? What new capability would it provide? How much money would it save? Often it's not that significant.
I see this misconception a lot. People see another paper was accepted (to a different conference/journal), see that it had improvement of X%, assume that it was accepted because of X% improvement, and assume that if they get X% improvement their paper will/should be accepted too. Neither of those assumptions is valid. You don't know why that other paper was accepted. Maybe it was accepted because the ideas it had were intriguing. Maybe it was accepted because the paper made other contributions. Maybe it was accepted because that conference/journal has lower standards. Maybe it was because that kind of improvement was worth one paper being published, once, but not two. Maybe it was because the reviewers were in a good mood that way. Maybe it was something else. Who knows. And just because reviewers want to accept one paper with X% improvement on task T, doesn't mean they'll want to accept 10 papers or 100 papers with X% improvement on task T.
Don't delegate your judgement to reviewers (that other paper was accepted, so it must be good). Form your own standards of what counts as valuable work. Then, strive to make your work meet those standards. That will enable you to articulate why your paper is significant and should be accepted, on its own merits.
This is a topic you should be discussing with your advisor/supervisor and your co-authors, as they know the work you're doing and (hopefully) know the domain area and can give you much more informed advice.
