Measures of performance in machine learning

Question

I'm new to machine learning and struggle to interpret the results I get from different measures of performance. If for several prediction models I have e.g. accuracy, precision, recall, F1, FPR, and MCC and want to find out which model performs best, what do I look for? I would assume accuracy is most important?

Also, what influence does the error have on the interpretation? E.g. method 1 has an accuracy of 87 +- 5% and method 2 has 84 +- 5% I would assume method 1 to be better. Is that correct?

Answer 1

Precision and recall are the two basic measures and most of the other measures can be written in terms of them. If a model has both better precision and better recall, then it can be seen as strictly better than another model. If two models are incomparable in the sense that another has better precision and another has better recall, then it depends on the application which one is better. For some applications it is essential to avoid false positives, and for some applications it is essential to avoid false negatives.