0
$\begingroup$

How would you evaluate a new metric or benchmark?

My concrete example: I want to benchmark code review. As others did [1, 2] , one can apply metrics from social network analysis (such as centrality, degree, ...) to a social network emerged from code review (who reviewed when whom?).

But how do I evaluate if the metrics I have chosen are any good or meaningful?

My first thoughts were expert interviews. Do you see other opportunities?

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ This seems very broad. It's unclear what you want the metrics to measure and whether that would make any difference. On the other hand, maybe it doesn't make much difference and I'm just too much of a theoretician to see that! $\endgroup$
    – David Richerby
    Aug 27, 2018 at 14:06
  • $\begingroup$ Oh I intentionally set the scope broadly, because I thought this is a very common question: There are tons of benchmarks out there (from HPC to network). But how do you evaluate the meaningfulness of a metric? E.g. if you want to estimate the complexity of a class, you could count spaces. However, I doubt this is correlated to the complexity. :) How do you make this more scientific? $\endgroup$
    – Michael Dorner
    Aug 27, 2018 at 14:13
  • $\begingroup$ This seems very broad to me too. How you evaluate a metric seems like it will depend on what you want it to measure, what its purpose is, and/or how you want to use the metric. I don't think there's going to be any single answer. ("I want to benchmark code review" is not specific enough, because it doesn't tell us more specifically what you want to measure and why.) $\endgroup$
    – D.W. ♦
    Aug 27, 2018 at 14:49

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Browse other questions tagged or ask your own question.