6
$\begingroup$

I've been thinking about ways of measuring code, and, quite frankly, I can't think of truly objective, semi-universal ways of evaluating the quality or "strength" of code to say, "Yes, this is better than that based on X, Y, and Z metrics" that are popular or exist. I've looked at SonarQube, and although I think their metrics are important, I feel like there are more fundamental concepts at work than what they have now.

But, of course, it's often said that certain languages are better than others because they have stronger guarantees or abstractions: C++ > C with abstraction, and Haskell > C++ with typing and purity and a whole slew of other features. But I was wondering whether it would be possible to actually quantify how much better one is than the other.

Are there existing papers that talk about ways of objective comparing coding paradigms or languages based on objective metrics?


A few of the resources I've found while search have to do with more business-y metrics like cost of quality or coverage of unit tests over development time. Also, measuring by number of lines seems fishy and not truly crucial (imagine if we measured math proofs' qualities by how many pages of paper it takes to print!):

Apparently the list of top 10 papers are also not based on measuring the efficiency of design in code, but are more focused on specific areas:

$\endgroup$
  • 1
    $\begingroup$ code is a human construction and therefore has a significant subjective aspect. in this way its like writing or art in places. there is a relatively new concept of "technical debt" but its hard to measure objectively. about all that can be measured "nearly" objectively about software quality are defects and that has a significant subjective aspect also. there are some objective measures that can measure code complexity and bad code will have too-low or too-high complexity. $\endgroup$ – vzn Sep 11 '15 at 15:31
  • $\begingroup$ @vzn I was wondering whether there was some sort of formal definition of any such benchmarks. $\endgroup$ – VermillionAzure Sep 12 '15 at 9:19
  • $\begingroup$ What exactly do you mean by "efficiency of design"? And I don't think there is a single metric that you can use, there's never going to be a single language that's the best option for everything and everyone, every time. $\endgroup$ – svick Sep 12 '15 at 13:35
  • $\begingroup$ @svick What I mean is that given the same design, different implementations may have different metrics than each other. The obvious ones are size, speed, robustness, and stability, but sometimes it's hard to quantify some of these (how do you actually "measure" how robust code is?) or maybe how do you evaluate the simplicity of a design? $\endgroup$ – VermillionAzure Sep 12 '15 at 20:42
3
$\begingroup$

The abstract conclusion is not really that "some languages are better than others" but that "some languages are better than others for some purposes". Different languages are somewhat like species in evolutionary biology. They have evolved to fill certain niches. There are many evolutionary niches. They have strengths and weaknesses in different areas. Sometimes these types of analysis can degenerate into what is known in stackexchange as "gorilla vs shark" debates.

The question seems to mix up certain distinct areas in CS study. Code quality is measured by defects and is more programmer and implementation related than language related. Defects can be high or low in any language. efficiency in CS is generally considered interchangeable with ("execution") complexity and measured with Big-O notation. There is also another notion of code/ programming complexity where some functionally equivalent implementations may have different levels of complexity in their programs. This is studied especially in OOP theory.

However recently there is some research along the lines you mentioned. There is a large database of programs with identical functions written in the Rosetta code database and some comparative/ objective/ scientific/ quantitative analysis has been done by researchers. See eg:

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.