2
$\begingroup$

I am writing various "program analysis" (aka static code analysis) algorithms.

I am looking for ideas for metrics to measure how well my algorithms "understand the code".

Some ideas i have so far :

classic points-to analysis - ability to determine (during compile time) pointer aliases and allocation sites

early object release - ability to predict when an object can be released (both in the context of explicit java references, or implicit C++ object usage)

I hope to find complex metrics, where the score can be improved by running deeper and longer algorithms on the source code.

Improve this question
$\endgroup$
7
  • $\begingroup$ I'm not aware of any widely-used metrics, but sometimes I see statements like "tool X running on the source code Y (Z lines of code) reported N problems, M of which seem false negatives (after manual inspection)". Often Y is some open source project. Also consider precision / recall on those "datasets", especially if your tool is unsound & incomplete (i.e. it could report both false negatives and false positives). $\endgroup$ – chi Apr 5 '19 at 13:48
  • $\begingroup$ This question seems too broad and open-ended to be a good fit here. Any community votes? (I think you'll find that "understand the code" is not well-defined and you need to narrow down your goals before you can find useful metrics to measure how well you've achieved them.) $\endgroup$ – D.W.♦ Apr 5 '19 at 15:22
  • $\begingroup$ Cross-posted: cs.stackexchange.com/q/106514/755, stackoverflow.com/q/55526200/781723. Please do not post the same question on multiple sites. Each community should have an honest shot at answering without anybody's time being wasted. $\endgroup$ – D.W.♦ Apr 5 '19 at 15:36
  • $\begingroup$ @D.W. The wording may indeed be off. We have "move hueristics" to help chess programs, and of course we dont claim that a computer "understands" chess. In reinforcement learning we have the concept of "auxiliary tasks" which are small, easy incidental tasks, which help the agent progress to the real, challenging tasks. Perhaps the term "code analysis" is better ? $\endgroup$ – YAZR Apr 5 '19 at 15:38
  • $\begingroup$ @d-w Yes. It was originally posted there, and it was (correctly?) suggested this is a better forum. $\endgroup$ – YAZR Apr 5 '19 at 15:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.