1
$\begingroup$

How do people measure performance overhead? Whenever someone is bragging about how their program or application performs better than another, they talk about particular measurements, eg time, performance etc.

Whilst I understand how time to perform a task can be calculated, I cannot understand how performance is measured, and I can't find out thought Google.

Are there multiple ways? Are they all accepted or do people have issues with certain methods?

| cite | improve this question | |
$\endgroup$
  • 2
    $\begingroup$ What do you call performance? How does it differ from execution time (assuming time performance is considered), or as you say "time to perform a task"? What is "performance overhead"? $\endgroup$ – babou Apr 9 '14 at 11:35
  • $\begingroup$ Thats a good question but I don't know how to answer it. I frequently read statements, particularly with regard to hypervisors and virtualisation, stating how their "performance overhead" is superior to other solutions. And they often don't define what they mean by performance overhead. $\endgroup$ – user16580 Apr 9 '14 at 12:03
  • $\begingroup$ These are very special kinds of programs for which overhead does have meaning. You should make that explicit in your question, so as to avoid downvotes. the answer is actually simple, you run applications directly on the computer and also under the virtualized machine. Then you compare running times. The result may depend on the type of application. $\endgroup$ – babou Apr 9 '14 at 13:26
  • $\begingroup$ If you have two versions A and B of the same program and B does things 20% faster than A (without doing anything incorrectly), then you apparently have 20% overhead in A. I don't think it's a notion that makes much sense without something to compare to. $\endgroup$ – Raphael♦ Apr 9 '14 at 21:43
3
$\begingroup$

The meaning of overhead is usually an additional expense occurring in addition to normal cost (whatever normal cost may be).

In the case of program, you may talk of overhead if it is run under conditions such that it will cost more than it normally should, in time or in memory usually, but it could be some other measure.

For example, at a time when compilers did not produce very good code, compiled programs code size or execution time was compared with programs handwritten in machine code. The quality of the compiler was measured by the overhead of the generated code compared to a similar program written in machine code. The overhead could be defined by the ratio of the two costs, or by the percentage to be added.

An overhead of 20% in code size meant that the compiled programhad a machine code version 20% larger than the hanwritten one. An overhead of 20% in speed meant that the compiled program took 20% more time to execute. An overhead of 20% in memory meant that the compiled program required 20% more memory to execute.

Nowadays, compilers are quite good and this kind of overhead would probably be negative. Compilers make the better code.

In the case of a virtual machine, you can have a program that run directly on the hardware, or runs under the control of a virtual machine that hides (and protect) somewhat the hardware. Running on a virtual machine will slow the program, and the variation in running time is the overhead (extra time cost) caused by the virtual machine. You just measure the time taken by the same program running in both situations, and you determine the percentage of additional time required when running in a virtual machine. It may depend on the type of application you use to test the overhead of the virtual machine.

Overhead is a measure of extra cost. It applies to devices or programs or procedure that cause an extra cost (but usually provide an extra servive).

You could also measure the time overhead incurred by your programs when you reduce the available real memory.

Overhead is a way of comparing two related performances, intended to achive the same result, separately measured.

Does this clarify things for you?

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ An addendum to this is that "overhead" can often often be measured relative to a theoretically minimum size. For example, if your architecture has 8kb pages and a page table entry is 32 bytes in size, then about 0.4% of memory is used by page tables. This means that virtual memory imposes roughly a 0.4% overhead in memory usage ($\frac{100\times 0.4}{100-0.4} \approx 0.4$). Similarly, a filesystem requires disk space to be reserved for allocation tables, directory entries, inodes, indirection blocks, and so on. This is an overhead on top of the space used by file data. $\endgroup$ – Pseudonym Apr 10 '14 at 0:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.