The answer to your question is much dependent on specifics.
I certainly agree with D.W. answer that for comparing performances of
complex systems, you need at least that kind of information (hardware
specs), if you are interested in software performance through blackbox
measurements. If box A has CPU, memory and disks that are twice as
fast as box B, while being otherwise identical, it is clear that the
same program will run twice as fast on A than it does on B. So if you
compare two programs, one running on A and the other on B, you have
to be aware of that.
However, the information you get from blackbox testing is very
limited. For one thing, you may have a faster CPU, but a slower disk.
Then, it is much harder to compare two software variants,
Generally the observed performance depends on very many factors that
are hard to identify, and hardware specs may be grossly insufficient.
Among other factors you have the quality of the algorithms used, the
quality of the programming, the quality of the compiler (optimizing or
not), the bias of the benchmark used, and probably some more.
Then the techniques for comparing a system (say a word processor), for
comparing language implementations and for comparing algorithms may be
quite different, due to complexity and also to formal definibility.
My personal opinion is that comparison of algorithms should, as much
as possible, not be hardware dependent. If possible, it should use
formal cost analysis, but that is not always simple to
achieve. Short of doing that, it should be based on benchmarking on a
virtual machine with appropriate primitives, so that the virtual
machine can count precisely the number of instructions of each type
executed. Of course that leaves the implementation quality bias and
the benchmark bias, but the process is formal enough to be disputable
on objective grounds. And the results are sometimes surprising.
It is to be noted that I refer here to cost analysis, rather than
asymptotic complexity analysis. Asymptotic complexity analysis is
cost unit independent and can only measure computational cost scaling
with input size, i.e. compare an algorithm with itself as input changes. But
comparing two algorithms, or two algorithm implementations, requires
defining common computational unit(s), which implies a precisely defined common
computational framework (as discussed also in this answer). A common
virtual machine seems the right way to do it.
Regarding more complex software, I know of cases where the benchmark
bias plays an important role. I have a friend who participates in
natural language parsing competitions, and his technology is clearly
put at a disadvantage by the way the benchmarks are provided (though
that is by no means intentional). Implicit assumption in benchmark
creation can play an important role. And this has nothing to do with
To conclude, hardware specs can be very important in some cases, and
it is usually better to divulge it. But it is far from being the whole
story, and should not be taken as such. And there are situations when
it should be irrelevant and may just be an indication that no proper
evaluation technique has been actually used for comparisons. Or at
least, you should be wary of hasty conclusions regarding the
technology used in the software and its quality.