Questions tagged [benchmarking]
Questions about methods for empirically investigating cost measures of algorithms and data structures.
20
questions
4
votes
0
answers
37
views
Benchmark of SAT solvers on random k-SAT instances at satisfiability threshold
I am looking for a solid reference (peer-reviewed publication) on the design and/or benchmarking of SAT solvers for random k-SAT ($4 \leq k \leq 8$) operating at satisfiability threshold.
The majority ...
- 41
2
votes
1
answer
225
views
How does the TEPS benchmark work and why is it relevant to real world problems?
Graph500 is a competition for supercomputers that uses a different benchmark "Traversed Edges Per Second", which is supposed to measure some notion of the communication bandwidth ability of ...
- 3,622
0
votes
1
answer
49
views
RPI cluster performance related to network performance
I'm writing my thesis and i have built a RPI cluster, containing 10 nodes which consists of RPI model 3b. I've them connected to two gigabit switches. I don't know the CAT of the cables. They are not ...
- 1
1
vote
0
answers
27
views
Where can I find some free benchmarks to evaluate a MCU? [closed]
At present, I'm designing a soft processor with single-precision floating point unit (FPU). I am going to put my soft core into an FPGA and do some performance evaluation. The benchmarks are supposed ...
- 11
2
votes
0
answers
176
views
Is the data-centric "push" model for SQL evaluation nonsense?
The traditional way to implement a SQL database is to parse the query, create a parse tree, and then evaluate it using the "volcano" model. In this model you call .next() on the root node of the tree ...
- 129
0
votes
0
answers
24
views
How to evaluate metrics or benchmarks?
How would you evaluate a new metric or benchmark?
My concrete example: I want to benchmark code review. As others did [1, 2] , one can apply metrics from social network analysis (such as centrality, ...
- 101
3
votes
1
answer
356
views
Why do I get really different results with my benchmarking code I made?
I'm doing research work for my last year in high school. My work is about processors and for the experimental part i've coded an app that can mesure how many Floating Point Operation can a processor ...
- 31
2
votes
0
answers
54
views
Does a PC age in terms of hardware and performance? [closed]
Let's say you are using a personal computer for 10 years. Assuming all software things are the same, will the computer get the same scores on benchmarks as 10 years ago? If not, why?
- 71
0
votes
0
answers
37
views
What are good algorithms for virtual machine benchmarking?
I am comparing execution efficiency of the same code compiled to JVM and Node, locally and on AWS Lambda. Not looking for precision, but just a rough notion of performance in difference virtual ...
- 101
0
votes
1
answer
146
views
Strategy to create a testbed and gold standard
I need to create a correct evaluation of the method I'm proposing. The problem is that in the field I'm working on there is no benchmark that I can use for this purpose. On the other hand, my method ...
- 1
1
vote
1
answer
27
views
Data normalization when control has multiple values in CS/CE
(This question was originally posted at https://academia.stackexchange.com/questions/68675 but was considered too specific for Academia.)
In computer science/engineering research papers relating to ...
- 113
3
votes
0
answers
166
views
How are benchmarks created?
Computing equilibria in games and the complexity thereof is imho still quite a young field in which a lot of work still is to be done (especially the former).
GAMUT (2004) is a very nice "suite of ...
- 1,304
7
votes
2
answers
417
views
Interesting small SAT problems?
I'm noodling around with making a hardware SAT solver on an FPGA, and I'm wondering if there are any interesting SAT problems smaller than, say, 50 variables both to stay within the limits of the FPGA ...
- 347
0
votes
0
answers
53
views
Does it make sense to work on DIMACS framework to test TSP algorithms?
Are there any reasons to work with the DIMACS framework, such as to test algorithms, coding, and heuristics of TSP (Traveling Salesman Problem)? It is nearly 15 years since establishment of the ...
- 101
1
vote
2
answers
210
views
Relative Importance in Graph Theory [closed]
I am working on an algorithm that ranks a set of nodes in a graph with respect to how relative this node is to other predefined nodes (I call them query nodes). The way how the algorithm works is ...
- 121
1
vote
0
answers
74
views
Explanation of algorithms used by NAS benchmark kernels
I'm not sure if this is the appropriate forum for asking the following question. Please guide me to the right forum.
I am trying to understand the algorithms used by kernels(such as Integer Sort) in ...
- 121
5
votes
4
answers
3k
views
What are flops and how are they benchmarked?
Apple has just proudly stated that their new mac pro will be able to give up to 7 teraflops of computing power. Flops stands for Floating Point Operations Per Second. How exactly is this benchmarked ...
2
votes
3
answers
13k
views
Calculating Floating point Operations Per Second(FLOPS) and Integer Operations Per Second(IOPS)
I am trying to learn some basic benchmarking. I have a loop in my Java program like,
...
- 123
0
votes
0
answers
57
views
How to generate inputs to evaluate the performance of an implementation?
How can we assert that the performance of an implementation matches the expected performance defined in terms of Big-$\mathcal{O}$ and friends by generating best/worst case inputs? I'm thinking of a ...
- 3,660
29
votes
2
answers
6k
views
Where to get graphs to test my search algorithms against?
I am implementing a set of path finding algorithms such as Dijkstra's, Depth First, etc.
At first I used a couple of self made graphs, but now I'd like to take the challenge a bit further and thus I'...
- 600