0
$\begingroup$

According to wikipedia, the Frontier supercomputer is the largest supercomputer in the world, at 1 exaFLOPS, and cost 600 million USD.

I am quite surprised that the largest supercomputer only has 1 exaFLOPs, because of the following:

  • The AMD Instinct MI250X has about 100 teraFLOPS, i.e. 0.1 petaFLOPS.

  • Hence, if you stick about 10000 MI250X's together, you get 1 exaFLOPS.

  • The price of an AMD Instinct MI250X is about 10k USD, so 10000 of them are about 100 million USD.

  • This would suggest that the actual workhorse of the supercomputer is only 20% of its cost (since Frontier cost 600 million USD).

Maybe I did my calculation incorrectly, but if not then this surprises me. Why are the GPU's such a small fraction of the actual cost?

ps. Not sure which stackexchange to use for this if computer science is the wrong one.

Improve this question
$\endgroup$
1
  • $\begingroup$ Can you replace "largest" by "fastest"? Largest is by physical volume/area while fastest is by computing speed. Here is the largest computer ever built. $\endgroup$
    – John L.
    Jun 18, 2022 at 16:45

1 Answer 1

Reset to default
1
$\begingroup$

You have to remember that sticking a bunch of GPUs together does not linearly increase the performance. You can argue that yeah, if all the GPUs are running then instructions are being run, but to actually efficiently run useful operations on a bunch of GPUs is no easy task.

Some factors for scaling up GPUs:

  • Communication: this is a big one. If you have a bunch of GPUs, how do you broadcast instructions to all of them, and put them all back together?
    • On this topic, you would need bridges or some sort of other hardware to communicate data. If you have to transfer large amounts of data across distances, you will lose time while you wait for data transfers.
  • Benchmark for instructions: There's a bunch of literature (and scandals) about how chip companies purposefully devise very specific tests to boost their flop count in the disgust of their competitors. A big historical one has been CISC vs RISC processors.
  • Architecture: Again, this has to do with linearity, but a supercomputer will have a very different design than your personal computer. A supercomputer will have specialized architecture and hardware just to make that 1 TF work, which will add to the cost. Also having a 1 of 1 will make the production cost more expensive than a mass produced item like an AMD graphics card.
Improve this answer
$\endgroup$
1
  • $\begingroup$ i think that's the wrong shot. while performance in real applications usually don't grow linearly, there are lot of them, and published SC performance figures are computed in a simple way. BTW, the same applies to the single GPU - its official perfromance is just peak performance computed by multiplying frequency by number of ALUs $\endgroup$
    – Bulat
    Jun 18, 2022 at 22:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.