0
$\begingroup$

I have looked for proof that a machine cannot compute itself in more than realtime (Which would allow infinite computing speed) and I came to the conclusion that it is impossible for any computer to simulate all other computers. However, I lack proof of this intuition.

So the idea is that the state of every (modern, non-quantum) computer can be represented as a finite bitstring (just every state of registers, memory cells, and hard-drive...). However, in order to predict what this computer would do given any state, you would need a longer or at least equally long (in some trivial cases this might work) bitstring. But since there is always a bigger computer (with a longer bitstring - representation) there is no computer which can compute all computers.

However, I am wondering about the following:

Let $f$ be a function which maps a computable function $g$ and a valid input $i_g$ of that function $g$, to the result of $g$ given $i_g$:

$$f(g,i_g) = g(i_g)$$

Can you proof that:

$$\exists i_f: f(f, i_f) \mbox{ does not halt, while } f(i_f) \mbox{ does.}$$

Note that $i_f$ is just a bitstring, and $g$ can be represented by a program and therefore can be a bitstring too...

$\endgroup$
7
  • $\begingroup$ Maybe useful: youtube.com/watch?v=92WHN-pAFCs $\endgroup$
    – Firzen
    Apr 16, 2020 at 12:00
  • 4
    $\begingroup$ Note the question of "can a computer simulate another" is different from "what is the slowdown when using a computer to simulate another"? A univrsal Turing machine can simulate any other TM, but it can't necessarily do so in the same number of steps the original would have taken. $\endgroup$
    – usul
    Apr 16, 2020 at 15:05
  • $\begingroup$ Does "all computers" mean "all computers which currently exist" (a large but finite number) or "all computers which can theoretically exist" (an infinite number)? $\endgroup$
    – Philipp
    Apr 16, 2020 at 15:25
  • 3
    $\begingroup$ This question seems confusing. Is the question about theoretical computer science which treats computers as an abstract mechanism, or are you asking a question about a thing we could actually build out of sand and metal? $\endgroup$ Apr 16, 2020 at 17:33
  • 1
    $\begingroup$ You might benefit from study of the linear speedup theorem, realizing there is no upper limit on $c$ in the statement of the theorem at the link. In imprecise language, we can trade a wider CPU bus for faster execution. $\endgroup$ Apr 16, 2020 at 21:18

4 Answers 4

1
$\begingroup$

I think you're looking for Stephen Wolfram's principle of Computational Irreducibility, which applies not just to simulating computers but simulation of any physical process that is computationally irreducible. It suggests that you can only simulate a computer in real time with an equally powerful or more powerful computer. I'm not aware if there are any mathematical proofs of this principle but this paragraph from his 1984 paper seems applicable:

One expects in fact that universal computers are as powerful in their computational capabilities as any physically realizable system can be, so that they can simulate any physical system. This is the case if in all physical systems there is a finite density of information, which can be transmitted only at a finite rate in a finite-dimensional space. No physically implementable procedure could then short cut a computationally irreducible process.

See also the full paper or the definition of computational irreducibility.

$\endgroup$
1
  • $\begingroup$ "It suggests that you can only simulate a computer in real time with an equally powerful or more powerful computer." This was exactly what I was searching for! $\endgroup$
    – TVSuchty
    Apr 17, 2020 at 15:07
20
$\begingroup$

Your intuition sounds wrong to me. It's not surprising you couldn't prove it, because it isn't true. Roughly speaking, it is possible to find a computer that can simulate all other computers.

A universal Turing machine can simulate any other Turing machine, and can be used to simulate any other program in any Turing-complete language. Thus, there is a reasonable sense in which it can simulate all other computers, if we make precise what we mean by "computers" in a particular way (e.g., if we accept the Church-Turing thesis).

If you want to know whether it is possible to simulate another computer and also do it faster, that's a different matter. You'd have to formalize what you mean by "faster" to make that question well-defined.

Your question at the end of the post is not well-formed; computability is a property of languages, not of a single value (like $f(f,i_f)$). Also, you defined $f$ to be a function of two arguments, so $f(i_f)$ doesn't type-check.

$\endgroup$
3
  • $\begingroup$ You are right, the question, in the end, is not well-defined - I try to fix it. However, I am not talking about turing machines, since they can use infintely memory... $\endgroup$
    – TVSuchty
    Apr 16, 2020 at 3:23
  • $\begingroup$ I tried to fix the question in the end. $\endgroup$
    – TVSuchty
    Apr 16, 2020 at 3:26
  • 6
    $\begingroup$ Turing machines cannot use infinite memory. At every step of execution a Turing machine only uses a finite amount of memory. However, it can always get more memory if needed. The definitions speaks of an "infinite tape" as a convenience, we could rephrase that to having only a finite amount of tape which gets extended when necessary. $\endgroup$ Apr 16, 2020 at 7:54
1
$\begingroup$

You are using intuition about real computers and trying to apply it to Turing Machines. Real computers have limits that Turing Machines do not.

When it comes to Turing machines, the increase in the size of the data set is not an issue. A Turing machine has an infinitely long tape, so adding a finite amount of extra data to it is no problem. All you need to do is show that for any given Turing Machine, there exists a "reified" version which encodes the rules of the original machine as data on the tape. This was proven by Alan Turing.

When it comes to real machines, which have finite limits, the story is much more complicated. For starters, no real computer is actually Turing Complete. They are actually finite automata, limited by things like memory. The "Turing Completeness" of real computers really argues that the abstract instruction set is theoretically unbounded (for example, you could hook it up to a tape drive of arbitrary length).

When you look at this different domain, the story fits more with your intuition. Computers tend to be unable to run faster than themselves. However, there is a devil in the details of how you define this. Optimization creates really complicated corner cases. For example, if I consider the loop in C++:

int m = 0;
int n = 0;
for(int i = 0 ; i < 1000000000; i++) {
    for (int j = 0; j < 1000000000; j++) {
        for (k = 0; k < 1000000000000; k++) {
            n = n + 1;
        }
    }
}
std::cout << "M is " << m << std::endl;

It would take a very long time for the computer to "run" this. It has to do a giant nested loop which takes a long time. However, if we reify this, permitting the computer to effectively look at the source code (or machine code, or whatever), its easy to see that that big loop does not affect the value of m. We can skip the loop entirely.

The ability to do this is at the root of all compilers. And, indeed, it gets done at runtime as well. Consider the java code

int sum = 0;
for (int i = 0; i < myArray.length; i++) {
    sum += myArray[i];
}

By the rules of the language, we are obliged to check that myArray[i] does not access a value past the end of myArray. We are supposed to check it every time. However, this is slow. So most java runtime environments are smart enoguh to recognize the bytecode emitted by the sort of pattern and only do the array length check once. Thus we have a case where simply executing the byte code as written is slower than reifying it into instructions, analyzing them mathematically, and emitting a new "faster" program.

Of course, its trivial to show that there must be at least one program which is as-fast-as-possible.

While I wont get into the details of it, microprocessors do this as well. No processor currently produced actually runs x86 or x64 instructions. x86 is an abominably slow instruction set. Instead, they all analyze the x86 instructions and emit "microcode" which is much faster to execute. For example, the microcode can show opportunities to pipeline which aren't always valid for a string of instructions, but can be shown to be valid for this one.

$\endgroup$
0
$\begingroup$

This depends on the parameters of the question asked. If you are talking about Turing Machines, then:

  • Any Turing machine can be represented as a bitstring (as you know). The description of the Turing machine is equivalent to the description of the program (code and state) currently running on that Turing Machine.
  • Any set of inputs can be represented as a bitstring (obviously)

Therefore, any Turing machine can indeed simulate any other Turing machine: Simply copy the code, state, and inputs for the current program from one Turing machine to the other, done.

There are a couple variations on this question that complicate issues, and it seems like you might be asking about one of these variations:

1) Rather than Turing machines, which are a theoretical construct which does not actually exist (since the material to construct a real Turing machine would be potentially infinite and there are not infinite atoms in the universe), use a real-life "computer", as we understand it. In particular, a "computer" can be modelled as a RAM (Random Access Memory) bitstring and a ROM (Read-Only Memory) bitstring; the RAM is the state and inputs, the ROM is the code. Indeed, one computer could simulate another computer simply by copying the RAM and ROM from one machine to another. The problem is that a computer has a fixed maximum size of RAM and ROM based on the materials used in construction. Since technology is continuously improving, it is reasonable to suppose that there will never be a machine which can simulate any arbitrary machine, because that machine would need to have more RAM and/or ROM capacity not only than any machine ever created, but also will ever be created, and that's (probably) impossible.

2) A Turing machine which not only can simulate ANY other Turing machine, but also can simulate EVERY other Turing machine. Like, all at once. That is not possible, as follows:

Any Turing machine can be described by a bitstring, as we know. The bitstring is of arbitrarily long but finite length (it is both required and important that the description of a Turing machine is finite). However, these are the only restrictions on the description of a Turing machine; in particular, the set of possible representations of possible Turing machines is infinitely large. Therefore, the bitstring representation of a Turing machine that can represent the infinitely large set of all Turing machines must be an infinitely long bitstring (due to information theory). This violates the premise above that all Turing machines must be representable by a finitely long bitstring.

$\endgroup$
9
  • $\begingroup$ I'm not sure about this answer; on 1 - it is not clear that the real-life computer talked about should have a fixed maximum memory size from construction time. Could we not plug in an external HDD if we run out on the current one; buy more tape when we need it. On 2 - The set of natural numbers is infinite, yet "while True: i++" will step through all of them; the same way the number of TMs is countably infinite and we can in a dovetailing way simulate them all simultaneously. $\endgroup$
    – O.O.
    Apr 16, 2020 at 16:39
  • $\begingroup$ @O.O. On 1: We can have an HDD of incredibly large size, that's true. However, even if we plugged in every HDD ever created or will ever be created of all types and sizes, the HDD space is still finite, because there are finite atoms in the universe and an HDD is composed of some of them. Therefore we can consider the memory size fixed from construction time and just say "it's arbitrarily large but still finite". So in theory if we say 99% of all HDDs and RAM sticks ever created are in this computer and 1% are in all other computers, then fine. That's why I said "probably" at the end. $\endgroup$
    – Ertai87
    Apr 16, 2020 at 17:32
  • $\begingroup$ @O.O. On 2: Countably infinite is still infinite. Even if you can enumerate every natural number by "while True: i++", you will still be enumerating an infinite series and that program will take (countably) infinite time and (countably) infinite space (the space is used to store the value of very very very VERY VERY VERY large n as n -> \infty). By similar logic, you cannot enumerate an infinite number of Turing machines in finite time/space, even if it's countably infinite. $\endgroup$
    – Ertai87
    Apr 16, 2020 at 17:36
  • $\begingroup$ @O.O. Basically the answer to problem 1 devolves into "well, what if we put every ore on the Earth into making a super computer that is so powerful that nothing will ever be more powerful?" and then I respond "Then we go to Mars and mine every ore there to make a computer even more powerful", then you say "Well I'll go to every planet in the solar system" and then I say "Well I'll go to Alpha Centauri and mine there" and so on. The argument is fundamentally ridiculous so I handwaved my way around it. $\endgroup$
    – Ertai87
    Apr 16, 2020 at 17:51
  • $\begingroup$ The simulation of all turing machines will never finish (it is infinite, and many of them will not halt); however, at any given simulation step we will have taken a known amount of steps in each machine, and only use a given finite amount of memory. Similarly in the "practical" scenario - there may be a maximum amount of information that can be stored in an optimal way in our part of the universe; but this requires a lot of handwaving - perhaps the answer instead should be that it is not very interesting to ask this question, and that we should frame it in the theoretical model instead. $\endgroup$
    – O.O.
    Apr 16, 2020 at 21:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.