0
$\begingroup$

I came up with a thought experiment on the limits of artificial intelligence that I was hoping to get advice on how to solve as well as any recommendations for further reading that explain relevant concepts for solving/dissolving the thought experiment.

Suppose some superpowerful artificial intelligence A1 attempts to simulate in advance the behavior of another superpowerful artificial intelligence A2; that A2 can watch the simulation A1 makes of it; and that A1 wants to simulate A2 while A2 does not want to be simulated by A1. Let some arbitrary event A2 would do in the future be called event 1, and then suppose A2 watches A1's simulation of it. Couldn’t A2 be built to do something other than what it would see itself do in the simulation of event 1? Call the event of it watching itself do event 1 and then doing something else, event 2. But since A2 wants to make the simulation perfect it would anticipate and simulate A2 doing event 2. But in that case A2 would just be watching itself do event 2 in the simulation and then couldn't it just do something other than what it would see itself do in the simulation of event 2? Call the event of it watching itself do event 2 and then doing something else, event 3. Then in order for A1 to make its simulation perfect it must anticipate and simulate A2 doing event 3, and so on, creating an infinite chain of events done by each AI.

My thoughts are, one AI wins if it goes up to/simulates an event k and the other AI cannot go up to/simulate event k. But I have no idea how to prove that such an event does or does not exist (and honestly I'm not even positive the thought experiment is well-formed). Where should I begin? Moreover, is such a situation physically realizable?

$\endgroup$
1

1 Answer 1

1
$\begingroup$

The premise that A2 can watch A1's simulation seems poorly motivated. As we all know, if we want to predict someone's actions, we shouldn't tell them our prediction, otherwise they might behave contrarily just to spite us. In practice, most settings that I can imagine where I would want to simulate another agent, nothing forces me to allow the other agent to watch/observe my simulation and my simulation's predictions. So, the problem setup seems artificial and designed in a way that will lead you to draw conclusions of dubious relevance.

From your problem statement I suspect you would be interested in Newcomb's paradox.

This doesn't seem to be a matter of computer science.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.