2
$\begingroup$

AIXI is a theoretical formulation for artificial general intelligence. While AIXI is computationally intractable, there are approximations of AIXI like https://arxiv.org/abs/0909.0801 that are computable.

Some time ago, I stumbled upon theoretical artificial general intelligence, and I would like to know what are the practical differences between approximations of AIXI and deep reinforcement learning methods.

Improve this question
$\endgroup$
2
  • $\begingroup$ Can you explain what you mean by "practical differences"? AIXI is a theoretical framework, whereas deep learning is a practical framework without theoretical grounding. $\endgroup$
    – Yuval Filmus
    Commented Jul 12, 2017 at 15:00
  • $\begingroup$ By practical differences, I want to know that when I am writing a computer program to play games (say, the go game), what are the considerations I have to make to choose between approximations of AIXI and deep reinforcement learning methods. I know that people rarely use AIXI-related methods, but I want to know if there are any circumstances where one might want to use AIXI approximations instead of deep reinforcement learning methods when actually implementing agent AIs. $\endgroup$
    – Petercommand Hsu
    Commented Jul 12, 2017 at 15:18

2 Answers 2

Reset to default
1
$\begingroup$

It's a bit of an apples and oranges comparison. AIXI is a proposal for how generalized learning should be structured, while neural networks are a specific learning implementation.

That being said, you probably wouldn't want to use an AIXI implementation for a known task such as game AI. Game AI is a specific learning task, so it's better and easier to use an AI specifically tailored to that task. The AlphaGo AI, for example, combined a deep learning neural network for finding interesting looking moves with a positional analysis engine for figuring out the strength of those moves. That sort of architecture works great for learning to play Go, but it's not general.

Improve this answer
$\endgroup$
0
$\begingroup$

Deep reinforcement learning is reinforcement learning where you use a deep neural network as the memory component (in RL terminology the Q-function). More details can e.g. be found here: https://www.intelnervana.com/demystifying-deep-reinforcement-learning/

Concerning RL and AIXA: "The general reinforcement learning problem is to construct an agent that, over time, collects as much reward as possible from an initially unknown environment. The AIXI agent (Hutter 2005) is a formal, mathematical solution to the general reinforcement learning problem." Source: https://www.aaai.org/ocs/index.php/AAAI/AAAI10/paper/viewFile/1722/2048

The problem is that AIXA is not computable so approximations of AIXA are necessary.

Putting it all together:
Deep reinforcement learning is a particular implementation of an AIXA approximation.

Personal note: I think it is no coincidence that the author of the abovementioned paper and several other papers in this area (Joel Veness) is now working for Google DeepMind...

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.