From LessWrongWiki, a seed AI

[I]mproves itself by recursively rewriting its own source code without human intervention.

Some would even say this could bring about an Intelligence Explosion. Many of these sources mention that it recursively "improves" itself. What exactly does that mean? How do you define "improve"?

Many people say that since it "improves" itself recursively, the jumps in intelligence would be ever-increasing. However, they also mention that an AI like this would have a goal; for example maximizing knowledge or maximizing human happiness. How would this translate into maximizing of the ability to maximize? You could also maximize for speed, let's say. How do you measure speed? A test, like the TSP? But then wouldn't the machine optimize specifically for solving the TSP, and nothing else? How would that translate into an increasing ability to optimize itself?

In other words, what would one maximize for in an implementation of Seed AI?

  • 2
    $\begingroup$ This question is basically "what is intelligence and how do we measure it?". If humans had a measure of this and an algorithmic method for increasing it, wouldn't they have done it already? Is such a thing even possible? Perhaps a more pertinent question is this: why are you getting information from LessWrongWiki? $\endgroup$ – quietContest May 5 '16 at 3:09
  • $\begingroup$ @tenCupMaximum Good points. Also; does LessWrongWiki have a history of being more wrong than the title suggests? I've never heard of it before this. I just figured that it was a trustworthy source... probably an incorrect assumption. $\endgroup$ – APCoding May 5 '16 at 3:12
  • 1
    $\begingroup$ Not sure it's a good source. Not sure what a "seed ai" is either but it sounds like sci-fi. This question would likely get better answers on sci-fi fantasy stack exchange, if my assumption is correct. Someone correct me if I'm wrong. $\endgroup$ – quietContest May 5 '16 at 3:16
  • $\begingroup$ @tenCupMaximum It is also called recursive-self-improvement. Some computer scientists have thought about this before...I don't know how serious they were though. $\endgroup$ – APCoding May 5 '16 at 3:19

Improves itself by recursively rewriting its own source code without human intervention.

A machine learning technique that has some of the properties of a Seed AI is Meta Genetic Programming.

With Genetic Programming (GP) the system automatically writes code to solve a predefined task. GP evolves computer programs according to a natural evolution paradigm, using various operators and parameters.

Meta-GP allows simultaneous evolution of operators and parameters along with the population of programs.

Of course one can imagine multiple meta-levels in an evolutionary computation system: more levels of populations acting on each other and/or populations that act upon themselves.

It's worth noting that there can be a Free Lunch for Meta-GP / Hyper-Heuristics, but these techniques:

  • introduce extra computational cost, which have to be weighed against potential advantages;
  • seem very sensitive to biases in the syntax from which the operators are generated and thus less robust.

Anyway it's an open research field.

How do you define "improve"?

Usually GP has an objective function (fitness function) to translate a desired behaviour.

Writing these fitness functions is a formidable task and it happens than your GP system optimizes for something other than it was intended for (may it happen for a Seed-AI-system?)

It's not clear if there is a way of doing without defining an objective function.

You can try to mimic Nature using a local, distributed, asynchronous form of competition or, according to the idea that "to achieve your highest goals, you must be willing to abandon them", applying Novelty Search,

but often fitness is still there (although behind the scene).

  • $\begingroup$ Great answer! I actually did my science fair project (I'm in 7th grade) on testing how good "nested genetic" (that's what I called them, because I just made up that algorithm by myself) algorithms were for the TSP. Thanks for telling me it's a real thing, and that my algorithm idea wasn't completely crazy :P $\endgroup$ – APCoding May 21 '16 at 15:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.