2
$\begingroup$

There are many problems that people can solve more easily than a machine learning algorithm. Often this is attributed to the extra context we have access to.

Image segmentation is an example of a problem that people are good at, but algorithms are not good at, probably because people have lots of external contextual knowledge they bring to bear on the problem. To date, the best techniques use a human in the loop, such as this technique called click carving from HCOMP 2016.

enter image description here

Are there problems that people can solve more easily than most or all machine learning algorithms, where people do not need to depend on access to external context?

Here is another example that is easy for people but hard for a particular algorithm implementation called XGBoost. A human can easily fill in the missing shapes with 100% accuracy, but XGBoost cannot score above 50% accuracy, and its accuracy drops as it gets more training data. Neural networks also seem to have difficulty with this problem.

Note that the person does not need external knowledge about the problem that the algorithm does not have, in order to perform better. All the person needs to see is the configuration of shapes, and they can guess from that what the missing shapes are. The only information the human needs is that the simplest description of the data is most likely the best description, and that the training data is randomly sampled. Both items of information are used in designing XGBoost, and most ML algorithms for that matter.

Easy for humans, hard for XGBoost

Are there other concrete examples like this? Historically, the XOR problem showed that single node perceptrons were inadequate, leading to today's neural network. Besides that, I do not know of any other historical examples.

Improve this question
$\endgroup$
12
  • $\begingroup$ Note bene: "context-free" is a term with a very specific meaning in CS. I suspect is had little to do with what you have here. (If it were, the problem would be easy for computers!) In fact, what do you mean by "context-free" here? Seems like I have to use a lot of context to fill in the gaps. $\endgroup$
    – Raphael
    Commented Apr 8, 2017 at 9:26
  • 1
    $\begingroup$ "A human can easily fill in the missing shapes with 100% accuracy" -- that's probably because humans intuitively assume the "correct" metric: a locally symmetric pattern. Computers don't have an arbitrary preferences for symmetry, so you have to tell them to prefer such things if you want them to come to the same result as (most?) humans. I suspect that if you define the metric precisely enough, this problem is fairly easy for computers. (You're probably talking about general-purpose ML which has to learn the metric from sample. That's not a fair comparison to humans.) $\endgroup$
    – Raphael
    Commented Apr 8, 2017 at 9:29
  • 1
    $\begingroup$ I doubt that the shape-filling thing cannot easily be captured by a machine learning algorithm. First, we use the knowledge that we read in lines / columns. So this is also some information that the computer should get. Then you basically do a binary sequence prediction where I'm pretty sure RNNs are awesome $\endgroup$
    – Martin Thoma
    Commented Apr 9, 2017 at 22:47
  • 1
    $\begingroup$ Also: This shape-filling thing looks a bit as if you could pose it as a lossless compression problem ("which shape do I have to fill in to make the compressed file the smallest?"). I'm certain that I could write an algorithm which does that FAR better than any human could ever do (just think of non-grid patterns / huge amounts of possible shapes). Interesting question, never the less! $\endgroup$
    – Martin Thoma
    Commented Apr 9, 2017 at 22:49
  • 1
    $\begingroup$ @yters What I'm saying is that probably both, XGBoost and NNs can solve this problem if encoded the right way and humans can probably not solve this problem if encoded the wrong way. $\endgroup$
    – Martin Thoma
    Commented Apr 10, 2017 at 14:58

1 Answer 1

Reset to default
0
$\begingroup$

This answer is an ongoing catalogue of instances I find.

  1. "A Deep Belief Net Learning Problem" explains why shallow networks cannot learn XOR problems, stating that deep networks can. However, empirically deep networks are not good at learning XOR problems, suggesting the XOR problem is one that is easy for humans but hard for neural networks.
Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.