In my highschool class we are learning about Artificial Intelligence, and especially the problems that come with machine learning. I was wondering if chatbots like cleverbot were good examples of overfitting. Obviously, when you get a reply that doesn't make sense, is that a result of the bot modeling a training dataset too closely, or is it something else completely? Thanks for the help
1 Answer
There is no expectation that an Machine Learning (ML) system will perform perfectly. Not just in the practical sense that we obviously haven't yet figured out how to make a human-level chatbot, but in a more formal sense that ML systems, particularly neural nets, but this is true for many other ML algorithms, approximate functions and this approximation is going to have some error.
So the fact that an ML system sometimes produces "incorrect" responses doesn't indicate anything. Overfitting is when the ML system performs extremely well on the training set and catastrophically poorly for even relatively small departures from the training set. To show this requires showing poor performance on many examples which are near the training set. Overfitting is unlikely when the training set is large compared to the number of parameters of the ML system (e.g. the weights of a neural network).
You can get nonsensical answers just as well from underfitting or false generalizations. The ML system can easily "learn" a "rule" that makes sense in some situations but not all but be either unable to model (underfitting) or unaware of (false generalization) the exceptions.
At the extremes, you're just off distribution, meaning the input you're giving the system is very unlike the input it was trained on. Obviously at this point there's no reason to expect it to perform well in any sense. An extreme example is training a chatbot on an English language corpus and then talking to it in Spanish.