-1
$\begingroup$

In a video Noam Chomsky said, that if these LLM get bigger, than also the things they are not good at get bigger too. He doesn't explain more details about this. So is this true? In what way do their problems get bigger? Sry for the little context of in what they are getting also more bad at.

here is the video: (it's short)

https://m.youtube.com/watch?v=K7S0zHIDMaI&pp=ygUPY2hvbXNreSBjaGF0Z3B0

Improve this question
$\endgroup$
4
  • $\begingroup$ Could you provide a link to the video? $\endgroup$
    – ShyPerson
    Apr 30 at 16:34
  • $\begingroup$ m.youtube.com/… $\endgroup$
    – user117640
    May 5 at 13:12
  • $\begingroup$ I think your interpretation of the video misses that the claim is specifically about understanding (human) language. I think the following quote from the video represents the claim more accurately: "The systems have absolutely no value with regard to understanding anything about language or cognition [...] The more they improve, the greater their flaws, for a simple reason: that these are systems do just as well for actual languages as for impossible languages". $\endgroup$
    – Discrete lizard ♦
    May 10 at 8:28
  • $\begingroup$ I didn't see clear statements related to the growth of LLMs in the video. Could you be more specific ? $\endgroup$
    – user16034
    May 10 at 8:57

2 Answers 2

Reset to default
2
$\begingroup$

First, it's important to understand that Chomsky is a linguist (a syntactician) and that the goals of linguistics are very different from artificial intelligence. In particular, the two domains differ greatly as to what constitutes a satisfactory theory of human language. For the modern study of syntax, it is to have a rigorously correct formalization of all of the grammatical sentences in a natural language; it should also make correct linguistic predictions. If there is an exception to this, the theory is considered wrong. This is very different from artificial intelligence, which has much less rigorous goals as to what constitutes success.

Chomsky is pointing out in the video that it’s also important in linguistics that the theory make actual claims about human language. So if the theory is just as good at non-human languages (like programming languages or even antibody therapies), it can’t make any interesting claims or predictions at all about human languages.

This of course ignores the fact that, notwithstanding the semantic hallucinations, LLMs have achieved strikingly accurate syntactic competence. So, even though the computer program to run the LLM makes no linguistic claims, the learning implicit in the neural network still has an implicit linguistic theory to manifest that competence.

But this in turn fails, because LLMs are just black boxes and do not in any way constitute satisfactory scientific explanation. No one has any idea how to translate billions of weights from an artificial neural network into a linguistic theory.

Improve this answer
$\endgroup$
-2
$\begingroup$

axiomatic mathematical theories are just entirely different from pattern matching 'expert system' ones.

The more patterns, potentially (but not necessarily) the less chance to consolidate and aggregate them into axioms.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.