I have had a somewhat hard time trying to understand how ChatGPT can "solve" some tasks that cannot be entirely cast as language-model-based rephrasing of textual subsets of the internet directed by a textual query.
For example, ChatGPT seems to be able to do calculations, even if not always appropriately used, execute code, extract literature references from input text, etc.
It seems to me that it is not simply one huge neural network, but has access to some additional programs like a calculator or some interpreters for programming languages, etc. So it would be more like a human-machine interface.
To which extend is my interpretation correct?