Does AI “understand”?

Most people outside the AI community (and some within) get hung up on this term and insist that large language models (LLM) cannot “understand” – the pejorative that has been used is that they are “stochastic parrots” that just repeat answers that sound probable. And to an extent they are correct, because that is how LLMs work - by associating probabilities with words and tokens. But let us take a deeper look at what we mean by this term.

What do I mean when I say that I understand chemistry? I mean that I have a working model (in terms of the periodic table and concepts like electronegativity and different kinds of bonding) for how molecules interact with each other. What do I mean when I say that I understand physics? I mean that I have a working model (in terms of forces and interactions, equations of motion, symmetry principles and concepts like spacetime) for how the physical universe is constituted. What do I mean when I say that I understand biology? I mean that I have a working model of how the body and natural selection work, and how proteins interact within the body. What do I mean when I say that I understand society? I mean that I have a working model of how human relationships work.

Now let us ask: does AI have such a working model on the universe of data it has been trained on? Indubitably the answer is yes! That is why such models are useful and predictive. But it does not mean that these models are the same that a human would come up with. In fact, it has been found that forcing a deep learning model to learn the way humans learn is not very productive. Of course, sometimes the models do show some kind of parallelism with the concepts that humans have formed, and then we happily say that the model is interpretable – namely that we can understand what is going on in terms of our own concepts. But more often than not, deep learning models are not readily interpretable by us.

This does not mean to say that the AI model is necessarily correct. But the same could be said about our own models, our own understanding. Either (or both) of these models could be biased, or unpredictive - it could be proved wrong as fresh empirical facts come in. Large parts of our present-day understanding of the physical universe and of human biology are probably wrong; bits and pieces of our working models are being falsified and refined every day.  This is how science progresses. And this is how reinforcement learning (RL) models work – by making mistakes and learning from them.

So let us move beyond this unproductive debate about whether LLMs “understand” – they do “understand” in the sense of having constructed a working model of their world, but in a way possibly very different from the way humans do. And that’s fine! A dog understands the world in a way very different from how we do, but no dog owner would say that his/her dog lacks understanding. Your dog may not understand Shakespeare or elementary particle physics, but it has a working model of the world quite adequate for its needs, and it does understand human emotions and a lot else besides. But at the end of the day, dogs and humans are both biological organisms, and mammals, while LLMs are not. If we choose to restrict the term “understanding” to biological organisms with brains, that is a matter of language, not a reflection of function.