On understanding, intuition, and Searle’s Chinese Room

You’ve just bought the latest in personal-assistant robots. You say to it: “Please put the dirty dishes in the dishwasher, then hoover the lounge, and then take the dog for a walk”. The robot is equipped with a microphone, speech-recognition software, and extensive programming on how to do tasks. It responds to your speech by doing exactly as requested, and ends up taking hold of the dog’s leash and setting off out of the house. All of this is well within current technological capability.

Did the robot understand the instructions?

Roughly half of people asked would answer, “yes of course it did, you’ve just said it did”, and be somewhat baffled by the question. The other half would reply along the lines of, “no, of course the robot did not understand, it was merely following a course determined by its programming and its sensory inputs; its microprocessor was simply shuffling symbols around, but it did not understand”.

Such people — let’s call them Searlites — have an intuition that “understanding” requires more than the “mere” mechanical processing of information, and thus they declare that a mere computer can’t actually “understand”.

The rest of us can’t see the problem. We — let’s call ourselves Dennettites — ask what is missing from the above robot such that it falls short of “understanding”. We point out that our own brains are doing the same sort of information processing in a material network, just to a vastly greater degree. We might suspect the Searlites of hankering after a “soul” or some other form of dualism.

The Searlites reject the charge, and maintain that they fully accept the principles of physical materialism, but then state that it is blatantly obvious that when the brain “understands” something it is doing more than “merely” shuffling symbols around in a computational device. Though they cannot say what. They thus regard the issue as a huge philosophical puzzle that needs to be resolved, and which may even point to the incompleteness of the materialist world-view.

The two sides then declare each others’ intuitions to be really weird, and utterly fail to agree.

John Searle, of course, presented his well-known “Chinese Room” thought experiment as a claimed refutation of the idea that mechanical processing of information was sufficient for understanding.^[1] The thought experiment supposes that a man who cannot speak Chinese is in a room filled with books of instructions for giving Chinese-language responses to Chinese-language questions. By mechanically following the instructions he can produce Chinese-language responses sufficient to engage in conversation with a native Chinese speaker and pass the Turing Test.

Searle then claims that, because the man doesn’t understand the conversation, that shows that the mechanical processing of information going on in the room is insufficient for understanding.

The “systems reply” points out that the man is included in the scenario as a piece of stage-magician’s trickery, there to divert attention from where the action actually is. The man could be removed and replaced with a mechanical device (the Searlites can hardly object). The room as a whole, though, does understand, assert the Dennettites.

What, then, is the Searlite’s argument that the mechanical room, replete with instructions, does not “understand”? Actually, they don’t have an argument, they just declare it intuitively obvious.

Searle then introduces a ploy to keep the man centre stage, and attempts to rebut the “systems reply” by suggesting that the man memorises the entire instruction set and then walks out of the room. Searle asserts that, since the man has only rote-memorised a whole set of rules, he still does not actually “understand” Chinese.

Let’s examine that claim. According to the thought experiment, the man is then walking around conversing with Chinese speakers and giving them no inkling of any lack of understanding. He must thus be giving sensible responses to (Chinese-language versions of) “What brilliant weather today!”, and “Is the train station that way?” and “Would you be so kind as to carry this heavy shopping bag for me?”.

That means that the man must be able to link strings of Chinese-language phonemes to what he is seeing with his visual system and to other knowledge he has about how the world works, and then use that information in constructing his reply. And yet Searle insists that the man cannot be “understanding” Chinese.

Well, let me tell you about a French boy, born in Paris to French parents but now living in London. He is a native speaker of French, though while at school and with his English-speaking friends he talks English fluently. But he doesn’t actually “understand” English. What he’s done is merely rote learn enough about how to respond to English-language speech with more English-language speech, such that he passes as understanding it. But he doesn’t. He understands French, but he doesn’t understand English; he is merely simulating that understanding in order to get by.

I hope that readers will regard that idea as preposterous. Of course the boy understands English! There is nothing about the concept “understanding” that is not fulfilled by that boy’s capability with the English language. Yet Searlites would have us believe that it would be entirely possible for the boy to have no actual understanding of English, despite being fully fluent in English in the company of his friends.

After all, that is what they are claiming about the man who has memorised the Chinese Room. They even claim this lack of understanding to be obvious! And yet, they never put their finger on what is missing. They don’t give an account of what “understanding” actually is and what is missing in the case of the Chinese Room or of the English-speaking French boy. They given no operational test for the presence of “understanding”, nor any method for distinguishing an English-speaking French boy who does “understand” English from an English-speaking French boy who does not.

This, to me, reveals the utterly absurd position that people are driven to by their initial ideological intuition that understanding must be more than the “mere” shuffling around of symbols in a computational device. I say that quite deliberately, since Searlites accuse Dennettites of exactly the same: being driven to an absurd position by the reverse ideological commitment (for which see below).

The only resort of the Searlites would be to argue that the man, who has memorised and wandered out of the Chinese Room, only passes the Turing Test in a very restricted sense, with a rule banning any reference to anything in the real-world. Indeed, Searle needs to insist that the man “doesn’t know that the story refers to restaurants and hamburgers, etc”.

The problem here is that nearly all speech does refer to the real world. An alien who knew the rules of grammar, but nothing about the world, might reply to “Did you like the music?” with “It was bright orange”, or “not unless Peter starts with a Q”, and thence fail the Turing Test.

If someone asks how many fingers they are holding up, any valid reply would necessitate knowing what the answer meant. Or, if one wants to be facetious, one could simply hold up a hamburger to Searle’s man and ask “what do you call this?”. This shows that Searle’s conception of a competent but non-understanding conversationalist is simply incoherent.

Of course, if the Searlites want to admit that, of course the man could not actually converse in Chinese, because he would not know about the linkages between Chinese-language phonemes and real-world knowledge, and so could not give a valid reply to “what do you call this?” and the multitude of similar questions, then I’d happily grant that, yes, that man does indeed lack understanding of Chinese. The remedy, then, would simply be to program in those missing linkages.

The whole point about speech is that most of the concepts in it are about the real world. Indeed, when constructing AI programs, the hardest bit to get right is not the algorithms for processing information, but the databases containing all the information about how the world works. For example, Google’s software is now pretty good at translating Chinese to English because it uses a dataset of 1.8 trillion tokens gathered from Google Web and News pages. Searle’s conception of a competent language speaker who is incompetent at the real-world linkages of the phonemes is simply self-contradictory.

What this whole argument lacks is an agreement about what “meaning” and “understanding” actually are. Someone like me would give a prosaic and straightforward account. I’d say that the “meaning” of a piece of information is how it relates to other pieces of information. Thus the “meaning” of the word “water” is a whole set of linkages to other concepts, including ”wet”, “transparent”, “rain”, “puddle”, “drink” and many more. For a computational device to “understand” the “meaning” of a symbol is then simply to be able to access these linkages and to use them appropriately in processing information.

Thus, if I say to my iPhone “Siri, Quelle heure est-il?”, and the iPhone correctly responds: “Il est deux heures”, then the iPhone has “understood” the “meaning” of my question, because it has correctly interpreted the strings of phonemes, pattern-matched them in a way that caused it to consult its inbuilt clock, returned that information to a speech processor, and then sent signals for the appropriate stream of sound to its speaker.

The fact that this is all entirely mechanical and deterministic doesn’t change that conclusion. Afterall, what do you think our brains are doing when we understand a question?

If, on the other hand, I were to say to my iPhone, “Siri, cik ir pulkstens?”, then the iPhone would not understand for the simple reason that it has not (yet) been programmed with the Latvian language. Thus it does not “know” (has not been programmed with access to) the linkages between Latvian-language phoneme-strings and other relevant symbols within its memory.

Thus, for now, my iPhone understands French but does not understand Latvian (though that might change in a future update).

That sort of argument, though, causes the Searlites to roll their eyes. They insist that there must be something more to it than that! Such an account could not possibly be all there is to “meaning” and “understanding”, that’s way too easy!

Searle is supposed to have produced an argument that the mere manipulation of symbols (syntax) cannot possibly produce meaning (semantics). I’ve never quite fathomed what this argument actually is, though it seems to involve the claim that the way symbols are shuffled in a computer is unrelated to their “meaning” and thus that the shuffling of symbols alone cannot produce meaning. And yet, again, this claim is made without any account of what “meaning” actually is.

In the account of meaning that I’ve just given, Searle’s claim is just untrue. Manipulating symbols in terms of their meaning (= in terms of their linkages to other symbols) is precisely and exactly what the computer is indeed doing.

I tried the above account of “understanding” on the philosopher Massimo Pigliucci on his website. Dr Pigliucci responded with:

As for my iPhone “understanding” things, I’m simply baffled by your insistence on this. And I do attribute it entirely to your ideological commitment.

But then, interestingly, he added:

As for iPhone intuitions, let me get this straight: are you claiming that my iPhone has the sort of internal mental states, a feeling of understanding, like I do when I read your words? Because if not, then you are simply playing word games.

Now I find that revealing. Dr Pigliucci is suggesting that understanding only counts as “understanding” if it involves a first-person subjective reflection on that understanding. Thus, one needs to consciously know that one understands in order to understand.

Let me clarify that I’m not suggesting anything of the sort regarding an iPhone. Of course an iPhone is not doing a first-person conscious reflection about the tasks that it is doing — it is merely doing those tasks. What I am suggesting, though, is taking the most straightforward and minimalist account of “understanding” that one can, one that fulfils the basic definition but no more.

According to the Oxford English Dictionary what is “meant” by something is what it is “intend to convey or refer to” or what it “signifies”. Further, to “understand” something is merely to “perceive the intended meaning”. Which bit of that is not fulfilled by the above iPhone example, or by the personal-assistant robot?

But, perhaps this explains the mutual bafflement of the two camps. While the Dennettites are taking a minimalist interpretation of the concept “understanding”, the Searlites may be doing the opposite. They are wrapping the concept up with the whole issue of consciousness. That would make the Chinese Room not about “understanding” per se, but about consciousness. (And it would perhaps have been helpful if Searle had said that in his original paper.)

The argument would then be that the mechanically-operating room is not consciousness, and that without being conscious the room cannot be understanding. The man is then a necessary part of the room since he supplies the consciousness, and since that is divorced from the speaking-Chinese function of the room, then the room cannot be “understanding”.

If the two camps are making such fundamentally different interpretations of what the whole discussion is actually about, that would explain their bafflement at each other’s intuitions on the topic.

It seems to me that conflating issues such as “meaning” and “understanding” with consciousness is unhelpful, and is a way to ensure we cannot easily make progress. The classic way to understand any issue is to break it down into component parts and to try to understand the parts. Thus, to make progress, we should seek accounts of “meaning” and “understanding” that are distinct from the issue of consciousness (an issue which is, I will readily concede, much harder to resolve).

One objection needs dealing with. The objector might point out that, in the same way that my personal-assistant robot responds correctly to the instructions issued, a thermostat responds correctly to a fall in temperature, and switches on a heater. Does the thermostat then “understand”?

To answer we need to realise that biological phenomena are very often continua rather than being binary. Properties such as “intelligence”, “awareness”, “meaning”, and “understanding” will always be matters of degree. That follows from the fact that animals possessing such capabilities all developed from a single fertilized egg that lacked them, and then developed gradually. It also follows from the biological absurdity of a child being radically different from its parents. Therefore a parent who lacked “intelligence”, “awareness” and “understanding” could not give birth to a child who did develop those capabilities. Therefore a binary conception of such traits is ruled out and they must be matters of degree.

One can always, then, “turn down the dial” of “intelligence” or “awareness” until all that is left is much simpler behaviour. This is not a problem for the conception of “understanding” that I’ve presented, it is simply how biology works, and is thus a necessary feature of any correct account of the concepts.

We use terms such as “intelligence” and “understanding” only for behaviours above a certain degree of complexity — though the threshold is, of course, merely one of convenience. Certainly, today’s ten-yr-olds have no problem with saying that an iPhone exhibits some degree of “intelligence”, “awareness” and “understanding”, even though those degrees are much less than possessed by a human.

Meanwhile, this whole discussion reveals how much our arguments depend on our intuitions and how, as a result, it is easy to misunderstand others who think differently.

Notes: [1] The original paper by Searle is online here as a pdf.