Philosophy, Perception and Neuroscience

John Smythies

Center for Brain and Cognition, University of California San Diego, La Jolla. California and Institute of Neurology, University College, London. smythies@psy.ucsd.edu

Perception 38(5) 638-651 (2009)

This paper presents the results of some recent experiments in neuroscience and perceptual science* that reveal the role of virtual reality in normal visual perception, and the use of television technology by the visual brain. This involves particularly the cholinergic system in the forebrain. This research throws new light on the nature of perception and the relation of phenomenal consciousness and its brain. It is directly relevant to criticisms by certain analytical philosophers of aspects of neuroscience relating to these matters. In particular I pay attention to their support for Direct Realism.

Introduction.

A number of recent experiments in visual neuroscience have profound implications for neurophilosophy, news of which has not filtered through as yet to the main body of philosophers. [start * ] Before presenting this evidence some clarification of terms is indicated. It is important to distinguish between several different Realist theories of perception in philosophy—two species of Direct Realism (Naive and Critical), Physiological Realism, Phenomenal Realism and Color Realism, to name the most prominent. Most discussion in this area has been based on the supposition that perception is a unified process. However, evidence from clinical neurology suggests that this is not so. Rather phenomenology and epistemology are based on activity in two different brain areas. In associative agnosia the patient can see objects perfectly well—her phenomenology is intact. She can pick up objects on a table and does not bump into things when she moves about. But she cannot recognize objects and does not realize what they are used for. Her epistemology is impaired. A patient with blindsight, in contrast, cannot see objects. She cannot pick up objects from the table and bumps into things when she moves around. But, when asked to “guess” what the object is out there, she is frequently correct, and can tell you how to use it. She can also correctly name colors. In the allied but distinct condition of aphasia, a patient can see and recognize objects, but just cannot find the name for them. Blindsight is mediated by direct connections between the lateral geniculate body and the higher visual cortex that bypass the primary visual cortex area 17. Thus epistemology and phenomenology have a profoundly different cerebral basis and their philosophical relationship needs to be re-evaluated in this context.

Realism

On this basis the various realist theories can be categorized as follows:—

Naive Realism. This states that both components of perception—phenomenal and epistemological—are direct. That is to say that (1) phenomenological objects seen are identical with physical objects seen (or at least with their surfaces (Stroll 1988) or at least part of the time) and (2) no process of inference is used when we ordinarily look at external objects and obtain information about them. Perception, in this view, has perfect fidelity. In contrast, in the old sense-datum theory we are supposed to infer the properties of the external physical objects we look at by inferences based on our experience of the sense-data that we experience directly.

Critical Realism. This maintains a direct epistemology but adopts a form of ecological realism as developed, for example, by J. J. Gibson (1979). Critical Realists do not presume that perception has perfect fidelity—only sufficient fidelity to allow organisms to act toward their environments in relatively safe and successful manners. It is presumed that any misinformation detected is neither total nor reliable. I will return to this topic later after we have looked at the evidence.

Physiological Realism (also known as the Representative Theory of Perception). This holds that phenomenological perception is always fully indirect. All phenomenal objects and events are constructions of brain mechanisms, and in no case is any phenomenological object identical in any way with the exterior physical object it represents. Physical objects stand at one end of a lengthy and complicated causal chain and phenomenal sensations stand at the other. However, the theory supports a direct epistemology. That is knowledge obtained by perception refers directly to external physical objects and their properties and no indirect inferences are involved. All knowledge is mediated by brain mechanisms but it is nevertheless direct. When I look at a red rose, neural impulses travel from the higher visual areas of my brain to the cognitive and language areas, and generate this knowledge directly. I do not examine a red sensation and deduce that there is a red rose out there, as I would deduce that a human had passed that way from a footstep in the sand. As Wright (1981, 1983) has repeatedly emphasized, sensations are phenomenal, but not epistemic, intermediates in perception.

Phenomenal Realism. On a somewhat different tack this holds that phenomenal properties (or qualia) are not conceptually reducible to physical or functional properties.

Color Realism. This is a subsection of direct realism which states that colors, as experienced, are located on the surfaces of exterior physical objects. Recently Hardin (2008) has published a detailed critique of this theory based on the experimental demonstration of a multifaceted lack of correspondence between the colors we experience and color-generating features of the physical world. Furthermore he shows that there exist whole families of experienced colors that cannot be realized by any material surface. Examples are super colors, and “impossible” red-green and yellow-blue binary colors that result from the filling-in of retinally-stabilized images. [end *}

Enigma variations

The evidence to be presented is also relevant to the present spate of activity in analytic (Wittgensteinian) philosophy, in particular in the area of the philosophy of neuroscience in which three influential works have recently been published (Bechtel, Mandik, Mundall, and Stufflebeam 2001; Bennett and Hacker, 2003; Bennett, Dennett, Hacker and Searle 2007). Some philosophers, aware of the steady inroads that cognitive neuroscience has made into their subject*, have made a number of spirited counter-attacks. The first I will consider is by Bennett & Hacker (2003) and Bennett et al (abbreviated henceforth as B & H) (2007). The claim made here is made that contemporary cognitive neuroscience is riddled with conceptual errors and that they, these philosophers, have demonstrated what these mistakes are.

Alleged mistake 1. Neuroscientists say that the brain sees, thinks, makes decisions, processes information, and uses codes and maps, etc., when it is really people, who do all these things. The brain and its activities “ ... only make it possible for us, not for it, to perceive and think, to feel emotions and to form and pursue projects.” (B & H. pp. 7-8).

Alleged mistake 2. Neuroscientists believe they can reduce “all mental and psychological attributes like beliefs, desires, understanding, and sensory experiences of every kind” to “theoretical, physiological accounts.” but “ ... there are internal, inexplicable things going on within us, [and] the things we do and experience cannot all by explained or accounted for by pointing to some physical origin or process ... The brain is merely another organ of the body, the purpose of which is to facilitate the various things human bodies do, such as thinking, walking, seeing, desiring, and interpreting ... .Each of us exists as a whole creature, ... that it is, in some enigmatic and marvelous way, much more than a sum of its parts.” (Denneson 2007). “The brain is not an organ of consciousness. One sees with one’s eyes and hears with one’s ears, but one is not conscious with one’s brain anymore than one walks with one’s brain.” (B & H p. 135)

These philosophers further claim that the proper task for neuroscience is limited to the description of how the brain works as an electro-chemical machine, in terms of neuronal properties and activities. The task of finding out how all this activity is related to human functions, such as seeing, thinking, feeling, deciding, being conscious, etc., in contrast, they claim, belongs to the field of analytical philosophy. The task here is to analyze these mentalistic concepts by finding the “correct” account of their “logico-grammatical character” (B & H, p, 110).

This account, however, contains some *errors.

Firstly, it supposes that the only branch of science that is relevant to the matter in hand is neurophysiology (to which we can add cybernetics). However, B & H ignore the extensive evidence from psychophysics and perceptual science (see Galton (1881), Gregory (1981), Vernon (1962), Ramachandran & Blakeslee (1998), Smythies and Ramachandran (1998), myself (1959/1960) and many more) who study the contents of phenomenal consciousness in such areas as the filling-in of scotomata, after-images, eidetic images, constancy effects, number forms, the stroboscopic phenomena, hallucinations of various kinds, synasthesiae, stabilized retinal image experiments, a very wide range of the effects of brain injuries on our experiences, the role of televisual and virtual reality mechanisms in visual perception, etc.. These studies produce information about the nature of phenomenal consciousness itself, which could not have been obtained by neurophysiological, or brain-imaging, methods. An account of the nature phenomenal consciousness that fails to mention any of this research, and takes note only of discussions about the meaning of words, is. I suggest, flawed*.

B & H also fail to understand the nature of the phantom limb phenomenon described by neurologists. For linguistic reasons, they claim that all real pains are located in the physical body. In the case of phantom limb pain they say that the felt pain is “real” (because, and only because, the patient shows the behavioral reactions, such as writhing, their theory requires for all and any ascription of ‘pain’) but its location is “illusory”—“His leg does not hurt because he has no leg.” They emphatically deny that phantom limb pain is in the head, or in the brain. They admit that ordinary headaches certainly occur in the head, but that is all—and even these occur in the meninges, and not in the brain.

This use of the word ‘illusion’ when referring to a phantom limb pain is, however, technically incorrect. In the first place, most people would say that a pain is a phenomenological event that we experience, and not an ascription derived from our behavior. Secondly, phantom limb pains are hallucinations not illusions. Illusions refer to such phenomena as ‘the-bent-stick-in-water’, or the Ames room. In an illusion a normal percept is distorted. It makes no sense, therefore, to say that an hallucination has illusory properties, as there is no real object ‘behind’ an hallucination to be distorted. This use of the term ‘illusion’ here is, I suggest, just propaganda. Since hallucinations cannot have illusory properties, the use of ‘illusion’ is designed subtly to insert in the mind of the jury the idea that such phenomena (so damaging to the theory) are ‘not real—not important—forget them’. *Moreover, this debate has become influenced by the experimental evidence that the entire perceptual theory supported by B & H that is central to all their arguments (i.e. Direct Realism) is itself incorrect (see below). It is also surely relevant that phantom limb pain is often real enough to drive the patient to suicide.

Physiological realism holds phantom limbs, and all other bodily sensations that we experience, are located in the body-image that we experience during waking hours, and they are not located in the physical body. Events in the latter constitute only the causal ancestors of bodily sensations. As the great Viennese neurologist Paul Schilder (1942) said, “The empirical method leads immediately to a deep insight—that even our own body is beyond our immediate reach, that even our own body justifies Prospero’s words, “We are such stuff as dreams are made on: and our little life is rounded by a sleep.’”

There is much evidence to suggest* that all sensations, including the body-image, are phenomena constructed by the representative mechanisms of perception, and involve a series of distributed, hierarchical, parallel neurocomputational and virtual reality mechanisms. These do not facilitate the perceptions that we carry out independently “in some enigmatic and marvelous way” (Denneson 2007), but, in conjunction with other features of the physical chain of perception, constitute the entire process of perception itself—all the way from photons reflected from external objects to sensations in phenomenal consciousness. It is not the case that a human being consists of a body and a brain plus a person for whom the brain acts as a “facilitator”—particularly as B & H claim “Human beings are not their bodies” (p. 134).

In the case of vision we can go further. Philosophers assume that vision consists simply of seeing what is ‘out there’. What is out there (‘reality’) is what we see. However, there is now *experimental evidence that this is not the case, and that what we actually see is always a mixture of reality and virtual realty. Phenomenologists (e.g. Fitzgerald, 1978; Libet, 1981; Schilder 1942; Teschner 1981; Wright, 1983) describe a visual field in which visual sensations are distributed in the form of a topographic map of that part of the external world being looked at during that instant—as Crick (1994 p. 159) describes it “We have for example a vivid internal picture of the external world.” They further suggest that this visual field is constructed by a representative mechanism along lines familiar to television engineers.

There is now considerable experimental evidence that the visual field is constructed by such a mechanism. The first data was obtained by studies of how vision returns after injuries to the occipital lobe. Schilder (1942) summarized the results of such studies carried out by neurologists such as Goldstein and Gelb in Germany in the 1930’s. One might expect after such an injury that vision returns by allowing one at first to see objects fuzzily with later clarification as healing progresses. However, the reality is quite different. The first thing to return is the perception of movement. On looking at a scene the patient sees no objects, but only pure movement usually rotary that occupies the visual field. *Then luminance is experienced but in the formless form of a uniform white Ganzfeld. Later ‘space’ or ‘film’ colors appear that float about unattached to objects (which are not yet visible as such). Then parts of objects appear—such as the handle of a teacup—that gradually coalesce to form fully constituted (phenomenal) objects, into which the film colors then enter. This indicates that the three basic cortical mechanisms for computing movement, color and shape have different rates of recovery. It is difficult to see how the theory of Direct Realism can account for this data. Are we to suppose that we have three separate components of “the brute fact of our existence that we see physical objects” (as modern philosophers like to put it) and that these three processes meld mysteriously into one during normal seeing?

Another related series of experiments has examined ‘binding’. This is how the brain allots the appropriate color to an object. The neurocomputations, related to our perception of the color, shape and movement of external objects, are carried out in anatomically separate brain areas. How these finally come together to produce the final unitary object, in which the color is inside the shape and both move together, is the still unsolved ‘binding problem’ in neuroscience. Recently a series of psychophysical experiments have also shown that, under certain experimental circumstances, the normal brain will allot the wrong color to an object (Treisman and Schmidt 1982: but see Quinlan 2003). Balint’s syndrome and psychedelic drugs like LSD induce a wide range of such phenomena (Smythies 1953; Robertson 2003),

The second body of evidence that supports a TV theory of perception came from a series of experiments that suggest* that we do not perceive what is actually ‘out there’ but what the brain computes is most probably ‘out there’, or, as Crick (1994) put it, “What you see is not what is really there; it is what your brain believes [substitute ‘computes’] is there.” Ramachandran and his colleagues carried out the first of these experiments. They showed that scotomata—the blind patches caused by local injury—are not perceived as patches of nothing, but that the brain fills them in with a continuation of whatever pattern surrounds them.

The third series of experiments was reported by Kovács, Papathomas, Yand and Fehér (1996). These workers took two photographs, one of a monkey's face and the second of a leafy tropical jungle. They converted these into two pastiches each composed of portions of each photo, so that in the location where one photo showed part of the monkey's face the other showed leafy jungle. Then each pastiche was shown separately to each retina, so that retinal rivalry occurred. Under these circumstances, the subject did not see what was actually there—that is the two pastiches alternating—but rather a complete monkey face alternating with a complete leafy jungle. Clearly the brain had suppressed the improbable mixed pastiche in favor of what it was familiar with (and thus computed what was more probable). Many other experiments, based on stimuli such as moving plaid patterns, have shown this phenomenon where the perception of an improbable input is suppressed by the brain, and replaced with the perception of what it computes to be more probable ones.

Even more crucial experiments (Kleiser, Seitz and Krekelberg 2004) have shown that, during a saccade (rapid movement of the eye), information coming from the eye is suppressed, and what we see is largely virtual reality created by the brain from processed memory. These authors expressed it thus:

“When you look into a mirror and move your eyes left to right, you will see that you cannot observe your own eye movements. This demonstrates the phenomenon of saccadic suppression: during saccadic eye movements, visual sensitivity is much reduced.”

In other words, 'filling in' has a temporal as well as a spatial dimension. This function is based on a widespread neural network, that includes the superior colliculus and parts of the thalamus. The purpose of this would appear to be to suppress the violent swings in the visual input (the world swirling round one) and resulting severe vertigo that would otherwise result. Since saccades occur very frequently, this means that virtual reality plays a major role in normal everyday seeing.

We now also know something of the mechanism by which the brain effects this function of mixing reality and virtual reality in visual perception. In this the brain uses technology already worked out by television engineers for their own purposes. These engineers found out that the efficiency of digital television transmission can be increased in the following manner. The scenes televised consist of a focus of attention (foreground), where the action is, plus a lot of background, where nothing much happens. The computations on which digital television is based are expensive in terms of computational time and monetary cost. To reduce this, the computations are concentrated on the foreground (’reality’), as transmitted by the TV camera, and much of the background is provided from memory combined with computations on what is likely to happen next, given the previous history of the program (‘virtual reality’). The former is accurate but expensive: the latter is relatively cheap but ‘fuzzy’. The art of TV engineering is to provide the optimum mixture of these two processes.

Interestingly enough the brain does the same. The input from the retina that carries information about ‘reality’ is carried by the optic nerve and radiation to the visual cortex where the axons terminate in layer IV. The input from the brain’s neurocomputationally active memory and prediction systems, that mediate the virtual reality component of visual perception, is carried by cortico-cortical connections to the visual cortex that terminate in layers I and II. The balance of these two systems is mediated by the neuromodulator acetylcholine. The cell bodies of these cholinergic neurons are located in the Nucleus Basalis of Meynart that is located in the basal forebrain. The axons of these neurons project to the cortex, where they activate stimulatory nicotinic receptors in layer IV, and also inhibitory muscarinic receptors in layers I and II. Now, when nothing much interesting is happening in the environment, there is a low level of activity in the Nucleus Basalis, in which case the cortico-cortical input to the visual cortex is active mediating virtual reality. Then, when a new stimulus is received that indicates that something important may be happening outside (for example “predator!”), this nucleus is activated, releasing acetylcholine at its axon terminals. The projection to layer IV then activates stimulatory nicotinic receptors on the bodies of those pyramidal neurons that receive the retinal input, which is thereby promoted. The simultaneous cholinergic projection to layers I and II stimulates inhibitory muscarinic receptors on the bodies of pyramidal neurons that receive input from cortico-cortical projections that mediate ‘virtual reality’, which is thereby inhibited. *The function of all this is to prevent the severe vertigo that would otherwise result. Thus the brain can now expend its energies in analyzing the new stimulus for salience. It seems to me* that Direct Realism (DR) has difficulties in accounting for this data. It is implausible * to claim that we see external objects directly when our eyes are still, but indirectly when we execute a saccade.*

Wittgensteinian philosophers say that the rules of grammar, that we must abide by to make sense, entail that the verb ‘see’ must have an object and that object must be a real physical object in the external world. This rule is supposed to be set by the way we learned how to use words at our mother’s knee. Therefore, in cases of hallucination, we must say “I seem to see X” and not “I see X”. Alternatively I can say “I am having an experience like I would have had had I been seeing X, but I am not seeing X or anything X-like”, and various things of that kind. However, one can raise objections to this*. These word games do not really help*, and merely serve to confuse the issue. Wittgenstein *always said that you should attend to what people actually say (and, one can add, not to what they should say to make your theory true). People, when they are describing their hallucinations, whether natural or evoked in the laboratory, invariably say “I see X” and not “I seem to see X” (or some such). I spent two years in the Psychological Laboratory at Cambridge studying the stroboscopic phenomena (Smythies 1959/1960). These are the geometrical patterns that fill the visual field when we look at a flashing stroboscopic light. My subjects were all psychologically sophisticated faculty members, or graduate students. When describing what happened, when they looked at the flashing light, they all said “I see geometrical patterns—squares, grids, stars” and proceeded to describe these in ordinary language. Not one of them said, “I seem to see geometrical patterns squares, grids, stars, etc.”, or things of that kind. B&H claim, “That what one seems to see, when one hallucinates a red apple, is red and round does not imply that one has a red, round visual hallucination.” (p. 158). My subjects were perfectly content with the idea that what they were seeing were visual hallucinations of various shapes (some were round) and colors (some were red)—there was no ‘seeming’ about it. So I suggest Wittgensteinian philosophers are wrong*, when they claim that a study of logic, speech and grammar, as they are actually used in the relevant circumstances, supports their case.

Much of this confusion arises from the use by philosophers of the unfortunate term ‘qualia’, defined as what it is like to experience X (for example red). It might be* better to stick to basic phenomenology and discuss instead what it is to experience X. It is obvious that one experiences the redness of a red veridical sensation, a red after-image, and a red hallucination in the same way, and that we see the redness of an external physical object in a different way—i.e. because the atoms of its surface are such as to reflect rays of light of a certain wavelength from its surface onto the retina, and because our subsequent red sensations are phenomenological, but not epistemic, intermediaries in the perception (Wright, 1981).

B & H continue, “For, whatever colours are, they are not “sensations in the sensorium” (p. 10). The recent evidence, that I have related above as to how vision actually works, shows that this unsubstantiated statement is simply untrue (Hardin 2008). Experienced colors are generated by neurocomputations. They exist in the visual field attached to phenomenal objects by specific neural mechanisms, which can be disrupted by certain cerebral lesions, so that these colors leave the phenomenal object and become space or film colors, as described earlier. Colors, as experienced, are not located on the surfaces of external physical objects.

Consider also the following reports from people who have taken the hallucinogenic drug mescaline (my italics)—

“”My mind was perfectly clear and active; ... stretched out upon the bed, with closed eyes, an ever-changing panorama of infinite beauty and grandeur, of infinite variety of colour and form, hurried before me.” (Prentiss and Morgan, 1895).

The next subject saw masses of transparent fruit,

“ ... to give the faintest idea of the perfectly satisfying intensity and purity of these gorgeous colour-fruits is quite beyond my power. All the colours I have ever beheld are dull compared with these.” (Weir Mitchell 1896).

Havelock Ellis himself (1897) reported “Perpetually some totally new kind of effect would appear in the field of vision: sometimes there was swift movement, sometimes dull, somber richness of colour, sometimes glitter and sparkle, ... I was further impressed, not only by the brilliance, delicacy, and variety of the colours, but even more by their lovely and various textures—fibrous, woven, polished, glowing, dull, veined and semitransparent ... ”.

A subject of Rouhier (1927) said “ ... high above me is a dome of the most beautiful mosaics, a vision of all that is most gorgeous and harmonious in colour. The prevailing tint is blue, but the multitude of shades, each with such wonderful individuality, make me feel that hitherto I have been totally ignorant of what the word colour really means. The colour is intensely beautiful. Rich, deep, deep, deep, wonderfully deep blue.”

Clearly there is more to color than B & H propose.

Philosophers are certainly correct when they invite neuroscientists and psychologists, who use ordinary terms like ‘map’, ‘code’, ‘information’, etc. in a technical sense, to explain just what that sense is. That is indeed helpful at times—but not in this case. Take ‘map’ for example. B & H make much of the claim that the brain cannot contain any maps. Maps are things you unfold and look at when trying to find your way around. The brain contains nothing like that. They say “ ... to use a map as a map, there has to be a map—but brains lack eyes and cannot read” ... nor is the brain, they say, familiar with the projective conventions used in maps, nor can the brain use the map, like we do, to guide its own behavior, as it has no behavior. However, one can reply that organisms can certainly contain maps that direct their movements. The US Air Force has warplanes that have maps of their target digitally coded into their directing computers that enable them to carry out precision bombing. It is this sense of having a map in its brain that neuroscientists employ. Take ‘codes’: for similar reasons B & H say that brains cannot contain codes either. However, Tsien (2007) has produced evidence that they do and just, in this instance, what this code is. By means of a technical tour-de-force he found a group of neurons (called a clique) in the mouse hippocampus, that respond only to stimuli generated by a potential nest (any place to curl up and sleep), regardless of its color, shape and materials it was constructed out of. Tsien says “A clique is a group of neurons that respond similarly to a select event and thus operate collectively as a robust coding unit.” Firing of this clique signals to the mouse’s neurocomputational system “That is a nest out there”. Other cliques respond to other conceptual inputs. These cliques respond to concepts and not merely to specific stimuli (shapes, patterns, colors etc). Tsien then shows that the activity of several cliques can be translated into a string of binary code, which reveals details of what the events the animal has experienced that lead to the formation of specific memories. So why should we say that the clique is not part of a coding mechanism? B & H’s claim that the word ‘code’ cannot be used meaningfully unless a human has encoded the message and another has decoded it, is arbitrary in the extreme. Do these authors object to the concept of the genetic code, as their ‘logic’ should force them? They do not say.

A further link between seeing and television is provided by the interesting fact that, if we illuminate an analog TV studio with a flashing light, geometrical patterns appear on the TV screen, whose form is determined by the geometry of the scan being used and the ratio of the frequencies of the scan and the flashing light. Many of the stroboscopic patterns I described above, that we see if we look at a flashing light, are functions of the way a linear scan can cover a flat surface (grids, spirals, stars and mazes). As the neurophysiologist Grey Walter (1950) said in discussing this phenomenon. “In other words, the televisual system behaves very much like the neuro-visual one.” Moreover the nature of link between our visual sensations and the physical objects they portray can be modeled very simply by television. The TV picture is the equivalent of the visual field. We can use a TV picture to watch, say, a football match (=perception)—or we can examine it as it really is, i.e. a flat glass screen covered with colored lines (in analog TV) or tiny colored dots (in digital TV)(=introspection). Nothing could be simpler. Thus a model of the way we experience our visual sensations (veridical and hallucinatory) is provided by the way we see images on a television screen as these are in themselves (constructed by a raster or pixels): and a model of the way we see physical objects is provided by the way we watch a football match on television. Although, as we have seen, common usage covers the use of the word ‘see’ for hallucinations, it is better in psychology to use the technical term ‘observe’ for hallucinations and veridical sensations encountered in experiments, and reserve the technical term ‘see’ for seeing objects.

*Physiological realism holds that there are no “inexplicable things going on within us”. We are not, “ ... whole creatures, ... that it are, in some enigmatic and marvelous way, much more than a sum of their parts.” A human being is a physical body with a phenomenal consciousness, which are both accessible, in their different ways, to scientific observation (Smythies 1994). Humans may be marvelous, but they are not enigmatic.

Some objections. [start *]

Having reviewed the relevant evidence we can now return to Critical Realism. We have seen how perceptual mechanisms in the visual cortex mix input from other areas of cortex (‘virtual reality’) with input from the retina (’reality’) in everyday perception. But it could asked “How do we distinguish between what is real (what’s ‘out there’ and thus doesn’t have to be filled in) and what is virtual (what’s not ‘out there’ and thus has to be filled in)?” In reply one must first note that it is not “we” that makes this distinction but our brains do. A scotoma to the brain is an area of inactivity surrounded by a larger area of normal activity. The dynamics of brain processes are such as to promote the spread of the surrounding normal activity into the inactive area. “We” do nothing in all this. In any case the brain is not primarily concerned with distinguishing between ‘real’ and ‘unreal’. Its main terms of reference are in terms of Darwinian salience—‘significant’ versus ‘unimportant’.

Likewise in the case of saccades, when the eyes are still, the NCC neurons in the visual cortex are stimulated mainly by impulses from the retina. During a saccade these neurons are stimulated mainly by cortico-cortical impulse modulated by memory and prediction. Thus ‘reality’ and ‘virtual reality’ in this context are technical terms borrowed from television technology that have nothing to do with the use of the word ‘real’ in common speech. The same analysis applies to the criticism that virtual percepts are hallucinations and are thus unreal—so how could the real and the unreal interact in the way described? The answer is that the sensations involved in hallucinations are just as real as are the sensations involved in veridical perception. It is merely their causal antecedents that are different. Their interaction in the way I have described described is mediated by a particular cholinergic mechanism. Virtual reality does not present “misinformation” but has been developed by evolution further to process sensory information in order to make the brain do its task more efficiently.

Another criticism has been raised by Fodor and Pylyshyn (1981) as follows. The judgement of what is most probably out there must be grounded in direct perception of what’s real rather than in computation alone, for how else could the computational chain ever be initiated? Logic, it is alleged, demands the hereditability of truth over inferences, so there must be a hereditary of reality over neurocomputation. Sometimes the virtual may intrude into the causal chain of perception but never in such a way as to overwhelm everyday life.

The reply to this is to point out that, in order to understand perception, we need to know how the brain works within the context of rigorous attention to our basic concepts used to describe this and how this knowledge is relevant. Studying the logic of statements about perception alone cannot substitute for this. There is no need to “ground our judgment of inferences of what is most probably real” on direct perception, since we do not make this judgement (our brains do that), and because no such inferences are involved, as explained above. The whole process of perception is carried out by neurocomputation in the brain (and eye). The brain perpetually computes what is most probably ‘out there’ by matching the sensory input against its relevant memory and prediction banks. All that happens is that when some wildly improbable input comes along (as in the monkey/forest pastiche) the retinal input (“reality’) to the visual cortex is inhibited and the cortico-cortical input from the memory banks (virtual reality) takes over temporarily—all orchestrated by neural mechanisms. The same process deals with the transmission of redundant information in the sensory inflow. All this makes the brain work more efficiently in a Darwinian sense.* We are not dealing here with the logical development of ideas by a philosopher, but with the details of the mode of action of an information-processing representative and control mechanism.

Thus, to recapitulate how the brain mediates perception. The map of the external world and its contents within view is thrown by the lens onto the retina, where it is converted into streams of impulses in the optic nerve and radiations that are relayed to a wide variety of locations in the visual cortex. Here the streams are processed by a series of parallel distributed neurocomputations, that includes input of virtual reality from processed memory and prediction, so that the final end product of this process is formed by activating specific brain areas including the NCCs of consciousness. From here impulses are propagated to the corpus striatum, the limbic system and frontal cortex mediating behavioral planning, emotional reactions and delivery of appropriate behavior respectively. Physiological Realism holds that here is no extra process, no “brute facts” of ‘direct perception’, tagged onto this. There is no “me” that peers out of my eyes, as ‘common sense’ believes (e.g. “I cannot see anything out of my right eye”), so that the external physical object mysteriously swims into my ken. This is the scientific argument against all forms of direct phenomenal perception.

Nevertheless, epistemologically this mechanism does mediate direct perception. This is because no inferences are involved—merely vastly complex neurocomputations backed up by an equally vast system of dynamic neurochemistry and neuroplasticity (Smythies 2002). Ecological Realism presents in fascinating detail how environmental factors modulate visual perception, but it cannot replace the neuroscientific account of how the brain works. [finish *

The neural correlates of consciousness.

The second influential work to be discussed was contained in the January 2004 issue of the Journal of Consciousness Studies that was devoted to a discussion by philosophers and neuroscientists on the topic of whether there are neural correlates of phenomenal consciousness and, if so, what they might be. No‘ and Thompson (abbreviated as N & T henceforth) presented the lead paper. They paid particular attention to Logothetis’s experiments on binocular rivalry. This was followed by a number of discussions. I will not report on the entire debate here but just pick out relevant aspects for our present purpose.

Everyone agrees that consciousness is somehow related to the activity of certain neurons in the brain, but there is little agreement about the nature of this (NCC) relationship, or even what consciousness is. *One first has to distinguish between the medical use of ‘consciousness’ (“the patient has now regained consciousness doctor”) and the phenomenal use (i.e. the content of consciousness—a person’s sensations (perceptual content), images and thoughts). N & T suggest that perceptual content has the following essential features—structural coherence, being intrinsically experiential, and active and attentional. They add that the figure-ground relation is a “global, non-atomistic property of visual experience”. They locate a figure-in-a-ground in “egocentric space defined by one’s whole body and the possible ways it can move.” (p. 15). They also mention that the visual field has the essential (topological) property of being unbounded.

They present the problem of the relation of experiential events to their correlated brain events in terms of ‘content’. They distinguish between A ‘agreeing in content’ and B ‘matching in content’, and claim that the Neural Correlates of Consciousness (NCC) relationship follows A rather than following B as the neuroscientists claim. They accuse neuroscientists of presenting a ‘snap-shot’ account of experience as a series of isolated episodes instead of what the authors regard as the ‘real active attentional’ flow. They further deny the internalist account of perceptual experience (p. 20), and conclude by saying à la Wittgenstein “Neurobiological processes ... causally enable (but do not constitute) our embodied mental life.” (p. 19), and that consciousness is not an internal occurrence of the mind-brain but is a “ ... complex set of capacities of embodied and situated agents.” (pp. 18-19).

Most of the commentators do not agree with most of this. Jack and Prinz (p. 51) claim that NT set impossibly high standards, misrepresent the neuroscientist’s case and spend most of their time digging graves for straw men. Baars (p. 29) claims the NT’s paper is a stew of confusion in which the authors frequently and skillfully move the goal posts to their own advantage. Freeman (P. 37) raises the issue of neurodynamics, neglected by N & T, and that is so important in understanding how the brain works. Metzinger (p. 67) raises the very important point of what criteria for ‘identity’ N & T use—surely essential to debate in the Identity Theory—but he does not enlarge on this further. Searle (pp. 80-82) mentions three important points:

1“The notion of a content match or a content agreement between the level of consciousness and the level of neurophysiology is strictly speaking nonsensical. It is a massive category mistake.”

2. “In spite of what N & T say, the binding problem remains a problem.” In fact, he says there are two binding problems, one relating to the construction of the actual visual field, and the other to the construction of the phenomenal objects within the visual field.

3. “If both the supervenience thesis and the NCC thesis are properly stated, the first is obviously true and the second is a trivial logical consequence of the first.”

My own response to this debate is as follows

1. The proper task at this primitive (natural history) stage of the science of phenomenal consciousness is not to define it, nor to explain it (Bechtel et al. 2001), but to describe it. A phenomenal consciousness consists of its five sensory fields as particular existents, plus the emotions, images and thoughts (plus (Berkeley) or minus (Hume) a Pure Ego) belonging to a particular individual. Some of these contents are extended spatial entities located in phenomenal space (e.g. visual and somatic sensations and images), whereas others (such as odors and thoughts) are not. Bechtel et al. (2001) give the same list but, oddly enough, leave out sensations. Many people write about consciousness without stressing that a phenomenal consciousness always belongs to some person as a part of a human organism. Each sensory field has particular contents (sensations) that change over time and that we can observe during experiments in psychophysics and perceptual science*.

2. N & T try to relate neuronal activity and events in phenomenal consciousness by talking of different types of ‘content’. Naturally the brain’s contents, i.e. neurons, and the contents of phenomenal consciousness, as listed above, are, at this level, different. So, finding ever more tortuous verbal ways of describing this difference is not likely to be helpful. It would be better to concentrate on describing the details of the actual mechanism required to construct a visual field, for example, as the workings of a representative mechanism, as detailed earlier. Experience is presented in phenomenal consciousness, not by any ‘snap-shot’ process (as the cinema does), but by a continuous process (as television does). Any televisual transmitting mechanism contains (approximately) the same information content (allowing for inclusion of virtual reality mechanisms*) at all levels, but in a variety of forms. All aspects of phenomenal consciousness represent the end results of a long series of neurocomputations. *Although we must always allow for the fact that we cannot rule out new developments in physics and cosmology that may alter our concepts of space, time and consciousness (Penrose 1990, Smythies 2003), short of that the buck stops in the cortex. There is no evidence at all that the brain unleashes any process of direct perception back into the external world, as Direct Realists claim it does. All sensations, including visual sensations, are parts of our own organisms and are not ‘projected’ into the outside world, as folk psychology mistakenly believes. *Physiological Realism holds that the red sensation I experience on looking at a tomato is a part of my own organism and is not part of the physical tomato. Only its causal ancestors are located in the physical tomato. This red sensation appears to the naive observer to be outside ‘my body’. However, this phenomenal relation of ‘outside’ in fact connects visual sensations and the somatic sensations that make up my ‘body image’ of neurology, and it does not connect my sensations and my physical body. “Externality” as Lord Brain said “is the cardinal problem”.

The third book under discussion is Philosophy and the Neurosciences (2001). Two points from this are relevant to our present discussion.

Mandik expresses Direct Realism when he says, “When I have a conscious experience of a fire engine being bright red and six feet away from me, the experience itself is neither bright red nor six feet away from me. The experience itself is a state of my nervous system that represents the fire engine as being bright red and six feet away from me.” (p. 313) This is an example of the fashionable amputation of ‘experiencing’ from what is ‘experienced’. If I look at a red fire engine six feet away from me, I can describe what is going on in terms of something vague going on in my head. That is supposed to be the experiencing, and that is supposed to be all that is going on in my brain. The result is that I see, by some mysterious process, the red fire engine before me—the object experienced—none of which is in my brain. This makes no neurological sense to me, and is, I suggest, incompatible with the evidence I presented earlier on how perception actually works—by a virtual reality supported neurocomputationally delivered televisual-like mechanism.

*The Churchlands (p. 457) present their defense of a thorough-going eliminative materialist account of all psychological entities and processes against a list of objections that have been leveled at this form of reductionism. There are currently three main theories of the relation of phenomenal events and their correlated brain events—the Identity Theory—they are simply identical; Eliminative Materialism (phenomenal events do not exist); and Functionalism (any suitable physical system (e.g. silicon-based) can be conscious, not just biological brains). I have reviewed the unsatisfactory nature of these theories elsewhere (Smythies 1994 a,b,c,d,e). Cartesian dualism (ghosts in machines) has generally regarded as having fallen by the roadside. So this whole field seems urgently in need of a better theory (see Smythies 1994a, 2003 for some suggestions).

Conclusion

The problem of basing your philosophy upon ‘common sense’ and the sanctity of common speech is that you are likely to sweep into your net all types of erroneous folk psychology, as detailed in this paper. Analytical philosophers certainly have an important role in neuroscience and psychology: which is help prevent neuroscientists and psychologists avoid errors in their own reasoning, and from getting entangled up in metaphysics, as when neuroscientists say that the visual field and the stimulus field are synonyms, or when they use two mutually incompatible theories of perception simultaneously (Smythies 1994a). But philosophers interested in perception and mind-brain relations also need, I suggest, to keep abreast of what is going on in clinical neurology, perceptual science and neuroscience.