Home

Time and the Observer

the Where and When of Consciousness in the Brain Endnote 1

Daniel Dennett and Marcel Kinsbourne

Part I Part II

ABSTRACT

Two models of consciousness are contrasted with regard to their treatment of subjective timing. The standard Cartesian Theater model postulates a place in the brain where "it all comes together": where the discriminations in all modalities are somehow put into registration and "presented" for subjective judgment. In particular, the Cartesian Theater model implies that the temporal properties of the content-bearing events occurring within this privileged representational medium determine subjective order. The alternative, Multiple Drafts model holds that whereas the brain events that discriminate various perceptual contents are distributed in both space and time in the brain, and whereas the temporal properties of these various events are determinate, none of these temporal properties determine subjective order, since there is no single, constitutive "stream of consciousness" but rather a parallel stream of conflicting and continuously revised contents. Four puzzling phenomena that resist explanation by the standard model are analyzed: two results claimed by Libet, an apparent motion phenomenon involving color change (Kolers and von Grunau), and the "cutaneous rabbit" (Geldard and Sherrick) an illusion of evenly spaced series of "hops" produced by two or more widely spaced series of taps delivered to the skin. The unexamined assumptions that have always made the Cartesian Theater model so attractive are exposed and dismantled. The Multiple Drafts model provides a better account of the puzzling phenomena, avoiding the scientific and metaphysical extravagances of the Cartesian Theater.

Part I

I'm really not sure if others fail to perceive me or if, one fraction of a second after my face interferes with their horizon, a millionth of a second after they have cast their gaze on me, they already begin to wash me from their memory: forgotten before arriving at the scant, sad archangel of a remembrance. --Ariel Dorfman, Mascara, 1988

When scientific advances contradict "common sense" intuitions, the familiar ideas often linger on, not just outliving their usefulness but even confusing the scientists whose discoveries ought to have overthrown them. We shall diagnose a ubiquitous error of thinking that arises from just such a misplaced allegiance to familiar images, and illustrate it with examples drawn from recent work in psychology and neuroscience. While this is a "theoretical" paper, it is addressed especially to those who think, mistakenly, that they have no theories and no need for theories. We shall show how uncontroversial facts about the spatial and temporal properties of information-bearing events in the brain require us to abandon a family of entrenched intuitions about "the stream of consciousness" and its relation to events occurring in the brain.

In Part 1, we introduce two models of consciousness, the standard Cartesian Theater and our alternative, the Multiple Drafts model, and briefly describe four phenomena of temporal interpretation that raise problems for the standard model. Two of these, drawn from the research of Libet, have been extensively debated on methodological grounds, but concealed in the controversy surrounding them are the mistaken assumptions we intend to expose. In part 2, we diagnose these intuitive but erroneous ideas, and exhibit their power to create confusion in relatively simple contexts. We demonstrate the superiority of the Multiple Drafts model, by showing how it avoids the insoluble problems faced by versions of the Cartesian Theater model. In part 3, we show how covert allegiance to the Cartesian Theater model has misled interpreters of Libet's phenomena, and show how the Multiple Drafts model avoids these confusions.

1. Two Models of Consciousness

1.1. Cartesian materialism: is there a "central observer" in the brain?

Wherever there is a conscious mind, there is a point of view. A conscious mind is an observer, who takes in the information that is available at a particular (roughly) continuous sequence of times and places in the universe. A mind is thus a locus of subjectivity, a thing it is like something to be (Farrell, 1950, Nagel, 1974). What it is like to be that thing is partly determined by what is available to be observed or experienced along the trajectory through space-time of that moving point of view, which for most practical purposes is just that: a point. For instance, the startling dissociation of the sound and appearance of distant fireworks is explained by the different transmission speeds of sound and light, arriving at the observer (at that point) at different times, even though they left the source simultaneously.

But if we ask where precisely in the brain that point of view is located, the simple assumptions that work so well on larger scales of space and time break down. It is now quite clear that there is no single point in the brain where all information funnels in, and this fact has some far from obvious consequences.

Light travels much faster than sound, as the fireworks example reminds us, but it takes longer for the brain to process visual stimuli than to process auditory stimuli. As Pöppel (1985, 1988) has pointed out, thanks to these counterbalancing differences, the "horizon of simultaneity" is about 10 meters: light and sound that leave the same point about 10 meters from the observer's sense organs produce neural responses that are "centrally available" at the same time. Can we make this figure more precise? There is a problem. The problem is not just measuring the distances from the external event to the sense organs, or the transmission speeds in the various media, or allowing for individual differences. The more fundamental problem is deciding what to count as the "finish line" in the brain. Pöppel obtained his result by comparing behavioral measures: mean reaction times (button-pushing) to auditory and visual stimuli. The difference ranges between 30 and 40 msec, the time it takes sound to travel approximately 10 meters (the time it takes light to travel 10 meters is infinitesimally different from zero). Pöppel used a peripheral finish line--external behavior--but our natural intuition is that the experience of the light and sound happens between the time the vibrations strike our sense organs and the time we manage to push the button to signal that experience. And it happens somewhere centrally, somewhere in the brain on the excited paths between the sense organ and muscles that move the finger. It seems that if we could say exactly where, we could infer exactly when the experience happened. And vice versa: if we could say exactly when it happened, we could infer where in the brain conscious experience was located.

This picture of how conscious experience must sit in the brain is a natural extrapolation of the familiar and undeniable fact that for macroscopic time intervals, we can indeed order events into the categories "not yet observed" and "already observed" by locating the observer and plotting the motions of the vehicles of information relative to that point. But when we aspire to extend this method to explain phenomena involving very short time intervals, we encounter a logical difficulty: If the "point" of view of the observer is spread over a rather large volume in the observer's brain, the observer's own subjective sense of sequence and simultaneity must be determined by something other than a unique "order of arrival" since order of arrival is incompletely defined until we specify the relevant destination. If A beats B to one finish line but B beats A to another, which result fixes subjective sequence in consciousness? (cf. Minsky, 1985, p.61) Which point or points of "central availability" would "count" as a determiner of experienced order, and why?

Consider the time course of normal visual information processing. Visual stimuli evoke trains of events in the cortex that gradually yield content of greater and greater specificity. At different times and different places, various "decisions" or "judgments" are made: more literally, parts of the brain are caused to go into states that differentially respond to different features, e.g., first mere onset of stimulus, then shape, later color (in a different pathway), motion, and eventually object recognition. It is tempting to suppose that there must be some place in the brain where "it all comes together" in a multi-modal representation or display that is definitive of the content of conscious experience in at least this sense: the temporal properties of the events that occur in that particular locus of representation determine the temporal properties--of sequence, simultaneity, and real-time onset, for instance--of the subjective "stream of consciousness." This is the error of thinking we intend to expose. "Where does it all come together?" The answer, we propose, is Nowhere. Some of the contentful states distributed around in the brain soon die out, leaving no traces. Others do leave traces, on subsequent verbal reports of experience and memory, on "semantic readiness" and other varieties of perceptual set, on emotional state, behavioral proclivities, and so forth. Some of these effects--for instance, influences on subsequent verbal reports--are at least symptomatic of consciousness. But there is no one place in the brain through which all these causal trains must pass in order to deposit their contents "in consciousness".

The brain must be able to "bind" or "correlate" and "compare" various separately discriminated contents, but the processes that accomplish these unifications are themselves distributed, not gathered at some central decision point, and as a result, the "point of view of the observer" is spatially smeared. If brains computed at near the speed of light, as computers do, this spatial smear would be negligible. But given the relatively slow transmission and computation speeds of neurons, the spatial distribution of processes creates significant temporal smear--ranging, as we shall see, up to several hundred milliseconds--within which range the normal common sense assumptions about timing and arrival at the observer need to be replaced. For many tasks, the human capacity to make conscious discriminations of temporal order drops to chance when the difference in onset is on the order of 50msec (depending on stimulus conditions), but, as we shall see, this variable threshold is the result of complex interactions, not a basic limit on the brain's capacity to make the specialized order judgments required in the interpretation and coordination of perceptual and motor phenomena. We need other principles to explain the ways in which subjective temporal order is composed, especially in cases in which the brain must cope with rapid sequences occurring at the limits of its powers of temporal resolution. As usual, the performance of the brain when put under strain provides valuable clues about its general modes of operation.

Descartes, early to think seriously about what must happen inside the body of the observer, elaborated an idea that is superficially so natural and appealing that it has permeated our thinking about consciousness ever since and permitted us to defer considering the perplexities--until now. Descartes decided that the brain did have a center: the pineal gland, which served as the gateway to the conscious mind. It is the only organ in the brain that is in the midline, rather than paired, with left and right versions. It looked different, and since its function was then quite inscrutable (and still is), Descartes posited a role for it: in order for a person to be conscious of something, traffic from the senses had to arrive at this station, where it thereupon caused a special--indeed magical--transaction to occur between the person's material brain and immaterial mind. When the conscious mind then decided on a course of bodily action, it sent a message back "down" to the body via the pineal gland. The pineal gland, then, is like a theater, within which is displayed information for perusal by the mind.

Descartes' vision of the pineal's role as the turnstile of consciousness (we might call it the Cartesian bottleneck) is hopelessly wrong. The problems that face Descartes' interactionistic dualism, with its systematically inexplicable traffic between the realm of the material and the postulated realm of the immaterial, were already well appreciated in Descartes' own day, and centuries of reconsideration have only hardened the verdict: the idea of the Ghost in the Machine, as Ryle (1949) aptly pilloried it, is a non-solution to the problems of mind. But while materialism of one sort or another is now a received opinion approaching unanimity Endnote 2, even the most sophisticated materialists today often forget that once Descartes' ghostly res cogitans is discarded, there is no longer a role for a centralized gateway, or indeed for any functional center to the brain. The brain itself is Headquarters, the place where the ultimate observer is, but it is a mistake to believe that the brain has any deeper headquarters, any inner sanctum arrival at which is the necessary or sufficient condition for conscious experience.

Let us call the idea of such a centered locus in the brain Cartesian materialism, since it is the view one arrives at when one discards Descartes' dualism but fails to discard the associated imagery of a central (but material) Theater where "it all comes together". Once made explicit, it is obvious that it is a bad idea, not only because, as a matter of empirical fact, nothing in the functional neuroanatomy of the brain suggests such a general meeting place, but also because positing such a center would apparently be the first step in an infinite regress of too-powerful homunculi. If all the tasks Descartes assigned to the immaterial mind have to be taken over by a "conscious" subsystem, its own activity will either be systematically mysterious, or decomposed into the activity of further subsystems that begin to duplicate the tasks of the "non-conscious" parts of the whole brain. Whether or not anyone explicitly endorses Cartesian materialism, some ubiquitous assumptions of current theorizing presuppose this dubious view. We will show that the persuasive imagery of the Cartesian Theater, in its materialistic form, keeps reasserting itself, in diverse guises, and for a variety of ostensibly compelling reasons. Thinking in its terms is not an innocuous shortcut; it is a bad habit. One of its most seductive implications is the assumption that a distinction can always be drawn between "not yet observed" and "already observed." But, as we have just argued, this distinction cannot be drawn once we descend to the scale that places us within the boundaries of the spatio-temporal volume in which the various discriminations are accomplished. Inside this expanded "point of view" spatial and temporal distinctions lose the meanings they have in broader contexts.

The crucial features of the Cartesian Theater model can best be seen by contrasting it with the alternative we propose, the Multiple Drafts model:

All perceptual operations, and indeed all operations of thought and action, are accomplished by multi-track processes of interpretation and elaboration that occur over hundreds of milliseconds, during which time various additions, incorporations, emendations, and overwritings of content can occur, in various orders. Feature-detections or discriminations only have to be made once. That is, once a localized, specialized "observation" has been made, the information content thus fixed does not have to be sent somewhere else to be rediscriminated by some "master" discriminator. In other words, it does not lead to a re-presentation of the already discriminated feature for the benefit of the audience in the Cartesian Theater. How a localized discrimination contributes to, and what affect it has on, the prevailing brain state (and thus awareness) can change from moment to moment, depending on what else is going on in the brain. Drafts of experience can be revised at a great rate, and no one is more correct than another. Each reflects the situation at the time it is generated (Kinsbourne, in preparation). These spatially and temporally distributed content-fixations are themselves precisely locatable in both space and time, but their onsets do not mark the onset of awareness of their content. It is always an open question whether any particular content thus discriminated will eventually appear as an element in conscious experience. These distributed content-discriminations yield, over the course of time, something rather like a narrative stream or sequence, subject to continual editing by many processes distributed around in the brain, and continuing indefinitely into the future (cf. Calvin's (1990) model of consciousness as "scenario-spinning".) This stream of contents is only rather like a narrative because of its multiplicity; at any point in time there are multiple "drafts" of narrative fragments at various stages of "editing" in various places in the brain. Probing this stream at different intervals produces different effects, elicits different narrative accounts from the subject. If one delays the probe too long (overnight, say) the result is apt to be no narrative left at all--or else a narrative that has been digested or "rationally reconstructed" to the point that it has minimal integrity. If one probes "too early", one may gather data on how early a particular discrimination is achieved in the stream, but at the cost of disrupting the normal progression of the stream. Most importantly, the Multiple Drafts model avoids the tempting mistake of supposing that there must be a single narrative (the "final" or "published" draft) that is canonical--that represents the actual stream of consciousness of the subject, whether or not the experimenter (or even the subject) can gain access to it.

The main points at which this model disagrees with the competing tacit model of the Cartesian Theater, may be summarized:

(1) Localized discriminations are not precursors of re-presentations of the discriminated content for consideration by a more central discriminator.

(2) The objective temporal properties of discriminatory states may be determined, but they do not determine temporal properties of subjective experience.

(3) The "stream of consciousness" is not a single, definitive narrative. It is a parallel stream of conflicting and continuously revised contents, no one narrative thread of which can be singled out as canonical--as the true version of conscious experience.

The different implications of these two models will be exhibited by considering several puzzling phenomena that seem at first to indicate that the mind "plays tricks with time." (Other implications of the Multiple Drafts model are examined at length in Dennett, forthcoming.)

1.2. Some "temporal anomalies" of consciousness

Under various conditions people report experiences in which the temporal ordering of the elements in their consciousness, or the temporal relation of those elements to concurrent activity in their brains, seems to be anomalous or even paradoxical. Some theorists (Eccles, 1977, Libet, 1982, 1985) have argued that these temporal anomalies are proof of the existence of an immaterial mind that interacts with the brain in physically inexplicable fashion. Others (Goodman, 1978, Libet, 1985b), while eschewing any commitment to dualism, have offered interpretations of the phenomena that seem to defy the accepted temporal sequence of cause and effect. Most recently, another theorist, (Penrose, 1989) has suggested that a materialistic explanation of these phenomena would require a revolution in fundamental physics. These radical views have been vigorously criticized, but the criticisms have overlooked the possibility that the appearance of anomaly in these cases is due to conceptual errors that are so deeply anchored in everyday thinking that even many of the critics have fallen into the same traps. We agree with Libet and others that these temporal anomalies are significant, but hold a different opinion about what they signify.

We will focus on four examples, summarized below. Two, drawn from the work of Libet, have received the most attention and provoked the most radical speculation, but because technical criticisms of his experiments and their interpretation raise doubts about the existence of the phenomena he claims to have discovered, we will begin with a discussion of two simpler phenomena, whose existence has not been questioned but whose interpretation raises the same fundamental problems. We will use these simpler cases to illustrate the superiority of the Multiple Drafts model to the traditional Cartesian Theater model, and then apply the conclusions drawn in the more complicated setting of the controversies surrounding Libet's work. Our argument will be that even if Libet's phenomena were not known to exist, theory can readily account for the possibility of phenomena of this pseudo-anomalous sort, and even predict them.

A. Color phi. (Kolers and von Grünau, 1976; See also Van der Waals and Roelofs, 1930, Kolers, 1972, and the discussion in Goodman, 1978) Many experiments have demonstrated the existence of apparent motion, or the phi phenomenon. If two or more small spots separated by as much as 4 degrees of visual angle are briefly lit in rapid succession, a single spot will seem to move. This is, of course, the basis of our experience of motion in motion pictures and television. First studied systematically by Wertheimer (1912; for a historical account, see Kolers, 1972, Sarris, 1989), phi has been subjected to many variations, and one of the most striking is reported in Kolers and von Grünau, 1976. The philosopher Nelson Goodman had asked Kolers whether the phi phenomenon would persist if the two illuminated spots were different in color, and if so, what would happen to the color of "the" spot as "it" moved? Would the illusion of motion disappear, to be replaced by two separately flashing spots? Would the illusory "moving" spot gradually change from one color to another, tracing a trajectory around the color wheel? The answer, when Kolers and von Grünau performed the experiments, was striking: the spot seems to begin moving and then to change color abruptly in the middle of its illusory passage toward the second location. Goodman wonders: "how are we able . . .to fill in the spot at the intervening place-times along a path running from the first to the second flash before that second flash occurs? "(1978, p.73) (The same question can of course be raised about any phi, but the color-switch in mid-passage vividly brings out the problem.) Unless there is precognition, the illusory content cannot be created until after some identification of the second spot occurs in the brain. But if this identification of the second spot is already "in conscious experience" would it not be too late to interpose the illusory color-switching-while-moving scene between the conscious experience of spot 1 and the conscious experience of spot 2? How does the brain accomplish this sleight-of-hand? Van der Waals and Roelofs (1931) proposed that the intervening motion is produced retrospectively, built only after the second flash occurs, and "projected backwards in time," (Goodman, 1978, p.74) a form of words reminiscent of Libet's "backwards referral in time." But what does it mean, that this experienced motion is "projected backwards in time"?

B. The cutaneous "rabbit". (Geldard and Sherrick, 1972, see also Geldard 1977, Geldard and Sherrick, 1983, 1986) The subject's arm rests cushioned on a table, and mechanical square-wave tappers are placed at two or three locations along the arm, up to a foot apart. A series of taps in rhythm are delivered by the tappers, e.g., 5 at the wrist followed by 2 near the elbow and then 3 more on the upper arm. The taps are delivered with interstimulus intervals between 50 and 200msec. So a train of taps might last less than a second, or as much as two or three seconds. The astonishing effect is that the taps seem to the subjects to travel in regular sequence over equidistant points up the arm--as if a little animal were hopping along the arm. Now how did the brain know that after the 5 taps on the wrist, there were going to be some taps near the elbow? The experienced "departure" of the taps from the wrist begins with the second tap, yet in catch trials in which the later elbow taps are never delivered, all five wrist taps are felt at the wrist in the expected manner. The brain obviously cannot "know" about a tap at the elbow until after it happens. Perhaps, one might speculate, the brain delays the conscious experience until after all the taps have been "received" and then, somewhere upstream of the seat of consciousness (whatever that is), revises the data to fit a theory of motion, and sends the edited version on to consciousness. But would the brain always delay response to one tap in case more came? If not, how does it "know" when to delay?

C. "Referral backwards in time".(Libet, 1965, 1981, 1982, 1985, Libet et al., 1979; see also Popper and Eccles, 1977, Dennett, 1979, Churchland, 1981, 1981b, Honderich, 1984.) Since Penfield and Jasper (1954) it has been known that direct electrical stimulation of locations on the somatosensory cortex can induce sensations on corresponding parts of the body. For instance, stimulation of a point on the left somatosensory cortex can produce the sensation of a brief tingle in the subject's right hand. Libet compared the time course of such cortically induced tingles to similar sensations produced in the more usual way, by applying a brief electrical pulse to the hand itself. He argued that while in each case it took considerable time (approximately 500 msec) to achieve "neuronal adequacy" (the stage at which cortical processes culminate to yield a conscious experience of a tingle), when the hand itself was stimulated, the experience was "automatically" "referred backwards in time."

Most strikingly, Libet reported instances in which a subject's left cortex was stimulated before his left hand was stimulated, which one would tend to think would give rise to two felt tingles: first right hand (cortically induced) and then left hand. In fact, however, the subjective report was reversed: "first left, then right." Even in cases of simultaneous stimulation, one might have thought, the left-hand tingle would be felt second, due to the additional distance (close to a meter) nerve impulses from the left hand must travel to the brain.

Libet interprets his results as raising a serious challenge to materialism: ". . . a dissociation between the timings of the corresponding 'mental' and 'physical' events would seem to raise serious though not insurmountable difficulties for the . . . theory of psychoneural identity." (1979, p.222.) According to Eccles, this challenge cannot be met:

This antedating procedure does not seem to be explicable by any neurophysiological process. Presumably it is a strategy that has been learnt by the self-conscious mind . . . the antedating sensory experience is attributable to the ability of the self-conscious mind to make slight temporal adjustments, i.e., to play tricks with time. (Popper and Eccles, 1977, p.364.)

D. Subjective delay of consciousness of intention. (Libet 1985, 1987, 1989; see also the accompanying commentaries) In other experiments, Libet asked subjects to make "spontaneous" decisions to flex one hand at the wrist while noting the position of a revolving spot (the "second hand" on a clock, in effect) at the precise time they formed the intention. Subjects' reports of these subjective simultaneities were then plotted against the timing of relevant electrophysiological events in their brains. Libet found evidence that these "conscious decisions" lagged between 350 and 400msec behind the onset of "readiness potentials" he was able to record from scalp electrodes, which, he claims, tap the neural events that determine the voluntary actions performed. He concludes that "cerebral initiation of a spontaneous voluntary act begins unconsciously" (1985, p.529). That one's consciousness might lag behind the brain processes that control one's body seems to some an unsettling and even depressing prospect, ruling out a real (as opposed to illusory) "executive role" for "the conscious self". (See the discussions by many commentators in BBS, 1985, 1987, 1989, and in Pagels, 1988, p.233ff, and Calvin, 1990, p.80-81. But see, for a view close to ours, Harnad, 1982)

In none of these cases would there be prima facie evidence of any anomaly were we to forgo the opportunity to record the subjects' verbal reports of their experiences and subject them to semantic analysis. No sounds appear to issue from heads before lips move, nor do hands move before the brain events that purportedly cause them, nor do events occur in the cortex in advance of the stimuli that are held to be their source. Viewed strictly as the internal and external behavior of a biologically-implemented control system for a body, the events observed and clocked in the experiments mentioned exhibit no apparent violations of everyday mechanical causation--of the sort to which Galilean/Newtonian physics provides the standard approximate model. Libet said it first: "It is important to realize that these subjective referrals and corrections are apparently taking place at the level of the mental 'sphere'; they are not apparent, as such, in the activities at neural levels." (1982, p.241)

Put more neutrally (pending clarification of what Libet means by the "mental 'sphere'"), only through the subjects' verbalizations about their subjective experiences do we gain access to a perspective from which the anomalies can appear. Endnote 3 Once their verbalizations (including communicative button-pushes, etc., (Dennett, 1982) are interpreted as a sequence of speech acts, their content yields a time series, the subjective sequence of the stream of consciousness. One can then attempt to put this series into registration with another time series, the objective sequence of observed events in the environment and in the nervous system. It is the apparent failures of registration, holding constant the assumption that causes precede their effects, that constitute the supposed anomalies (cf. Hoy, 1982).

One could, then, "make the problems disappear" by simply refusing to take introspective reports seriously. But while some hearty behaviorists may comfortably cling to the abstemious principle, "Eschew Content!" (Dennett, 1978), the rest of us prefer to accept the challenge to make sense of what Libet calls "a primary phenomenological aspect of our human existence in relation to brain function" (1985, p.534).

The reports by subjects about their different experiences . . . were not theoretical constructs but empirical observations. . . . The method of introspection may have its limitations, but it can be used appropriately within the framework of natural science, and it is absolutely essential if one is trying to get some experimental data on the mind-brain problem. (Libet, 1987, p.785)

In each example an apparent dislocation in time threatens the prima facie plausible thesis that our conscious perceptions are caused by events in our nervous systems, and our conscious acts, in turn, cause events in our nervous systems that control our bodily acts. To first appearances, the anomalous phenomena show that these two standard causal links cannot be sustained unless we abandon a foundational--some would say a logically necessary--principle: causes precede their effects. It seems that in one case (subjective delay of awareness of intention), our conscious intentions occur too late to be the causes of their bodily expressions or implementations, and in the other cases, percepts occur too early to have been caused by their stimuli, The vertiginous alternative, that something in the brain (or "conscious self") can "play tricks with time" by "projecting" mental events backwards in time, would require us to abandon the foundational principle that causes precede their effects.

There is a widespread conviction that no such revolutionary consequence follows from any of these phenomena, a conviction we share. But some of the influential arguments that have been offered in support of this conviction persist in a commitment to the erroneous presuppositions that made the phenomena appear anomalous in the first place. These presuppositions are all the more insidious because although in their overt, blatant forms they are roundly disowned by one and all, they creep unnoticed back into place, distorting analysis and blinding theory-builders to other explanations.

2. The Models in Action: Diagnosing the Tempting Errors

2.1. The representation of temporal properties versus the temporal properties of representations

The brain, as the control system responsible for solving a body's real-time problems of interaction with the environment, is under significant time pressure. It must often arrange to modulate its output in light of its input within a time window that leaves no slack for delays. In fact, many acts can only be ballistically initiated; there is no time for feedback to adjust the control signals. Other tasks, such as speech perception, would be beyond the physical limits of the brain's machinery if they did not utilize ingenious anticipatory strategies that feed on redundancies in the input (Libermann, 1970).

How, then, does the brain keep track of the temporal information it manifestly needs? Consider the following problem: since the toe-brain distance is much greater than the hip-brain distance, or the shoulder-brain distance or the forehead-brain distance, stimuli delivered simultaneously at these different sites will arrive at Headquarters in staggered succession, if travel-speed is constant along all paths. How (one might be tempted to ask) does the brain "ensure central simultaneity of representation for distally simultaneous stimuli"? This encourages one to hypothesize some "delay loop" mechanism that could store the early arrivers until they could be put "in synch" with the latecomers, but this is a mistake. The brain should not solve this problem, for an obvious engineering reason: it squanders precious time by committing the full range of operations to a "worst case" schedule. Why should important signals from the forehead (for instance) dawdle in the ante-room just because there might someday be an occasion when concurrent signals from the toes need to be compared to (or "bound to") them?

The brain sometimes uses "buffer memories" to cushion the interface between its internal processes and the asynchronous outside world (Sperling, 1960, Neisser, 1967, Newell, Rosenbloom and Laird, 1989), but there are also ways for the brain to utilize the temporal information it needs without the delays required for imposing a master synchrony. The basic design principle is well illustrated in an example in which a comparable problem is confronted and (largely) solved, though on a vastly different temporal and spatial scale.

Consider the communication difficulties faced by the far-flung British Empire before the advent of radio and telegraph, as illustrated by the Battle of New Orleans. On January 8, 1815, fifteen days after the truce was signed in Belgium, over a thousand British soldiers were killed in this needless battle. We can use this debacle to see how the system worked. Suppose on day 1 the treaty is signed in Belgium, with the news sent by land and sea to America, India, Africa. On day 15 the Battle is fought in New Orleans, and news of the defeat is sent by land and sea to England, India, etc. On day 20, too late, the news of the treaty (and the order to surrender) arrives in New Orleans. On day 35, let's suppose, the news of the defeat arrives in Calcutta, but the news of the treaty doesn't arrive there until day 40 (via a slow overland route). To the Commander in Chief in Calcutta, the battle would "seem" to have been fought before the treaty was signed--were it not for the practice of dating letters, which permits him to make the necessary correction.

These communicators solved their problems of communicating information about time by embedding representations of the relevant time information in the content of their signals, so that the arrival time of the signals themselves was strictly irrelevant to the information they carried. A date written at the head of a letter (or a dated postmark on the envelope) gives the recipient information about when it was sent, information that survives any delay in arrival. Endnote 4 This distinction between time represented (by the postmark) and time of representing (the day the letter arrives) is an instance of a familiar distinction between content and vehicle, and while the details of this particular solution is not available to the brain's communicators (because they don't "know the date" when they send their messages), the general principle of the content/vehicle distinction is relevant to information-processing models of the brain in ways that have not been well appreciated. Endnote 5

In general, we must distinguish features of representings from the features of representeds (Neumann, 1990b); someone can shout "softly, on tiptoe" at the top of his lungs, there are gigantic pictures of microscopic objects, and oil paintings of artists making charcoal sketches. The top sentence of a written description of a standing man need not describe his head, nor the bottom sentence his feet. To suppose otherwise is to confusedly superimpose two different spaces: the representing space and the represented space. The same applies to time. Consider the spoken phrase "a bright, brief flash of red light." The beginning of it is "a bright" and the end of it is "red light". Those portions of that speech event are not themselves representations of onsets or terminations of a brief red flash (Cf. Efron, 1967, p.714). No informing event in the nervous system can have zero duration (any more than it can have zero spatial extent), so it has an onset and termination separated by some amount of time. If it represents an event in experience, then the event it represents must itself have non-zero duration, an onset, a middle, and a termination. But there is no reason to suppose that the beginning of the representing represents the beginning of the represented. Endnote 6

Similarly, the representing by the brain of "A before B" does not have to be accomplished by first:

a representing of A,

followed by:

a representing of B.

" B after A" is an example of a (spoken) vehicle that represents A as being before B, and the brain can avail itself of the same freedom of temporal placement. What matters for the brain is not necessarily when individual representing events happen in various parts of the brain (as long as they happen in time to control the things that need controlling!) but their temporal content. That is, what matters is that the brain can proceed to control events "under the assumption that A happened before B" whether or not the information that A has happened enters the relevant system of the brain and gets recognized as such before or after the information that B has happened. (Recall the Commander in Chief in Calcutta: first he is informed of the battle, and then he is informed of the truce, but since he can extract from this the information that the truce came first, he can act accordingly.) Systems in various locations in the brain can, in principle, avail themselves of similar information-processing, and that is why fixing the exact time of onset of some representing element in some place in the brain does not provide a temporal landmark relative to which other elements in the subjective sequence can--or must--be placed.

How are temporal properties really inferred by the brain? Systems of "date stamps" or "postmarks" are not theoretically impossible (Glynn, 1990), but there is a cheaper, less foolproof but biologically more plausible way: by what we might call content-sensitive settling. A useful analogy would be the film studio where the sound track is "synchronized" with the film. The various segments of audio tape may by themselves have lost all their temporal markers, so that there is no simple, mechanical way of putting them into apt registration with the images. But sliding them back and forth relative to the film and looking for convergences, will usually swiftly home in on a "best fit." The slap of the slateboard at the beginning of each take provides a double saliency, an auditory and a visual clap, to slide into synchrony, pulling the rest of the tape and the frames into position at the same time. But there are typically so many points of mutually salient correspondence that this conventional saliency at the beginning of each take is just a handy redundancy. Getting the registration right depends on the content of the film and the tape, but not on sophisticated analysis of the content. An editor who knew no Japanese would find synchronizing a Japanese soundtrack to a Japanese film difficult and tedious but not impossible. Moreover, the temporal order of the stages of the process of putting the pieces into registration is independent of the content of the product; the editor can organize scene three before organizing scene two, and in principle could even do the entire job running the segments "in reverse."

Quite "stupid" processes can do similar jiggling and settling in the brain. The computation of depth in random-dot stereograms (Julesz, 1971) is a spatial problem for which we can readily envisage temporal analogues. If the system receives stereo pairs of images, the globally optimal registration can be found without first having to subject each data array to an elaborate process of feature extraction. There are enough lowest-level coincidences of saliency--the individual dots in a random dot stereogram--to dictate a solution. In principle, then, the brain can solve some of its problems of temporal inference by such a process, drawing data not from left and right eyes, but from whatever information-sources are involved in a process requiring temporal judgments. (See Gallistel, 1990, esp. pp. 539-49, for a discussion of the requirements for "spatiotemporal specification".)

Two important points follow from this. First, such temporal inferences can be drawn (such temporal discriminations can be made) by comparing the (low-level) content of several data arrays, and this real time process need not occur in the temporal order that its product eventually represents. Second, once such a temporal inference has been drawn, which may be before high-level features have been extracted by other processes, it does not have to be drawn again! There does not have to be a later representation in which the high-level features are "presented" in a real time sequence for the benefit of a second sequence-judger. In other words, having drawn inferences from these juxtapositions of temporal information, the brain can go on to represent the results in any format that fits its needs and resources--not necessarily a format in which "time is used to represent time".

There remains a nagging suspicion that whereas the brain may take advantage of this representational freedom for other properties, it cannot do so for the property of temporal sequence. Mellor explicitly enunciates this assumption, deeming it too obvious to need support:

Suppose for example I see one event e precede another, e*. I must first see e and then e*, my seeing of e being somehow recollected in my seeing of e*. That is, my seeing of e affects my seeing of e*: this is what makes me--rightly or wrongly--see e precede e* rather than the other way round. But seeing e precede e* means seeing e first. So the causal order of my perceptions of these events, by fixing the temporal order I perceive them to have, fixes the temporal order of the perceptions themselves. . . . the striking fact . . . should be noticed, namely that perceptions of temporal order need temporally ordered perceptions. No other property or relation has to be thus embodied in perceptions of it [our italics]: perceptions of shape and colour, for example, need not themselves be correspondingly shaped or coloured. (Mellor, 1981, p.8)

We believe this is false, but there is something right about it. Since the fundamental function of representation in the brain is to control behavior in real time, the timing of representings is to some degree essential to their task, in two ways. First, the timing may, at the outset of a perceptual process, be what determines the content. Consider how to distinguish a spot moving from right to left from a spot moving from left to right on a motion picture screen. The only difference between the two may be the temporal order in which two frames (or more) are projected. If the brain determines "first A, then B" the spot is seen as moving in one direction; if the brain determines "first B, then A" the spot is seen as moving in the opposite direction. This discrimination is, then, as a matter of logic, based on the brain's capacity to make a temporal order judgment of a particular level of resolution. Motion picture frames are usually exposed at the rate of 24 per second, and so the visual system can resolve order between stimuli that occur within about 50msec. This means that the actual temporal properties of signals--their onset times, their velocity in the system, and hence their arrival times--must be accurately controlled until such a discrimination is made. But once it is made, locally, by some circuit in the visual system (even as peripherally as the ganglion cells of the rabbit's retina!--Barlow and Levick, 1965), the content "from left to right" can then be sent, in a temporally sloppy way, anywhere in the brain where this directional information might be put to use. In this way one can explain the otherwise puzzling fact that at interstimulus intervals at which people are unable to perform above chance on temporal order judgments, they perform flawlessly on other judgments which logically call for the same temporal acuity. Thus Efron (1973) showed that subjects could easily distinguish sounds, flashes and vibrations that differed only in the order in which two component stimuli occurred at a fraction of the interstimulus interval at which they can explicitly specify their order.

A second constraint on timing has already been noted parenthetically above: it does not matter in what order representations occur so long as they occur in time to contribute to the control of the appropriate behavior. The function of a representing may depend on meeting a deadline, which is a temporal property of the vehicle doing the representing. This is particularly evident in such time-pressured environments as the imagined Strategic Defense Initiative. The problem is not how to make computer systems represent, accurately, missile launches, but how to represent a missile launch accurately during the brief time while one can still do something about it. A message that a missile was launched at 6:04:23.678 am EST may accurately represent the time of launch forever, but its utility may utterly lapse at 6:05am EST. For any task of control, then, there is a temporal control window within which the temporal parameters of representings may in principle be moved around ad lib.

The deadlines that limit such windows are not fixed, but rather depend on the task. If, rather than intercepting missiles, you are writing your memoirs or answering questions at the Watergate hearings (Neisser, 1981), you can recover the information you need about the sequence of events in your life in order to control your actions in almost any order, and you can take your time drawing inferences.

These two factors explain what is plausible in Mellor's claim, without supporting the invited conclusion that all perceptions of temporal order must be accomplished in a single place by a process that observes seriatim a succession of "perceptions" or other representations. Once the perceptual processes within an observer have begun to do their work, providing the necessary discriminations, there is no point in undoing their work in order to provide a job for a yet more interior observer.

Causes must precede effects. This fundamental principle ensures that temporal control windows are bounded at both ends: by the earliest time at which information could arrive in the system, and by the latest time at which information could contribute causally to control of a particular behavior. Moreover, the principle applies to the multiple distributed processes that achieve such control. Any particular process that requires information from some source must indeed wait for that information; it can't get there till it gets there. This is what rules out "magical" or precognitive explanations of the color-switching phi phenomenon, for example. The content green spot cannot be attributed to any event, conscious or unconscious, until the light from the green spot has reached the eye and triggered the normal neural activity in the visual system up to the level at which the discrimination of green is accomplished. Moreover, all content reported or otherwise expressed in subsequent behavior must have been "present" (in the relevant place in the brain, but not necessarily in consciousness) in time to have contributed causally to that behavior. For instance, if a subject in an experiment says "dog" in response to a visual stimulus, we can work backwards from the behavior, which was clearly controlled by a process that had the content dog (unless the subject says "dog" to every stimulus, or spends the day saying "dog dog dog . . ." etc.) And since it takes on the order of 100msec to execute a speech intention of this sort, we can be quite sure that the content dog was present in (roughly) the language areas of the brain by 100msec before the utterance. Working from the other end, we can determine the earliest time the content dog could have been computed or extracted by the visual system from the retinal input, and even, perhaps, follow its creation and subsequent trajectory through the visual system and into the language areas.

What would be truly anomalous (indeed a cause for lamentations and the gnashing of teeth) would be if the time that elapsed between the dog-stimulus and the "dog"-utterance were less than the time physically required for this content to be established and moved through the system. But no such anomalies have been uncovered. It is only when we try to put the sequence of events thus detectable in the objective processing stream into registration with the subject's subjective sequence as indicated by what the subject subsequently says that we have any sign of anomaly at all.

2.2. Orwellian and Stalinesque Revisions: the Illusion of a Distinction

Now let us see how the two different models, the Cartesian Theater and Multiple Drafts, deal with the presumed anomalies, starting with the simpler and less controversial phenomena. The Cartesian Theater model postulates a place within the brain where what happens "counts"; that is, it postulates that the features of events occurring within this functionally definable boundary (whatever it is) are definitive or constitutive features of conscious experience. (The model applies to all features of subjective experience, but we are concentrating on temporal features.) This implies that all revisions of content accomplished by the brain can be located relative to this place, a deeply intuitive--but false--implication that can be illustrated with a thought experiment.

Suppose we tamper with your brain, inserting in your memory a bogus woman wearing a hat where none was (e.g., at the party on Sunday). If on Monday, when you recall the party, you remember her, and can find no internal resources for so much as doubting the veracity of your memory, we could all agree that you never did experience her; that is, not at the party on Sunday.

(figure 1 about here)

Of course your subsequent experience of (bogus) recollection can be as vivid as may be, and on Tuesday we can certainly agree that you have had vivid conscious experiences of there being a a woman in a hat at the party, but the first such experience, we would insist, was on Monday, not Sunday (though it doesn't seem this way to you).

We lack the power to insert bogus memories by neurosurgery, but sometimes our memories play tricks on us, so what we cannot yet achieve surgically happens in the brain on its own. Sometimes we seem to remember, even vividly, experiences that never occurred. We might call such post-experiential contaminations or revisions of memory Orwellian, recalling George Orwell's chilling vision of the Ministry of Truth in 1984, which busily rewrote history and thus denied access to the (real) past to all who followed.

Orwellian revision is one way to fool posterity. Another is to stage show trials, carefully scripted presentations of false testimony and bogus confessions, complete with simulated evidence. We might call this ploy Stalinesque. Notice that if we are usually sure which mode of falsification has been attempted on us, the Orwellian or the Stalinesque, this is just a happy accident. In any successful disinformation campaign, were we to wonder whether the accounts in the newspapers were Orwellian accounts of trials that never happened at all, or true accounts of phony show trials that actually did happen, we might be unable to tell the difference. If all the traces--newspapers, videotapes, personal memoirs, inscriptions on gravestones, living witnesses, etc.--have been either obliterated or revised, we will have no way of knowing which sort of fabrication happened: a fabrication first, culminating in a staged trial whose accurate history we now have before us, or rather, after a summary execution, history-fabrication covering up the deed: no trial of any sort actually took place.

The distinction between reality and (subsequent) appearance, and the distinction between Orwellian and Stalinesque methods of producing misleading archives, work unproblematically in the everyday world, at macroscopic time scales. One might well think these distinctions apply unproblematically all the way in. That is the habit of thought that produces the cognitive illusion of Cartesian materialism. We can catch it in the act in a thought experiment that differs from the first one in nothing but time scale.

(figure 2 about here)

Suppose a long-haired woman jogs by. About one second after this, a subterranean memory of some earlier woman--a short-haired woman with glasses--contaminates the memory of what you have just seen: when asked a minute later for details of the woman you just saw, you report, sincerely but erroneously, that she was wearing glasses. Just as in the previous case, we are inclined to say that your original visual experience, as opposed to the memory of it seconds later, was not of a woman with glasses. But due to the subsequent memory-contaminations, it seems to you exactly as if at the first moment you saw her, you were struck by her eyeglasses. An Orwellian, post-experiential revision has happened: there was a fleeting instant, before the memory contamination took place, when it didn't seem to you she had glasses. For that brief moment, the reality of your conscious experience was a long-haired woman without eyeglasses, but this historical fact has become inert; it has left no trace, thanks to the contamination of memory that came one second after you glimpsed her.

This understanding of what happened is jeopardized, however, by an alternative account. Your subterranean earlier memories of that short-haired woman with the glasses could just as easily have contaminated your experience on the upward path, in the processing of information that occurs "prior to consciousness" so that you actually hallucinated the eyeglasses from the very beginning of your experience.

(figure 3 about here)

In that case, your obsessive memory of the woman with glasses would be playing a Stalinesque trick on you, creating a "show trial" for you to experience, which you then accurately recall at later times, thanks to the record in your memory. To naive intuition these two cases are as different as can be: told the first way (figure 2) you suffer no hallucination at the time the woman jogs by, but suffer subsequent memory-hallucinations: you have false memories of your actual ("real") experience. Told the second way (figure 3) you hallucinate when she runs by, and then accurately remember that hallucination (which "really did happen in consciousness") thereafter. Surely these are distinct possibilities, no matter how finely we divide up time?

No. Here the distinction between perceptual revisions and memory revisions that works so crisply at other scales is not guaranteed application. We have moved into the foggy area in which the subject's point of view is spatially and temporally smeared, and the question Orwellian or Stalinesque? (post-experiential or pre-experiential) need have no answer. The boundary between perception and memory, like most boundaries between categories, is not perfectly sharp, as has often been noted.

There is a time window that began when the long-haired woman jogged by, exciting your retinas, and ended when you expressed--to yourself or someone else--your eventual conviction that she was wearing glasses. At some time during this interval, the content wearing glasses was spuriously added to the content long-haired woman. We may assume (and might eventually confirm in detail) that there was a brief time when the content long-haired woman had already been discriminated in the brain but before the content wearing glasses had been erroneously "bound" to it. Indeed, it would be plausible to suppose that this discrimination of a long-haired woman was what triggered the memory of the earlier woman with the glasses. What we would not know, however, is whether this spurious binding was before or after the fact--the presumed fact of "actual conscious experience". Were you first conscious of a long-haired woman without glasses and then conscious of a long-haired woman with glasses, a subsequent consciousness which wiped out the memory of the earlier experience, or was the very first instant of conscious experience already spuriously tinged with eyeglasses? If Cartesian materialism were true, this question would have to have an answer, even if we--and you--could not determine it retrospectively by any test. For the content that "crossed the finish first" was either long-haired woman or long-haired woman with glasses. But what happens to this question if Cartesian materialism is false (as just about everyone agrees)? Can the distinction between pre-experiential and post-experiential content revisions be maintained?

An examination of the color phi phenomenon will show that it cannot. On the first trial (i.e., without conditioning), subjects report seeing the color of the moving spot switch in mid-trajectory from red to green--a report sharpened by Kolers' ingenious use of a pointer device which subjects retrospectively-but-as-soon-as-possible "superimposed" on the trajectory of the illusory moving spot: such pointer locations had the content: "The spot changed color right about here."(Kolers and von Grünau, 1976, p.330.) Recall Goodman's (1978, p. 73) expression of the puzzle: "how are we able . . .to fill in the spot at the intervening place-times along a path running from the first to the second flash before that second flash occurs?"

Consider, first, a Stalinesque mechanism: in the brain's editing room, located before consciousness, there is a delay, a loop of slack like the "tape delay" used in broadcasts of "live" programs which gives the censors in the control room a few seconds to bleep out obscenities before broadcasting the signal. In the editing room, first frame A, of the red spot, arrives, and then, when frame B, of the green spot, arrives, some interstitial frames (C and D) can be created and then spliced into the film (in the order A,C,D,B) on its way to projection in the theater of consciousness. By the time the "finished product" arrives at consciousness, it already has its illusory insertion.

(figure 4 about here)

Alternatively, there is the hypothesis of an Orwellian mechanism: shortly after the awareness of the first spot and the second spot (with no illusion of apparent motion at all), a revisionist historian of sorts, in the brain's memory-library receiving station, notices that the unvarnished history of this incident doesn't make enough sense, so he "interprets" the brute events, red-followed-by-green, by making up a narrative about the intervening passage, complete with midcourse color change, and installs this history, incorporating his glosses, frames C and D (in figure 4), in the memory library for all future reference. Since he works fast, within a fraction of a second--the amount of time it takes to frame (but not utter) a verbal report of what you have experienced--the record you rely on, stored in the library of memory, is already contaminated. You say and believe that you saw the illusory motion and color change, but that is really a memory hallucination, not an accurate recollection of your original awareness.

How could we see which of these hypotheses is correct? It might seem that we could rule out the Stalinesque hypothesis quite simply, because of the delay in consciousness it postulates. In Kolers' and von Grünau's experiment, there was a 200msec difference in onset between the red and green spot, and since, ex hypothesi, the whole experience cannot be composed by the editing room until after the content green spot has reached the editing room, consciousness of the initial red spot will have to be delayed by at least that much. (If the editing room sent the content red spot up to the theater of consciousness immediately, before receiving frame B and then fabricating frames C and D, the subject would presumably experience a gap in the film, a noticeable delay of around 200msec between A and C).

Suppose we ask subjects to press a button "as soon as you experience a red spot." We would find little or no difference in response time to a red spot alone versus a red spot followed 200msec later by a green spot (in which case the subjects report color-switching apparent motion). This could be because there is always a delay of at least 200msec in consciousness, but aside from the biological implausibility of such a squandering of time, there is the evidence from many quarters that responses under conscious control, while slower than such responses as reflex blinks, occur with close to the minimum latencies that are physically possible; after subtracting the demonstrable travel times for incoming and outgoing pulse trains, and the response preparation time, there is little time left over in "central processing" in which to hide a 200msec delay. So the responses had to have been initiated before the discrimination of the second stimulus, the green spot. This would seem overwhelmingly to favor the Orwellian, post-experiential mechanism: as soon as the subject becomes conscious of the red spot, he initiates a button-press. While that button press is forming, he becomes conscious of the green spot. Then both these experiences are wiped from memory, replaced in memory by the revisionist record of the red spot moving over and then turning green halfway across. He readily and sincerely (but mistakenly) reports having seen the red spot moving towards the green spot before changing color.

If the subject were to insist that he really was conscious from the very beginning of the red spot moving and changing color, the Orwellian theorist would firmly explain to him that he is wrong; his memory is playing tricks on him; the fact that he pressed the button when he did is conclusive evidence that he was conscious of the (stationary) red spot before the green spot had even occurred. After all, his instructions were to press the button when he was conscious of a red spot. He must have been conscious of the red spot about 200msec before he could have been conscious of it moving and turning green. If that is not how it seems to him, he is simply mistaken.

The defender of the Stalinesque (pre-experiential) alternative is not defeated by this, however. Actually, he insists, the subject responded to the red spot before he was conscious of it! The directions to the subject (to respond to a red spot) had somehow trickled down from consciousness into the editing room, which unconsciously initiated the button-push before sending the edited version (frames ACDB) up to consciousness for "viewing". The subject's memory has played no tricks on him; he is reporting exactly what he was conscious of, unless he insists that he pushed the button after consciously seeing the red spot; his "premature" button-push was unconsciously (or preconsciously) triggered (cf,. Velmans, 1991).

Where the Stalinesque theory postulates a button-pushing reaction to an unconscious detection of a red spot, the Orwellian theory postulates a conscious experience of a red spot that is immediately obliterated from memory by its sequel. So here is the rub: we have two different models of what happens in the phi phenomenon: one posits a Stalinesque "filling in" on the upward, pre-experiential path, and the other posits an Orwellian "memory revision" on the downward, post-experiential path, and both of them are consistent with whatever the subject says or thinks or remembers. Note that the inability to distinguish these two possibilities does not just apply to the outside observers who might be supposed to lack some private data to which the subject had "privileged access". You, as a subject in a phi phenomenon experiment, could not discover anything in the experience from your own first-person perspective that would favor one theory over the other; the experience would "feel the same" on either account. (As the interstimulus interval is lengthened, of course, subjects pass from seeing apparent motion to seeing individual stationary flashes. There is an intermediate range of intervals where the phenomenology is somewhat "paradoxical": you see the spots as two stationary flashers and as one thing moving. This sort of apparent motion is readily distinguishable from the swifter, smoother sort of apparent motion of cinema, for instance, but your capacity to make this discrimination is not relevant to the dispute between the Orwellian and the Stalinesque theorist. They agree that you can make this discrimination under the right conditions; what they disagree about is how to describe the cases of apparent motion that you can't tell from real motion--the cases in which you really (mis-)perceive the illusory motion. To put it loosely, in these cases is your memory playing tricks with you, or are just your eyes playing tricks with you? You can't tell "from the inside".)

We can see the same indistinguishability even more clearly when we see how the two different models handle the well-studied phenomenon of metacontrast (for a review, see Breitmeyer, 1984). If a stimulus is flashed briefly on a screen and then followed, after a brief inter-stimulus-interval, by a second "masking" stimulus, subjects report seeing only the second stimulus. (And if you put yourself in the subject's place you will see for yourself; you will be prepared to swear that there was only one flash.) The standard description of such phenomena is that the second stimulus somehow prevents conscious experience of the first stimulus (in other words, it somehow waylays the first stimulus on its way to consciousness). But people can nevertheless do much better than chance if required to guess whether there were two stimuli. This only shows once again that stimuli can have their effects on us without our being conscious of them. This standard line is, in effect, the Stalinesque model of metacontrast: the first stimulus never gets to play on the stage of consciousness; it has whatever effects it has entirely unconsciously. But we have just uncovered a second, Orwellian model of metacontrast: subjects are indeed conscious of the first stimulus (which would "explain" their capacity to guess correctly) but their memory of this conscious experience is almost entirely obliterated by the second stimulus (which is why they deny having seen it, in spite of their tell-tale better-than-chance guesses). Endnote 7

Both the Orwellian and the Stalinesque version of the Cartesian Theater model can deftly account for all the data--not just the data we already have, but the data we can imagine getting in the future. They both account for the verbal reports: one theory says they are innocently mistaken while the other says they are accurate reports of experienced "mistakes". (A similar verdict is suggested in the commentaries of Holender, 1986; see especially Dixon, 1986, Erdelyi, 1986, Marcel, 1986, Merikle and Cheesman, 1986.) They agree about just where in the brain the mistaken content enters the causal pathways; they just disagree about whether that location is pre-experiential or post-experiential. They both account for the non-verbal effects: one says they are the result of unconsciously discriminated contents while the other says they are the result of consciously discriminated but forgotten contents. They agree about just where and how in the brain these discriminations occur; they just disagree about whether to interpret those processes as happening inside or outside the charmed circle of consciousness. Finally, they both account for the subjective data--whatever is obtainable "from the first-person-perspective"--because they agree about how it ought to "feel" to subjects: subjects should be unable to tell the difference between misbegotten experiences and immediately misremembered experiences. So, in spite of first appearances, there is really only a verbal difference between the two theories (cf. Reingold and Merikle, 1990). They tell exactly the same story except for where they place a mythical Great Divide, a point in time (and hence a place in space) whose fine-grained location is nothing that subjects can help them locate, and whose location is also neutral with regard to all other features of their theories. This is a difference that makes no difference.

Consider a contemporary analogy. With the advent of word-processing and desktop publishing and electronic mail, we are losing the previously quite hard-edged distinction between pre-publication editing, and post-publication correction of "errata". With multiple drafts in electronic circulation, and with the author readily making revisions in response to comments received by electronic mail, calling one of the drafts the canonical text--the text of "record", the one to cite in one's own publications--becomes a somewhat arbitrary matter. Often most of the intended readers, the readers whose reading of the text matters, read only an early draft; the "published" version is archival and inert. If it is important effects we are looking for, then, most if not all the important effects of writing a text are now spread out over many drafts, not postponed until after publication. It used to be otherwise; virtually all of a text's important effects happened after appearance in a book or journal and because of its making such an appearance. All the facts are in, and now that the various candidates for the "gate" of publication can be seen no longer to be functionally important, if we feel we need the distinction at all, we will have to decide arbitrarily what is to count as publishing a text. There is no natural summit or turning point in the path from draft to archive.

Similarly--and this is the fundamental implication of the Multiple Drafts model--if one wants to settle on some moment of processing in the brain as the moment of consciousness, this has to be arbitrary. One can always "draw a line" in the stream of processing in the brain, but there are no functional differences that could motivate declaring all prior stages and revisions unconscious or preconscious adjustments, and all subsequent emendations to the content (as revealed by recollection) to be post-experiential memory-contamination. The distinction lapses at close quarters.

Another implication of the Multiple Drafts model, in contrast to the Cartesian Theater, is that there is no need--or room--for the sort of "filling in" suggested by frames C and D of figure 4. Discussing Kolers' experiment, Goodman notes that it

"seems to leave us a choice between a retrospective construction theory and a belief in clairvoyance" (1978, p.83) What then is "retrospective construction"?

Whether perception of the first flash is thought to be delayed or preserved or remembered [our italics], I call this the retrospective construction theory--the theory that the construction perceived as occurring between the two flashes is accomplished not earlier than the second.

It seems at first that Goodman does not choose between a Stalinesque theory (perception of the first flash is delayed) and an Orwellian theory (the perception of the first flash is preserved or remembered), but his Orwellian revisionist does not merely adjust judgments; he constructs material to fill in the gaps:

each of the intervening places along a path between the two flashes is filled in . . . with one of the flashed colors rather than with successive intermediate colors. (p.85)

What Goodman overlooks is the possibility that the brain doesn't actually have to go to the trouble of "filling in" anything with "construction", for no one is looking. As the Multiple Drafts model makes explicit, once a discrimination has been made once, it does not have to be made again; the brain just adjusts to the conclusion that is drawn, making the new interpretation of the information available for the modulation of subsequent behavior. Recall the Commander in Chief in Calcutta; he just had to judge that the truce came before the battle; he didn't also have to mount some sort of pageant of "historical reconstruction" to watch, in which he receives the letters in the "proper" order.

Similarly, when Goodman (1978) proposes that "the intervening motion is produced retrospectively, built only after the second flash occurs and projected backwards in time," this suggests ominously that a final film is made and then run through a magical projector whose beam somehow travels backwards in time onto the mind's screen. Whether or not this is just what Van der Waals and Roelofs (1930) had in mind when they proposed "retrospective construction," it is presumably what led Kolers (1972, p.184) to reject their hypothesis, insisting that all construction is carried out in "real time." Why, though, should the brain bother to "produce" the "intervening motion"? Why not just conclude that there was intervening motion, and encode that "retrospective" content into the processing stream? This would suffice for it to seem to the subject that intervening motion had been experienced.

Our Multiple Drafts model agrees with Goodman that retrospectively the brain creates the content (the judgment) that there was intervening motion, and this content is then available to govern activity and leave its mark on memory. But our model claims that the brain does not bother "constructing" any representations that go to the trouble of "filling in" the blanks. That would be a waste of time and (shall we say?) paint. The judgment is already in, so the brain can get on with other tasks! Endnote 8

Goodman's "projection backwards in time," like Libet's "backwards referral in time," is an equivocal phrase. It might mean something modest and defensible: a reference to some past time is included in the content. On this reading it could be a claim like "This novel takes us back to ancient Rome . . ," which almost no one would interpret in a metaphysically extravagant way, as claiming that the novel was some sort of time travel machine. This is the reading that is consistent with Goodman's other views, but Kolers apparently took it to mean something metaphysically radical: that there was some actual projection of one thing at one time to another time. As we shall see, the same equivocation bedevils Libet's interpretation of his phenomena.