The sentence "The earth is warming due to human activity." Represents a
certain state of affairs – the warming of the earth due to human activity.
The vehicle carrying this content is the token sentence – the pattern of light
and dark shapes on your screen, or of ink if this page is printed.
Vehicles of representation are straightforward material objects;
patterns of ink or chalk or light, arrangements of physical matter, and so on.
How, exactly, does the sentence quoted above carry information about the
world? How, exactly, does the pattern of colour carry information about the
tree?.
Once we’ve distinguished the vehicle and the content of the
representation, it is clear that understanding mental representation requires
understanding representational vehicles, the things in the world referred to
by those vehicles (contents), and the relation between the two. The latter
two problems under the name of ‘mental content’ or ‘meaning’ (see
elsewhere in this Field Guide). Understanding a relation requires
understanding its relata, and so the nature of the vehicles deserves close
attention. (Many philosophers recognise that there is more to content than
simply reference to physical objects and states of affairs, for example
Frege’s Sinn (‘sense’), or the proposition expressed by a
representation. But focussing on referential content will illustrate the
problems well enough.)
Analog vs digital
The evident power of digital computers led early cognitive scientists to
the view that it was the digital nature of the computer’s internal
representations that was significant in their representational and
computational prowess.
Computational digital representation were contrasted with analog
representations. For example, letters of the alphabet are digital
representations. There are, in English, exactly 26 different types of letter,
and every token letter (on this page, for example) is definitely one of the 26.
Digital representations are discrete entities: in a given scheme of digital
representation there are finitely specifiable distinct types of representation,
and for every representational token, it is clear which, if any, type it
instantiates. Numerals provide another good example of digital
representation. Quantities are represent by the arrangement of a discrete
number of possible digits. In the decimal system, there are 10 such digits
(‘0’ … ‘9’). In the binary system, there are two (‘0’ and ‘1’).
In contrast, analog representations do not admit of definite
type-identity. A photograph is a good example of an analog representation:
it represents via the arrangement of colour patches, and the colour values
are drawn from a continuous spread of possible colours. Similarly the place
of a given colour patch is measured on a continuous rather than a discrete
scale.
The power of digital computers to simulate analog systems (e.g.
computer weather simulations) raises questions about the importance of the
difference between analog and digital representation.
The analog/digital distinction refers to more than just representational
vehicles, however. Digital computers use digital processes to manipulate
digital representations. The difference in character between digital and
analog systems might be best attributed to the form of processing, rather
than the vehicles of representation.
Digital representations, as the name suggests, evoke the image of
numerical systems. A representational scheme in which a unique digit may
be associated with each representational vehicle is a paradigm case of a
digital system of representation.
The digital/analog distinction looks to be a straightforward and useful
way to classify representations. Analog watches represent the passage of
time by the smooth flow of the hour, minute and second hands across the
dial; digital watches represent the passage of time by changing the
numerical readout in synchrony with the flow of time, in one second jumps.
There are important differences between these two forms of
representation. The flow of time has a clear visual analog on the watch: 2
hours is a sweep of 60º by the hour hand, and so on. The flow of time on a
digital watch does not have the same visual analog. To read a digital watch
you need to understand decimal numerals, whereas to understand the
analog watch you need not have a grasp of numerals at all. Innumerate
readers of an analog watch may still correlate sunrise and sunset with
certain positions, and recognise the relationship between the movement of
the watch and the flow of time.
In short, reading a digital watch requires grasping an abstract
representational scheme which makes use of arbitrary vehicles (in this case
numerals).
There are other important features illustrated by this example. The
analog watch uses a representational medium which is smooth and
continuous – there are no step-functions involved in determining the
appropriate vehicle for a given time. Put another way, there are an indefinite
number of states of the analog watch which require the real numbers for an
accurate assessment. The digital watch has a finite number of possible
states (all the possible times accurate to the second), and there is no way
to represent times in between the possible represented times. For example,
if the digital watch is has a second digit, but no 10ths of a second digit, then
it can represent 3:25:43 and 3:25:44, but it cannot represent 3:25:43.5.
The analog watch makes use of a representational vehicle which
reliably covaries with what it represents – time. Thus as time progresses,
the watch hands sweep around the dial a distance directly proportional to
the elapsed time. Digital watches are quite different. The digits change with
each second, however there is no other simple covariation between vehicle
and content. More generally the structure of the analog watch – the internal
relations between the hands and the dial – exactly matches the relations
which hold between hours, minutes and seconds. There is no such similarity
of structure between the numerals on a digital watch and the structure of
time.
The analog/digital distinction thus seems to mark an important
representational difference. In an automated system, the task of reading an
analog watch places very different demands than does the task of reading
a digital watch.
But there are any number of cases which cause difficulty for this
distinction. What of watches where the second hand ticks, in discrete steps,
from second to second? This case retains a certain amount of the structural
covariation features of the paradigm analog watch, but introduced
step-function discontinuities. Is it now digital or analog?
Computational Representation
The fundamental structure and function of most modern computers is
usually attributed to John Von Neumann’s work in the 1940’s and
1950’s. The key idea is that of a stored program computer. That is, the
program which directs the computer’s actions is stored explicitly in the
computer’s memory. The conceptual underpinning of the stored program
computer is the notion of an automated formal system.
We find in the CPU of a modern computer an electronic circuit set up
so that certain patterns of electrical activity which we interpret as
instructions (e.g. the instruction to add two numbers together) are so
structured that they cause the computer to do just that. The meaning of the
instruction matches its causal effect on the CPU.
Von Neumann architectures take their basic idea from Turing’s
universal machine. You have one place to store tokens (in Turing’s
example this was a linear tape divided into boxes), where the tokens are
typed as above. The trick is to set up the basic workings of the machine so
that it can read its instructions off the tape along with the data it is
processing.
Instead of tape, computers make use of electronic memory. In the
memory is stored the program (a list of instructions about what to do when)
and the data (e.g. two numbers to add, or the works of Shakespeare to
search for a keyword, or whatever).
There has to be a finite number of basic instructions that are ‘wired
into’ the computer. That is, the computer is hard-wired to understand this
finite set of instructions. Typically, these basic instructions include collecting
binary numbers from particular memory addresses, adding two numbers
together, and placing numbers in memory addresses. From the basic set of
instructions can be built quite complex programming instructions.
We are particularly interested in what kind of representation such
computers make use of. Prima facie, there are two different kinds of
representation to be found in the computer.
First, there is the representation of the program. The instruction to
add two numbers together is a primitive ‘machine language’ instruction.
When represented inside the computer, this instruction will in fact be strings
of 1’s and 0’s whose electrical properties have the effect of adding two
binary numbers. This sort of machine language instruction illustrates the first
sort of representation we encounter. And each one represents a particular
program instruction.
The second sort of representation is the representation of the data. In
this case, the data are numbers, and they are represented by binary
numerals in the familiar way. But computers can represent all sorts of other
data: letters, symbols, pictures, graphs and so on. All of these sorts of data
are coded in binary form, according to some coding scheme (e.g. ASCII:
A=65, B=66 etc.).
This distinction is important in studying the computer’s apparent skill
at ‘understanding’ representations. It is first and foremost the program
instructions that the computer ‘understands,’ if it understands at all. The
computer can follow the program instructions (e.g. to add two numbers)
because it is hard wired to do so. It is program instructions that drive the
intuition that programs can properly be said to read and understand symbol
systems. On the other hand it can not be so readily argued that computers
understand the data that they store.
Connectionism
Just as the idealised neuron is a simple processing device with many
inputs and one output, the basic connectionist unit is also a simple
processing device with many inputs and one output. In the neuron, the
inputs and output are typically measured in terms of the amplitude or
frequency of electrical impulses (though this is a simplification). In crude
terms, the value of a given neuron's output is a weighted sum of the values
of all its inputs. It is these basic features of neurons which are also found in
the connectionist 'unit'.
The interesting characteristics of these units is that when combined
into large networks, the network as a whole can exhibit cognitively
interesting behaviour.
There are a number of different forms of representation identifiable in
connectionist networks. Networks are capable of ‘local’ representation,
typically in schemes where each unit has a single semantic interpretation.
The most obvious form of representation in connectionist networks is
that found in the activation of the network’s units.
The figure illustrates a network discussed by Paul Churchland which is
designed to distinguish between mines and rocks on the ocean floor on the
basis of sonar echo ‘profiles’ (Churchland, 1989b).
Fig.2
In this network, the input activation vector is a representation of the echo
profile, and the activation of the two output nodes represents the presence
of a mine or a rock, respectively. That a particular input was caused by a
mine, for example, is represented by a certain activation of the output units.
The activation vector of the hidden units at first seems not to represent
anything much at all, but close inspection shows that the network operates
to divide the ‘activation space’ into two sections, each associated with an
output node (see Figure 3). That is, the multi-dimensional ‘space’ of
hidden activations (where each hidden unit determines one dimension) is
partitioned such that mine echoes cause activation in one half of the space,
and rock echoes cause activation in the other half. The output units are then
sensitive to whether the activation of the hidden units is in one partition or
the other.
Fig.3
The information about mines and rocks that this network can be said to
represent is held in its connectivity. A network’s connectivity consists in
what unit is connected to what other units and with what strength (or
‘weight’). This information allows different networks to process information
differently.
In the mine/rock network, the weight connectivity represents that echo
profile A has come from a rock, echo profile B has come from a mine, and
so on. Thus the connectivity of the network as a whole (and no particular
connection or proper subset of connections) represents all these things.
The individual items of information that can be extracted discretely – for
example, that echo profile A comes from a rock – cannot be distinguished
by any discrete decomposition of the connectivity of the network. Only the
whole connectivity – all the information about which unit is connected to
which other unit with what weight – represents each ‘item’ of information.
This can be contrasted with computational and linguistic
representational schemes where discrete units of representation
correspond to discrete items of information. Thus in a computational
representation of the play Hamlet there is a discrete component of the
computer’s memory that represents Hamlet’s anguished question ‘To
be, or not to be?’ Most of the computer’s memory could be damaged and
many other lines from the play lost, but as long as certain pieces were left
behind, this line would still be represented. But if damage were done to the
connectivity of the mine/rock network, damage would be done to all the
information held about echo profiles – nothing would be left unscathed.
A last point to emphasise about connectionist networks is that they
suggest a radically different notion of computation than the familiar serial,
von Neumann architecture computers. A key difference is that all the stored
knowledge is made use of in both learning and processing, since the
knowledge is all stored superpositionally in the weights, and the weights are
involved both in the processing of inputs to produce outputs, and in the
readjusting of weights in learning.
Imagery
The mental imagery debate was largely sparked by experimental work
in psychology where results suggested that subjects were rotating and
scanning internal pictures at measurable rates.
The basic idea is a simple one. When we imagine a scene or an
object, it seems to us that we are observing a picture of that scene or
object. The experimental work shows that more than just seeming to view
internal pictures, we can operate on them just as if they were pictures -
rotate them at fixed speeds, scan them for information, zoom in and out,
and so on.
The controversy arises in interpreting the significance of these results
for understanding cognitive architecture. In particular, do these experiments
show that rather then there be symbols or sentences in the head, that
thinking is carried out (at least sometimes) in a pictorial or non-symbolic
medium?
The two positions in this debate may be dubbed ‘pictorialism’ and
‘descriptionalism’. According to pictorialism, mental images represent in the
manner of pictures, while according to descriptionalism, mental images
represent in the manner of sentences.
A key point in this debate is that you need to distinguish between the
pictorial nature of the experience of thinking of the image, and the pictorial
nature of the representation which gives rise to the experience. It seems
quite possible that a symbolic representation (e.g. list of sentences ) could
give rise to an imagistic experience (e.g. a matrix).
A second key point is that all systems of representation require
decoding. Even pictures can be culturally specific (e.g. photographs,
mirrors), illustrating that you have to learn to see them.
These considerations suggest that the psychological evidence is not
conclusive on the architectural question. Nonetheless, the discussion raises
the interesting question: how would be determine if a given representation
(or scheme of representation) is symbolic or pictorial?
Are symbolic schemes completely intertranslatable with imagistic
schemes?
Block argues that to the extent that the mind does make use of
imagery, it is unavailable to cognitive science, and is rather the domain of
neuroscience proper. This is because the functional architecture required
for operations over pictures would need to be more sophisticated than the
architecture which underpins symbolic systems.
Sterelny argues that the possibility that the imagistic evidence could
be explained by a symbolic representational base renders the hypothesis
that internal representation is pictorial non-explanatory.
Representational Schemes and Genera
Representation tokens are usually thought of as members of a general
scheme of representation. Thus letters are members of an alphabet; words
are members of a lexicon, and decimal numerals are members of the set of
10 digits digits ‘0’ … ‘9’. There are exceptions. When I use a glass to
represent the earth, a plate to represent the sun, and a fork to represent
the moon in a description of their relative positions during a solar eclipse,
then the representational vehicles (the glass, plate and fork) are not
members of any previously defined scheme of representation. They
constitute an ad hoc scheme invented for this particular use, and they have
no relationship to one another as representational vehicles. [incomplete
distinction]
It is generally assumed in the cognitive sciences that there are three
genera of representation, and they are defined in terms of the relation
which holds between representational vehicle and content. (John Haugeland
says that this is the canonical view in the sense that ‘almost everybody
expects almost everybody else’ to believe it.) Thus 'logical'
representational schemes such as languages and formal symbol systems
have compositional semantics – there are clear rules which define the
meaning of a complex representation in terms of the meaning of the atomic
constituents of that complex representation. 'Iconic' schemes such as scale
models and pictures are isomorphic to their contents – the representational
vehicle shares structure with the content. ‘Distributed' schemes such as
holograms and connectionist weight vectors superpose many contents in
one vehicle.
Sentences
Logics, natural languages, sign systems, computer programming
languages.
It is clear that one distinctive feature of many logical schemes is a
fairly complex sort of compositional semantics, like that found in natural
languages, rather than the simple compositionality of concatenation found in
pictures. For example, a scheme which can distinguish between conjunction
and disjunction shows significant complexity in its compositional semantics.
The ability to represent negation and complex, abstract combinations of
atomic contents is a source of the significant utility of some logical
representational schemes. Similarly predication is another compositional
form which adds depth and complexity to a representational system.
Necessary for this sophisticated sort of compositionality, it would
seem, is the ability to group vehicles according to well-defined types. This is
because sophisticated compositionality depends on general rules of
composition, and such general rules need a well-defined domain to operate
over. And type-identification itself seems to require that the vehicles be
discrete (as contrasted with the continuous nature of some iconic vehicles,
for example scale models).
Considering the very loose constraint on the range of possible vehicles
for logical schemes, and that they can represent pretty much anything, the
only thing that can be said, in general, about the relation between vehicles
and contents is that it is arbitrary.2 Of course, in a logical scheme with
compositional semantics, the relation between complex vehicles and their
contents won’t be arbitrary because the content of the vehicle will relate in
fixed ways to the contents of its component parts.
So a refinement to the canonical account is the idea that the
representing relation of logical schemes of representation be arbitrary for
atomic vehicles. Sophisticated compositionality might however be a
sufficient condition on a scheme’s being logical.
Pictures
Iconic representations represent relations among different properties.
So, if the velocity of a car is represented by the height of a rectangle, and
the time spent travelling at that velocity by the length of that same
rectangle, then the distance travelled by the car in that time will be
represented by the area of the rectangle. If the representation of velocity
changes, the area automatically changes; it does not need to be
recalculated because the relational structures have been preserved in the
representation.
Fig.4
Canonically, isomorphism is the defining relation of iconic schemes. The key
notion is that iconic representation obtains when there is a reproduction of
structure. Thus a bust of Immanuel Kant represents Kant’s head and neck
because they share structural features. Mathematically, structure is
understood as a set of elements and a set of relations over those elements.
So in Figure 4, there is a 1-1 mapping from possible car velocities to
possible heights of the rectangle, from possible durations to possible
widths, and from possible areas to possible distances.3 These mappings
are trivial because each domain is the real numbers. Critically, there is also
a mapping from the relation between time, velocity and distance in the car
to the relation between the height, width and area of the rectangle. So a
structure in the square is the same as one in the body moving with uniform
velocity. Thus the representational relation – the relation between the
representational vehicle and the content – is that of identity. The structure
of the content is reproduced in the representation.
Generalise to scale models, covariation of numerical values, isomorphism.
Pictures, maps, scale models, graphs, charts and diagrams.
Superpositional Representation
Connectionist networks have brought superpositional representation to
prominence in the cognitive sciences. A representation of a contents c1 and
c2 is superpositional if the resources used to represent c1 are identical with
the resources used to represent c2. That is, one and the same token
representation has the role of representing both contents. This is actually a
more familiar kind of encoding than is often recognised.
Consider the example of sound. A piano is played, causing a pattern
of air pressure at the point at which a microphone picks it up (which then
transduces the air pressure pattern into a pattern of voltage). A singer
sings, similarly causing a characteristic pattern of air pressure. If the piano
and singer make a noise at the same time, then their characteristic patterns
of air pressure are superposed (everywhere, but in particular) at the point
where the microphone picks up the signal and transduces it. Thus the
electrical encoding of the sounds of the piano and the voice is a
superpositional representation. This may then be recoded on a CD and
reproduced through an amplifer and speakers.
Holograms are another well-understood form of superpositional
representation: the holographic plate records a large number of incident
light arrays (from different angles), and (under the right conditions) is able
to reproduce all these light arrays, each radiating in a distinct direction,
hence as the viewer moves around the hologram, she sees different
scenes.
Many connectionist networks exhibit superpositional encoding. The
complex connectivity of the network – the vector composed of the weights
of each connection between units (‘synapse’) in the network – constitute
a superpositional memory. Networks learn to process information in various
ways, and the ‘knowledge’ they thereby learn is encoded in the the
network’s weights. Generally speaking, no particular weight or group of
weights is responsible for the representing of any particular content.
An important consequence of this form of data storage is graceful
degradation. Damage to part of a network, or part of a hologram leads to a
partial degradation of the whole image. This is in contrast to both pictorial
and logical forms of representation where the loss of part of the
representation (damage to a hard disk, or the tearing of painting, has no
effect on the representing of the other parts of the representation.
Another important consequence of this form of representation as it is
employed in connectionist networks is that it generalising inferences to be
drawn by the act of representing. In a famous, but perhaps over-simple
example, Ramsey, Stich and Garon trained a network to store
(superpositionally) a number of propositions, for example ‘dogs have fur’.
The network generalised from the `knowledge´ that dogs have fur, paws,
fleas and legs, and that cats have fur, paws and fleas, to the `knowledge´
that cats have legs.
Thus the connectionist use of superpositional representation has
initiated a re-evaluation of representation in general by positing questions
such as ‘Are representations distinct from processes of inference?,’ and
‘Might different kinds of representation permit different kinds of cognition?’.
Explicit and inexplicit
A good place to start with these worries is Dennett’s ‘Styles of Mental
Representation’. Dennett distinguishes between explicitly represented
information, which is held in the ordinary kind of explicit representations with
which we are familiar – sentences, program instructions, well formed
formulae of a formal system and so on – and inexplicitly (Dennett says
‘tacitly’) represented information, which is simply embodied by a system or
organism. One kind of inexplicit knowledge is the know-how of the digital
computer. It inexplicitly knows how to follow certain rules; viz. machine
instructions. Dennett understands one sort of inexplicitly represented
information as the information that a system has which allows it to use and
manipulate explicit representations, but which is not itself encoded explicitly.
For example, the ability to add two binary digits of given length is typically
wired into a CPU, and so we could say that it inexplicitly represents how to
add.
The more recent literature on this topic (Kirsh 1989, Clark 1993) argues
convincingly that there is no clear, absolute distinction between the explicit
and inexplicit representation of information. Encryption of English sentences,
for example, can yield representations which have all the same features as
English sentences, but don’t appear to represent their contents explicitly.
With skill, however, some readers might be able to decrypt on sight, without
laborious processing. In such a case I think we have to say that the
information which was inexplicit to the (untrained) decoder has become
explicit to her as she becomes skilled at decryption. In contrast information
about the constitution of the USA is explicitly represented somewhere in my
University’s library, but right now it is opaque to me. I have to go through
quite a lot of processing to get at that information. As far as I’m concerned
right now, the information is not explicit to me, and hence inexplicit.
These sorts of consideration lead Kirsh to argue that explicitness is
relative to the system using the information, and isn’t captured by
superficial features such as concatenation, spatial isolatibility and so on.
Clark extends this analysis and argues that information is properly explicit
only if, along with being easily deployable, it can be deployable in many
different ways. If information is present in the system and easily available to
a single process, it might still not be fully explicit. The content of ‘the grass
is green’ is available for all sorts of different uses to us (answering
questions about grass, about greenness, about living things, about colour,
etc. etc.); however the information about addition embedded in the CPU of
a computer, while available for the purpose of adding two binary digits of a
fixed length, isn’t available for theorising about the properties of addition.
So determining whether certain information is represented explicitly or
inexplicitly depends on the availability of that information to various
processes in the system, and so explicitness is relative to different possible
contents, and to the systems which makes use of those contents.
Notes
<1> This is not all there is to its content, however; the content also includes
an aspect of the visual presentation of the tree itself - how it looks from a
certain perspective.
<2> Colin McGinn [2] p. 178 makes a similar claim.
<3> I say 'roughly' because there are a number of idealisations going on
here. The rectangle may not be of indefinite size because we don't the have
the resources to draw very big rectangles. Note also that we are assuming
that the car in question has uniform velocity throughout the associated time
period.
 Top of page
|
