DAC Hands as Molecules Representational Gestures Used for Developing Theory in A Scientific Laboratory Article Annotation

Basic response: this kind of response demonstrates comprehension of the text by summarizing key points without extensive reliance on direct quotations.

Extra response: to earn a participation point, you may add to the above “basic response” additional reflections including but not limited to the following: generative questions for discussion among your classmates; substantive questions on issues you don’t understand; critical assessments of the readings or alternative interpretations that you find more compelling; or comparison between concepts from this week’s texts and those from another week or another course.

Topics in Cognitive Science 9 (2017) 719–737
Copyright © 2017 Cognitive Science Society, Inc. All rights reserved.
ISSN:1756-8757 print / 1756-8765 online
DOI: 10.1111/tops.12276
This article is part of the topic “2016 Rumelhart Prize Issue Honoring Dedre Gentner,” Jeffrey Loewenstein and Arthur B. Markman (Topic Editors). For a full listing of topic
papers, see: http://onlinelibrary.wiley.com/doi/10.1111/tops.2017.9.issue-2/issuetoc
When Gesture Becomes Analogy
Kensy Cooperrider, Susan Goldin-Meadow
Department of Psychology, University of Chicago
Received 2 May 2016; received in revised form 17 January 2017; accepted 24 March 2017
Abstract
Analogy researchers do not often examine gesture, and gesture researchers do not often borrow
ideas from the study of analogy. One borrowable idea from the world of analogy is the importance
of distinguishing between attributes and relations. Gentner (1983, 1988) observed that some metaphors highlight attributes and others highlight relations, and called the latter analogies. Mirroring
this logic, we observe that some metaphoric gestures represent attributes and others represent relations, and propose to call the latter analogical gestures. We provide examples of such analogical
gestures and show how they relate to the categories of iconic and metaphoric gestures described
previously. Analogical gestures represent different types of relations and different degrees of relational complexity, and sometimes cohere into larger analogical models. Treating analogical gestures as a distinct phenomenon prompts new questions and predictions, and illustrates one way
that the study of gesture and the study of analogy can be mutually informative.
Keywords: Gesture; Analogy; Metaphor; Iconicity; Relational reasoning; Abstract reasoning
1. Eggs and omelets, attributes and relations
There are a number of differences between an egg and an omelet. One is raw, the
other cooked. One is small and oval, the other flat and round. If one wanted to represent
such features in gesture, it would be easy enough. For instance, you could show the
small, contained shape of the egg by holding up a closed fist. For comparison, you could
Correspondence should be sent to Kensy Cooperrider, Department of Psychology, University of Chicago,
5848 S. University, Chicago, IL 60637. E-mail: kensy@uchicago.edu
720
K. Cooperrider, S. Goldin-Meadow / Topics in Cognitive Science 9 (2017)
Fig. 1. Four gestures produced in the course of repeated reference to an “egg” and an “omelet.” In the first
series (top), the speaker produces iconic gestures that represent concrete aspects of the egg and the omelet. In
the second series (bottom), his gestures do not represent attributes, but instead relate these two entities in
time, with the egg on his left (earlier) and the omelet on his right (later). The second series constitutes a simple analogical model.
then model the shape of the omelet with two hands. Indeed, the cosmologist Sean Carroll
produces gestures very much like this at the start of a 2009 lecture on the origins of the
universe. They occur as he says: “We can take an egg and turn it into an omelet.” These
are what gesture researchers call iconic gestures—gestures in which handshapes and
movements represent visuospatial features of the world.
Immediately afterward, Carroll produces another pair of gestures as he continues: “But
it’s very hard to take an omelet and turn it back into an egg.” In this second series, the gestures have a more abstract character. As he says “omelet,” he twists to his right and brings
both hands firmly down on the right side of his gesture space; as he says “egg,” he brings
his hands back across his body and then down on the left side of his gesture space. This
time nothing about his handshapes distinguishes the “egg” from the “omelet”; indeed, nothing about the gestures differs except for the locations where the hands are placed in space.
Whereas in the first series Carroll’s gestures represent attributes of the egg and the omelet—
in particular, their size and shape—in the second series, he highlights a relation between
those objects (Fig. 1). The key relation is a temporal one: Thanks to the irreversibility of
time, the omelet must happen after the egg. Carroll conveys this relation by placing his
K. Cooperrider, S. Goldin-Meadow / Topics in Cognitive Science 9 (2017)
721
gestures along a left–right axis representing time: The space to his left (where the gesture
for egg is produced) represents an earlier time than the space to his right (where the gesture for omelet is produced). For Carroll’s purposes, the egg and the omelet are interesting because they illustrate time’s arrow, a bedrock principle of cosmic order. For our
purposes, what is interesting is that Carroll’s second gesture series, in which attributes
are abstracted away and relations take center stage, has all the features of an analogy.
The importance of the distinction between attributes and relations will be obvious to
those familiar with the wide-ranging work of Dedre Gentner and her collaborators. Over
several decades of research, Gentner has shown how this distinction can illuminate some
of the most important topics in cognitive science: how figurative language is processed,
the relationship between language and thought, how children learn, the history of scientific reasoning, the difference between human and animal intelligence, and many more.
Research by other scholars, too, has explored the many ways in which higher-order cognition depends on relational reasoning (Doumas, Hummel, & Sandhofer, 2008; Goldstone,
Medin, & Gentner, 1990; Hofstadter & Sander, 2013; Holyoak & Thagard, 1995; Markman, 1997). Yet in the world of gesture studies, this powerful distinction has not yet
taken root. Our aim in this paper is to show that the contrast between attributes and relations—as well as related analytic tools from Gentner’s structure mapping framework—
can cast light on gesture and cognition. In particular, these tools call attention to different
classes of gesture that have previously been lumped together, but which—given the
importance of relational structure in human thinking—should not be. Thus, the study of
analogy can inform research on gesture, opening up a host of new questions (see also
Cooperrider, Gentner, & Goldin-Meadow, 2016). Our bet is that, in turn, the study of gesture can inform research on analogy, providing a new arena within which to investigate
how analogical thinking is manifest in everyday behavior.
We are certainly not the first to try to bring these two areas into contact. In recent
years, several researchers have begun to use structure mapping (and other conceptual
mapping approaches) to understand meaningful body movements. For example, Taub
(2001) has analyzed iconicity in American Sign Language as involving a structured mapping between a mental representation (e.g., of a tree) and a certain configuration of the
body (e.g., a forearm held vertical, with the fingers spread). Several researchers since
have used similar approaches to explain features of sign language metaphors (e.g., Meir,
2010) and to account for findings about sign language processing and learning (e.g.,
Emmorey, 2014). The iconicity identified in sign language is also ubiquitous in the spontaneous gestures that speakers produce, and some have begun to use conceptual mapping
frameworks to analyze these gestures (e.g., Clark, 2016; Parrill & Sweetser, 2004). Any
gesture that depicts what it refers to can be analyzed through the lens of structure mapping. For instance, the egg-as-fist gesture involves a mapping between a mental representation of an egg, with features of roundness, smallness, and so on, and a bodily
representation of an egg, with the fist forming a small round shape. As described in
structure mapping theory, the mappings in this example are selective (Gentner, 1983).
The forearm protruding from the fist has no counterpart in the mental representation of
an egg, and the yolk does not have any counterpart in the bodily representation. Structure
722
K. Cooperrider, S. Goldin-Meadow / Topics in Cognitive Science 9 (2017)
mapping approaches may indeed prove to be a powerful lens through which to analyze
iconic gestures, and it may generate predictions that would not have otherwise been
made. Our focus is not on these relatively concrete gestures, however, but on what are
commonly called “metaphoric gestures.” In iconic gestures, the visuospatial properties of
the gesture directly represent the visuospatial properties of a concrete referent, be it an
object, action, or spatial relationship. In so-called metaphoric gestures, visuospatial properties of gesture are used to represent abstract ideas and relationships, concepts that have
no inherent visuospatial properties.1 In Section 2, we elaborate on a key difference
between different kinds of metaphoric gestures, one directly inspired by one of Gentner’s
early observations about figurative language.
2. Some metaphoric gestures are analogies
For as long as researchers have studied gesture, they have noted that gestures often
render abstract thoughts into visible form. The Roman rhetorician Quintillian, writing in
antiquity, and the German psychologist Wilhelm Wundt, writing in the 19th century, both
comment on such gestures (discussed in Kendon, 2004). David Efron (1972), who made
systematic observations of gesture without the benefit of video, separated out a class of
what he called “logicotopographic gestures,” which represent a “diagram of the ideational
structure of discourse” (Kendon, 2004, p. 93). More recently, in the influential typology
put forward by McNeill (1992), “metaphoric gestures” are considered a distinct type in
which “abstract content is given imagery in the form of objects, space, movement, and
the like” (p. 145). This is a sprawling, colorful, and ubiquitous category. Indeed, McNeill
comments that, in certain kinds of discourse such as conversations and lectures, metaphorics are “among the most frequent of all gestures” (p. 163). Given the diversity of this
class, it is natural to ask whether more fine-grained distinctions might be possible—and,
more important—useful. That is, are there distinctions that might matter for the kinds of
questions gesture researchers—and cognitive scientists generally—care about?
Here, we draw an analogy. The state of research on gestural metaphors today might be
likened to the state of research on verbal metaphors before Gentner (1983). For a long
time, verbal metaphor was an undifferentiated category, and “metaphor” and “analogy”
were often treated as rough synonyms (as they often still are today in informal language).
Gentner and colleagues have proposed that both metaphor and analogy are non-literal
comparisons that relate a target concept to a base concept; however, analogy is a special
case, one in which the two concepts share relations but do not necessarily share attributes
(Gentner, 1983, 1988; Gentner & Markman, 1997). Consider two figurative expressions,
“The sun was a grapefruit” and “That job is a jail.” In the first, the target (sun) shares
with the base (grapefruit) salient attributes—for example, both are spherical. In contrast,
in “That job is a jail,” the target (job) shares no salient attributes with the base (jail), but
it does share salient relations—for example, both are confining. The “job as jail” figurative expression is a special case of non-literal comparison in which relations are privileged over attributes, and it is this type that Gentner labels analogy.
K. Cooperrider, S. Goldin-Meadow / Topics in Cognitive Science 9 (2017)
723
Continuing with our own analogy, we might ask: If there is a special case of non-literal comparison in which relations are paramount, might there also be a special case of
abstract gesture in which relations are paramount? Yes, there is—and we call this special
case analogical gesture. The critical feature for this distinction is what the gesture represents.2 A series of minimally contrasting examples will help clarify what sets analogical
gestures apart from other metaphoric gestures and from iconic gestures (see Table 1).
Again, iconic gestures use space (base representation) to represent concrete spatial
attributes and relations of whatever is being described (target representation). An example
of an attribute iconic would be a person using two C-shaped hands, spread apart, to represent the large size of a beach ball. A relational iconic might represent the spatial relation between two objects, for example, using two flat palms, one on top of the other, to
show that one book is above another on a shelf. Metaphoric gestures also use space (base
representation) to represent attributes and relations of whatever is being described (target
representation), but those attributes and relations are not inherently spatial. A case of an
attribute metaphoric would be a speaker using two C-shaped hands, spread apart, to
convey the importance of an idea by representing it as large in size. A relational
metaphoric—the subtype we are calling an “analogical gesture”—might be a speaker
using two flat palms, one on top of the other, to show that one idea is more important
than another by representing its placement on a vertical scale of importance, with one
idea being higher (i.e., more important) than the other. In the relational iconic example,
the hands show an actual spatial relation—vertical position on a shelf. In the corresponding analogical gesture example, the hands represent a relation that is not inherently spatial—relative importance—as a spatial relation.
Our focus in what follows will be on analogical gestures—metaphoric gestures that
foreground relations. But, first, it is worth saying a bit more about metaphoric gestures
that highlight attributes, which have figured prominently in prior treatments of abstract
gesture. A common case of an attribute metaphoric involves representing an ethereal
abstraction as though it had some of the properties of an actual, physical object. Mittelberg (2008) describes several such gestures produced in introductory linguistics classes.
One professor refers to “grammar” while putting both hands out, palms facing each other,
Table 1
Subtypes of iconic and metaphoric gestures
Iconic Gestures
(space represents space)
Metaphoric Gestures
(space represents non-space)
Attributes
attribute iconics
represented
example: two-handed gesture
representing the size of an entity
attribute metaphorics
example: two-handed gesture representing the size
of an idea to convey its importance
Relations
relational iconics
represented
example: two-handed gesture
representing one entity above another
in space
relational metaphorics (= analogical gestures)
example: two-handed gesture representing one
entity above another to convey its relative
importance
724
K. Cooperrider, S. Goldin-Meadow / Topics in Cognitive Science 9 (2017)
as though holding a small box (Mittelberg, 2008, p. 126). Grammar is thus treated as an
object of a certain size and shape. A different lecturer refers to the idea of a “main verb”
while extending a cupped hand, as though holding the verb out for inspection (p. 129).
Grammars and verbs do not, of course, have any physical form—they are intrinsically
abstract ideas that become, in McNeill’s terms, “entified” in gesture (McNeill, 1992, p.
154). Other attested examples of metaphoric gestures in which attributes are central
include gestures showing the “size” of a duration (e.g., Cooperrider & N
u~nez, 2009) or a
number (e.g., Winter, Perlman, & Matlock, 2013), as though it were the size of an object.
Another example is the precision grip gestures produced with words like “specifically” or
“especially,” which recast the notion of specificity in terms of small size (Kendon, 2004;
Lempert, 2011).
Now that we have situated analogical gestures within a general framework, we can
consider some complexities. A first complexity is that analogical gestures do not necessarily represent relations between abstract entities. Our opening example involved an
egg and an omelet. Unlike grammar or other ethereal ideas, eggs and omelets are concrete—you can pick them up, drop them on the floor, or eat them. What is abstract is
the relation between the egg and omelet—that the egg is necessarily the precursor of
the omelet. Thus, analogical gestures use spatial relations to represent non-spatial relations between entities or ideas, no matter whether those entities are themselves abstract
or concrete. A second complexity is that analogical gestures can represent both attributes and relations at the same time—it is not either–or, much as in the case of verbal
metaphors that highlight both common attributes and common relations (Gentner, 1988).
This would be the case if, for instance, Carroll had depicted the shape features of the
egg and the omelet while also using space to represent the temporal relation between
them. Gestures that represent actions—whether iconic or metaphoric—commonly represent both relations (who acts on whom) and attributes (what the action was like). A
third complexity is that, much as the distinction between literal and non-literal similarity is a continuum in structure mapping theory (e.g., Gentner & Markman, 1997), the
distinction between iconic and metaphoric gestures may also be thought of as a continuum. Intermediate cases exist in which a gesture dimly echoes an actual spatial relationship, but schematizes it to degree that makes it quite abstract. For example, if a
speaker contrasts the United States and Australia by gesturally placing the United States
on her left and Australia on her right, this placement echoes an actual relationship on
an east–west axis, but also schematizes the relationship, in that the countries are at different latitudes.
To our knowledge, analogical gestures have not been treated as a separate class
before. Our first task is thus to characterize this new class in more detail: In Section 3, we consider how analogical gestures vary in the type and complexity of the
relational structure involved; in Section 4, we describe how analogical gestures can
cohere over time to form larger analogical models. After characterizing analogical gestures, we can turn to the question of why cognitive scientists might care about them,
drawing on prior findings about analogical reasoning in general and about gesture and
cognition.
K. Cooperrider, S. Goldin-Meadow / Topics in Cognitive Science 9 (2017)
725
3. Representing relations in analogical gestures
Once we import a focus on relations and relational structure into the study of
gesture—and into the study of abstract gestures in particular—a number of new questions
pop into focus. To start, what kinds of relational structures are represented in gesture,
and how complex do these relational structures get? Perhaps the most basic relation that
is represented in gesture is that of difference, a kind of ontological relation. Gesture is so
commonly used to contrast two entities, ideas, or propositions that the practice shows up
in our spoken idioms (“on the one hand” and “on the other hand”). Again, it does not
matter whether the entities contrasted are themselves concrete or abstract, what matters is
that space is used—not to represent an actual spatial relationship between the entities in
the world—but to represent that they are fundamentally different.
Also common is using gesture to place entities in a spatial arrangement that represents
order along some abstract dimension, such as value, magnitude, time, and others. The
best studied case of this use of gesture to date is temporal gestures, which locate entities
or events according to a “timeline” (as seen in the egg and omelet example). Temporal
gestures were first described in detail in English (Cienki, 1998), and have since been
studied in global languages like Mandarin, as well as in indigenous languages like
Aymara (South America) and Yupno (Papua, New Guinea) (see Cooperrider, N
u~nez, &
Sweetser, 2014, for a review). Although these different languages use strikingly different
spatial relations to represent temporal relations, all share the core feature that temporal
relations are systematically represented in gesture as an ordered spatial arrangement.
Scales of magnitude and valence are extremely common in everyday language and graphical culture and are likely widely realized in gesture as well, although they are not yet
well studied (for some examples, see contributions in Cienki & M€uller, 2008). Indeed,
any dimension that is construed as having an ordered relational structure—with a less
pole and more pole—is ripe for representation in analogical gestures.
In the simplest case, a speaker may place a single entity within an implicit ordered
arrangement—placing “tomorrow” to the right (on an implicit horizontal timeline) or “the
boss” up high (on an implicit vertical scale of power). Such gestures may be said to be
tacitly relational—and thus analogical—insofar as they relate a mentioned entity to other
unmentioned entities. The relation between an entity at one time point and the same entity
at another time point can also be captured in gesture—these comparisons are commonly
realized in gesture as movements. Increases and decreases in an entity’s magnitude, for
instance, may be represented as movements along the vertical axis (e.g., Cooperrider et
al., 2016). Relatedly, changes in scalar properties such as pitch are also represented in
gesture as shifting positions in vertical space. Indeed, such gestures are sometimes used in
music education and in teaching tonal languages like Mandarin. The same gestural
devices can be used to capture the relation between two entities on a single dimension.
Relational structure can get much more complex than a change in a single entity, or a
relation between two entities on a single dimension. One type of further relational complexity is when two entities change at the same time. An example is the notion of a
726
K. Cooperrider, S. Goldin-Meadow / Topics in Cognitive Science 9 (2017)
trade-off—the idea that, as one thing undergoes a change, something else undergoes an
opposite change. In a 2014 television interview, Christine Lagarde, acting director of the
International Monetary Fund, produced a gesture to represent this concept while discussing how to manage inflation and interest rates. She says: “Again, it’s a question of
measure and balance. You know, obviously inflation rates and interest rates go together
and sort of balance out. There is a trade-off between the two.” As she says “trade-off”
she holds both hands flat, palms facing downward, and rocks them in an alternating
motion (see Fig. 2A). In a single gestural image, she thus captures the idea that there are
two different entities—inflation rates and interest rates—and as one changes, the other
changes in a roughly opposite way.3
For comparison, consider another example of a gesture representing the notion of a
trade-off, also involving an alternating movement pattern (see Fig. 2B). In this case,
James Steinberg, former Deputy Secretary of State, is making the point that the interests
(A)
(B)
Fig. 2. Two examples of gestures produced to represent the notion of a “trade-off.” Both examples involve
alternating motions of the hands, thus capturing the idea that, as one entity changes, another changes in the
opposite way.
K. Cooperrider, S. Goldin-Meadow / Topics in Cognitive Science 9 (2017)
727
of emerging powers like India, China, and Brazil are not fundamentally at odds with the
interests of the United States. He says: “So it’s not a trade-off. This is not a 19th century
competition of powers.” As he says “trade-off,” he moves both hands, fists closed, several
times in an alternating in-and-out motion. Both examples involve alternating two-handed
movements, but different hand shapes (open palms in the first example, fists in the second) and different directions of movement (up-down in the first example, and in-out in
the second), making it clear that there is more than one way to represent a trade-off in
gesture.
A trade-off is what Gentner (1983) termed a relational abstraction—a complex relational structure that comes to be stored as a whole (and that may have a verbal label, as
in this case). Other examples of abstractions that may be ripe for gestural representation
include reciprocity, quid quo pro, pre-emption, domino effect, Catch 22, vice versa, circular reasoning, vicious cycle, homeostasis, and many others. Although there does not
appear to be a rigidly conventional “trade-off gesture”—as evidenced by variation in the
forms used in our two examples—it remains an open question as to whether such relationally complex gestures only occur after an abstraction is in place. That is, do complex
relational gestures show up only after an abstraction has already become crystallized and
labeled, or might these gestures reflect abstract relational structure as it is being discovered and articulated for the first time?
4. Analogical models in gesture
Analogies take time to express. Just as it is not always possible to capture an apt comparison in a single word or phrase, it is not always possible to express a comparison in a
single gesture. After all, we only have two hands, ten fingers, three spatial dimensions,
and so much room within our reach. The trade-off gestures just described are remarkable
in that they compress a substantial amount of relational information into a quick movement. While there are clear limits on what a single gesture can represent, speakers can
skirt these limits by building up relational structure over time, gesture by gesture. We
will refer to sequences of analogical gestures that cohere into larger spatial models of
relational structure as analogical models.
Most work on gesture has taken single, isolated gestures as the unit of analysis. There
are good practical reasons for this tendency, as looking at models rather than one-off gestures can be messy. Still, several researchers have noted that such models exist and have
begun to study them more systematically. McNeill (2005), for example, noted how gesture features often recur over time, forming threads of visuospatial imagery that he has
called “catchments.” He also noted a related phenomenon by which speakers establish
referents in space and then maintain those locations over stretches of discourse (akin to
establishing referential loci in sign language; see Emmorey, 2002; Liddell, 1995; Taub,
2001). More recent studies have explored the conditions under which space is used to
€ urek, 2015; So, Kita, & Goldin-Meadow, 2009). In
track locations (e.g., Perniss & Ozy€
another line of work, both signers in their signs and speakers in their gestures have been
728
K. Cooperrider, S. Goldin-Meadow / Topics in Cognitive Science 9 (2017)
found to build up models over time when describing concrete spatial layouts (Emmorey,
Tversky, & Taylor, 2000). In this case, rather than merely echoing a previously established mapping between a particular referent and a particular location, the speaker or
signer adds new spatial information that integrates with—and extends—information presented earlier.
The egg and omelet example contains the rudiments of an analogical model in the
sense just described: The gestures are produced one after another and build on each other
to form a single model. Perhaps more interesting would be cases in which a barebones
relational model is established and then elaborated with subsequent gestures. Take, for
example, gestures that accompany the phrase vice versa,4 another of the relational
abstractions mentioned earlier. In a first example, in Fig. 3, the interviewer Peter Slen is
posing a question about how American political parties are often perceived as having
ownership over particular issues. He asks: “Have there been issue ownership changes,
where the Democrats owned it for awhile and now the Republicans own it, or vice
versa?” As the gesture sequence begins, both of Slen’s hands are out in front of his body.
As he says “Democrats,” he brings his left hand down on his left side (Fig. 3A); then
with “Republicans,” he brings his right hand down on the other side (Fig. 3B). Finally, as
he says “vice versa,” he rotates both hands over top of each other, as though manipulating something in the space between the two previously established locations (Fig. 3C).
Fig. 3. Two examples of gesture sequences that cohere into analogical models, both involving the phrase
“vice versa.” In each case, entities are first laid out on different sides of the speaker’s gesture space (A,B;
D,E) and then the relation between them is manipulated with a third gesture (C,F).
K. Cooperrider, S. Goldin-Meadow / Topics in Cognitive Science 9 (2017)
729
In a similar example, the psychologist Steven Pinker is arguing that grammatical errors
are often a matter of using the wrong style for the occasion. He says: “Using the wrong
style in either direction. . . a formal style when an informal one is called for—or vice
versa—are both grammatical errors.” His first gestures lay out the two entities involved:
he brings his right hand crisply down on his right as he says, “formal style” (Fig. 3D),
and then moves the same hand over to the left and crisply down when saying, “informal
style” (Fig. 3E). Completing the sequence, he brings his hand between the two established positions, extends two fingers, and twists his wrist as though reassigning the entities to different positions (Fig. 3F). As with the two trade-off examples, the variation in
form in the vice versa examples suggests that the gesture is not conventionalized.
Both vice versa sequences involve an integrated series of three gestures: The first two
map entities to locations, and then the third operates on these locations with a new gesture. Longer and more complex sequences can, of course, occur. In a line of research
conducted in collaboration with Dedre Gentner (Cooperrider et al., 2016), we have elicited explanations of positive and negative feedback systems, two examples of complex
relational abstractions. Such systems cut across a number of content domains and are far
from obvious, often eluding undergraduates (Goldwater & Gentner, 2015; Rottman, Gentner, & Goldwater, 2012). Although participants were not encouraged to gesture in our
studies, almost all did. Many produced at least rudimentary analogical models, and some
produced strikingly consistent and fully realized ones. An example of a consistent analogical model in gesture is partially illustrated in Fig. 4. The participant is introducing the
basic structure and behavior of a positive feedback system. She says: “If there are two
factors, one increases. That makes the other increase. And that, in turn, makes the first
one increase more.” To start, the participant lays out the two factors simultaneously in
space (see Fig. 4A), one represented by her left hand and one by her right. She then
shows a series of increases as vertical movements, first with her left hand (Fig. 4B), then
with her right (Fig. 4C), and then again with her left (Fig. 4D). Her spatial model does
not stop there: It is maintained throughout as she summarizes the overall dynamics of a
positive feedback system, and then contrasts it with a negative feedback system.
Fig. 4. Examples of an analogical model in gesture, produced during an explanation of a positive feedback
system. A first gesture (A) establishes the causal factors as locations in space; three subsequent gestures represent increases to the first factor (B), the second factor (C), and then the first factor again (D), as upward
movements.
730
K. Cooperrider, S. Goldin-Meadow / Topics in Cognitive Science 9 (2017)
Interestingly, examples of well-developed analogical models in gesture are not limited
to explanations of scientific or technical concepts. Another case of such models comes
from Enfield’s analysis of how Lao speakers express everyday kinship concepts in gesture
(Enfield, 2005). The richness of the phenomenon comes across especially vividly during
explanations of the notion of a “sibling exchange.” In Laos, when a couple marries, it is
common for a sibling of the wife and a sibling of the husband to also marry—a multifamily arrangement known as a “sibling exchange.” However, there are also important
restrictions: While an older sibling of the husband can marry an older sibling of the wife,
and a younger sibling of one can marry a younger sibling of the other, it is not allowed
for an older sibling of one to marry a younger sibling of the other. As a speaker explains
this prohibition, he produces a series of gestures that use left and right space to contrast
two families and vertical space to arrange the children within each family by age
(younger in age is represented as lower in space). Accordingly, to show that an older sibling of the bride and an older sibling of a groom may marry, the speaker places two
hands above his head, showing that they are both “high” and at the same “height.” Later,
to represent the prohibited arrangement in which a younger sibling of one spouse marries
an older sibling of the other, he crosses his forearms and thereby shows the violation of
“parallelism.” Enfield frequently refers to this phenomenon as the production of gestural
“diagrams.” The word is apt in that it connotes sparseness and perhaps also a focus on
relations rather than attributes. However, we favor analogical model because the term diagram applies equally well to concrete visuospatial content (e.g., a diagram of the circulatory system) and to purely abstract content (e.g., a diagram of an organizational
hierarchy) (for discussion of parallels between diagrams and gestures, see Tversky, 2011).
5. Questions and predictions
Once we start seeing gesture through the eyes of the structure mapping framework that
Gentner and colleagues have developed, a number of promising possibilities for further
research suggest themselves. In this section, we consider a few of the most tantalizing,
using as a guide questions and predictions that have come out of work on analogical reasoning and gesture more generally, and out of our own collaboration with Dedre Gentner.
Some of these questions and predictions concern analogical gestures specifically, whereas
others concern relational gestures generally, be they iconic or metaphoric.
5.1. Processing analogical gestures
A first question that arises is whether listeners process analogical gestures and, if so,
whether this processing shares features with how analogy and metaphor are processed by
listeners more generally. There is abundant evidence that listeners glean information from
speakers’ gestures, with much of the work focusing on processing iconic gestures (see
Hostetter, 2011, for a review). Early findings suggest that analogical gestures also convey
information and may have important consequences for reasoning. For example, Jamalian
K. Cooperrider, S. Goldin-Meadow / Topics in Cognitive Science 9 (2017)
731
and Tversky (2012) found that observing temporal gestures that represented phenomena
as either circular or linear, or that depicted events as happening sequentially or in parallel, led participants to reason about these phenomena in correspondingly different ways.
Listeners have also been found to process analogical gestures that represent aspects of
sound spatially. For instance, when people judge the pitch of a sung note, their ability to
do so is biased if the singer also produces gestures that represent pitch vertically (Connell, Cai, & Holler, 2013). Relatedly, gestures that represent pitch contours (e.g., a falling
tone represented with a gesture moving rightward and downward) can help English
speakers learn Mandarin words (Morett & Chang, 2014).5
In several studies on metaphor processing, Gentner has shown that people are better at
processing incoming information when it is consistent with a mapping already established
in the discourse (e.g., Gentner, 2001; Gentner, Bowdle, Wolff, & Boronat, 2001; see also
Thibodeau & Durgin, 2008). This phenomenon has been called the “metaphor consistency
effect.” For example, after reading sentences involving one of two widespread temporal
models used by English speakers—either the ego-moving model (“We are approaching the
deadline”) or the time-moving model (“The deadline is approaching us”)—subjects were
slower to process a target sentence that switched to the other model, compared to a target
that maintained the initial model. We should expect a similar processing cost in listeners
when a speaker switches between analogical models in gesture. Interestingly, note that such
a processing cost could, in some cases, be beneficial in the long term. In an educational context, being exposed to multiple models for the same concept might slow the learner down,
but in the end lead the learner to new insights (i.e., desirable difficulties; see Bjork & Bjork,
2014), whether by implicitly comparing models or some other mechanism.
Good analogies relate a base and a target in a systematic, structurally consistent way
(Gentner, 1983; Gentner & Markman, 1997). For instance, there must be a one-to-one correspondence between elements in the base representation and elements in the target representation. Violations of structural consistency in analogical gestures should impede processing,
or lead to erroneous inferences. Consider the analogical model of positive feedback discussed earlier (Fig. 4) in which the speaker assigns a factor to each hand and shows a series
of increases in those factors. Once the assignment of factors to hands has been made, showing a movement of the wrong hand would be a violation of structural consistency. Interestingly, there is already evidence that adults—and even young children—track the locations
of entities assigned to abstract locations in gesture space (Gunter, Weinbrenner, & Holle,
2015; Smith & Kam, 2015). A question for future work is how violations of structural consistency impact the processing of extended analogical models.
5.2. A relational shift in gesture?
A major focus of Genter’s research has been analogical thinking in learning and development. Early on, she noted a “relational shift” that plays out in children’s learning (e.g.,
Gentner, 1988; Gentner & Rattermann, 1991). When first learning about a given domain,
children focus on objects and attributes, shifting to a focus on relations as they learn
more. An example of this shift is seen in children’s preferences for—and interpretations
732
K. Cooperrider, S. Goldin-Meadow / Topics in Cognitive Science 9 (2017)
of—metaphors. Young children judge metaphors that focus on attributes (as in the earlier
“The sun is a grapefruit” example) as more apt than metaphors that focus on relations (as
in the earlier “That job is a jail” example). Some metaphors allow either kind of interpretation, one based on shared attributes between the base and target attributes, and one
based on shared relations (e.g., “Tires are shoes”). Interestingly, in cases of these “double
metaphors,” children prefer an interpretation based on attributes, whereas adults prefer
relational interpretations (Gentner, 1988; Gentner & Clement, 1988). We might predict
that these patterns would extend to gestures. Children might begin by producing metaphoric gestures that highlight attributes and only later come to produce metaphoric gestures
that highlight relations—that is, analogical gestures. In fact, we might also make this prediction for iconic gestures, that is, that children would first produce gestures that depict
attributes of objects and actions, and only later purely relational iconics. The same developmental shift could also be seen in the ability to process gestures. Young children may
not be able to make sense of the analogical gestures we have described, and when presented with abstract gestures that represent both attributes and relations, they might key
on attributes and ignore the relational information.
A similar relational shift may be observed in the acquisition of expertise (Rottman
et al., 2012) and even in the history of science (Gentner & Jeziorski, 1993). Indeed, the
relational shift is not driven by development per se, but by learning about a new domain.
Thus, in the right domain, it might be possible to observe an increased use of analogical
gestures (and relational iconics) among adult experts compared to adult novices. In fact,
Gentner and colleagues have begun to examine evidence for a relational shift in another
visuospatial medium—sketching. Jee et al. (2014) found that, relative to novices, geoscience experts included more relational symbols such as arrows in their sketches.
5.3. Learning
Relations are important for how we understand the world but are, at their core, nonobvious. Gentner (2003, 2010) has proposed that language plays a critical role in highlighting relations so that we can notice them, remember them, and reason about them.
One way this happens is that relational words like middle and barrier invite learners to
compare instances where such labels are used, and this process of comparison leads
learners to form new, abstract knowledge. Could relational gestures—whether iconic or
analogical—play the same kind of role in highlighting relations for learners? And how
would the role of gesture compare to the role of language? As our examples have shown,
analogical gestures make use of a limited vocabulary for representing relational structure
as spatial structure—entities are locations, conceptual dimensions are ordered spatial
arrangements, changes are movements, and so on.6 Seeing gestures that use this same
restricted vocabulary for disparate phenomena may lead learners to compare those
phenomena and pull out valuable abstractions.
One case where the power of relation-highlighting gestures has already been demonstrated is in learning about mathematical equivalence (Goldin-Meadow, 2003), a principle
involved in solving problems such as: 4 + 3 + 6 = ____ + 6. An effective solution
K. Cooperrider, S. Goldin-Meadow / Topics in Cognitive Science 9 (2017)
733
strategy is to group the first two addends (4 and 3) and put the sum into the blank. Such
a strategy can be demonstrated in gesture by using a “V” hand shape to simultaneously
point to the 4 and 3—thus highlighting the relation of grouping—and then pointing to the
blank. In fact, learners benefit from this grouping gesture even when the fingers of the
“V” point to the wrong numbers, suggesting that highlighting a relation helps over and
above highlighting particular numbers or locations (Goldin-Meadow, Cook, & Mitchell,
2009). Highlighting the grouping relation in this way also helps children generalize the
strategy to new types of equivalence problems (Novack, Congdon, Hemani-Lopez, &
Goldin-Meadow, 2014). Novack et al. (2014) compared the effects of the relational
grouping gesture just described to a more action-like version of the gesture in which the
learner mimics picking up the two addends. Note that both versions of the gesture include
relational information about grouping, but the action-like gesture also represents potentially distracting action attributes, such as a grasping hand shape. The key finding was
that the children who learned grouping through the purely relational gesture performed
better on transfer problems.
An important caveat is that, before learners can benefit from relational gestures, they
must first understand how those gestures represent relational structure. In the case of the
grouping gesture just described, the relation may be relatively accessible because the entities
that are related—numbers on a whiteboard—are physically present in the speech situation.
In many of the analogical gestures we have considered, however, the entities are not only
not present in the speech situation, but they also have no concrete manifestation as entities.
If a learner fails to understand the mappings involved in an analogical model expressed in
gesture, that model may be more of a stumbling block than a stepping stone. Analogical gestures may be a powerful tool, but only when deployed at the right time, for the right learner.
Indeed, emerging evidence suggests that gesture sometimes helps and sometimes hinders,
depending on the specifics of the gesture, the situation, and the learner (Congdon, 2016;
Goldin-Meadow & Beilock, 2010; Post, Van Gog, Paas, & Zwaan, 2013).
6. Conclusion
Analogies are happening all around us. In lecture halls, courtrooms, laboratory meetings,
and television interviews, they are being proposed, elaborated, and evaluated. Much prior
work has focused on the analogies that show up in language. But they also show up in
another medium: the gestures people produce when they talk. This fact has two important
implications. One is that there is a naturalistic and omnipresent source of data on analogical
reasoning and thinking that is going untapped by researchers interested in analogy. Looking
closely at how analogies manifest in gesture may offer fresh insights into the analogical
mind, insights that might be hard to glean from language alone. Another implication is that,
just as analogy researchers may be missing out on an illuminating source of evidence, gesture researchers may be missing out on an illuminating theoretical framework, one that has
been developed and fine-tuned over decades by Dedre Gentner and her colleagues. In short,
gesture can inform our understanding of analogy, and vice versa.
734
K. Cooperrider, S. Goldin-Meadow / Topics in Cognitive Science 9 (2017)
Acknowledgments
We thank Dedre Gentner for illuminating discussions of these ideas and for inspiring
our paper, as well as Spencer Kelly and two anonymous reviewers for comments on an
earlier draft. We gratefully acknowledge funding from the NSF Spatial Intelligence and
Learning Center (SBE 0541957) and NICHD (R01-HD47450).
Notes
1. In both Taub’s (2001) and Emmorey’s (2014) treatments, metaphorical iconicity in
sign language is said to involve a “double mapping.” The first mapping is between
some abstract idea and a visuospatial mental representation; the second mapping is
between that visuospatial mental representation and the bodily representation. For
simplicity, we will treat this as a single mapping between an abstract idea and its
gestural representation.
2. It can be misleading to try to judge the abstractness of a gesture by its co-occurring
speech. For instance, if a speaker produced an attribute metaphor in speech, “The
sun was a grapefruit,” while representing an imagined spherical object between the
hands, this would be an iconic gesture because it represents an actual visuospatial
attribute of the sun—its spherical shape. Even relational metaphors—that is, analogies—in speech do not necessarily go with gestures that represent relations. If a
speaker said, “This job is a jail,” with a gesture representing someone holding
prison bars, the gesture would be representing attributes—what it looks like to hold
prison bars—and not the relation of confinement more generally. Of course, to get
the intended meaning, it is necessary to identify common relational structure
between job and jail: The job is confining to the person who has it, just as a jail is
confining to a prisoner. But that is not what the gesture represents. In what we are
considering analogical gestures, relations themselves are given visuospatial representation, and the speech may or may not be obviously metaphorical.
3. A possible interpretation of Lagarde’s gesture is that it is inspired by the image of
a balance scale, with parts that move up and down in complementary fashion. Such
a motivation would not make the gesture iconic—however, she is not representing
actual movements in space but abstract changes. It is nonetheless interesting to note
that some analogical gestures may be inspired by ready-made concrete images such
as a balance scale, whereas others may be cobbled together on the spot.
4. Etymologically, the phrase vice versa involves a spatial analogy—the Latin means
“in-turned position”—but this is probably opaque to most English speakers.
5. Not all analogical gestures representing properties of sound have been found to
have such consequences. For instance, Kelly, Hirata, Manansala, and Huang (2014)
found that gestures representing vowel length contrasts did not benefit English
speakers’ learning of Japanese words.
K. Cooperrider, S. Goldin-Meadow / Topics in Cognitive Science 9 (2017)
735
6. Interestingly, many of the components of this basic vocabulary also show up in
everyday language (Lakoff & Johnson, 1980) and in diagrams (Tversky, 2011).
References
Bjork, E. L., & Bjork, R. A. (2014). Making things hard on yourself, but in a good way: Creating desirable
difficulties to enhance learning. In M. A. Gernsbacher & J. Pomerantz (Eds.), Psychology and the real
world: Essays illustrating fundamental contributions to society (2nd ed., pp. 59–68). New York: Worth.
Cienki, A. (1998). Metaphoric gestures and some of their relations to verbal metaphorical expressions. In
J.-P. Koenig (Ed.), Discourse and cognition: Bridging the gap (pp. 189–204). Stanford, CA: Center for
the Study of Language and Information.
Cienki, A., & M€uller, C. (Eds.) (2008). Metaphor and gesture. Philadelphia, PA: John Benjamins.
Clark, H. H. (2016). Depicting as a method of communication. Psychological Review, 123(3), 324–347.
Congdon, E. (2016). Learning mathematics through action and gesture: Children’s prior knowledge matters
(Unpublished doctoral dissertation). Chicago, IL: University of Chicago.
Connell, L., Cai, Z. G., & Holler, J. (2013). Do you see what I’m singing? Visuospatial movement biases
pitch perception. Brain and Cognition, 81(1), 124–130.
Cooperrider, K., Gentner, D., & Goldin-Meadow, S. (2016). Spatial analogies pervade complex relational
reasoning: Evidence from spontaneous gestures. Cognitive Research: Principles and Implications, 1, 28.
https://doi.org/10.1186/s41235-016-0024-5.
Cooperrider, K., & N
u~
nez, R. E. (2009). Across time, across the body: Transversal temporal gestures.
Gesture, 9(2), 181–2006.
Cooperrider, K., N
u~nez, R., & Sweetser, E. (2014). The conceptualization of time in gesture. In C. M€
uller,
A. Cienki, E. Fricke, S. Ladewig, D. McNeill, & J. Bressem (Eds.), Body-language-communication (vol.
2, pp. 1781–1788). New York: Mouton.
Doumas, L. A. A., Hummel, J. E., & Sandhofer, C. M. (2008). A theory of the discovery and predication of
relational concepts. Psychological Review, 115, 1–43.
Efron, D. (1972). Gesture, race, and culture. The Hague: Mouton.
Emmorey, K. (2002). Language, cognition, and the brain: Insights from sign language research. Mahwah,
NJ: Lawrence Erlbaum.
Emmorey, K. (2014). Iconicity as structure mapping. Philosophical Transactions of the Royal Society B, 369,
20130301.
Emmorey, K., Tversky, B., & Taylor, H. A. (2000). Using space to describe space: Perspective in speech,
sign, and gesture. Spatial Cognition & Computation, 2, 157–180.
Enfield, N. J. (2005). The body as a cognitive artifact in kinship representations. Current Anthropology, 46
(1), 51–81.
Gentner, D. (1983). Structure-mapping: A theoretical framework for analogy. Cognitive Science, 7, 155–170.
Gentner, D. (1988). Metaphor as structure mapping: The relational shift. Child Development, 59, 47–59.
Gentner, D. (2001). Spatial metaphors in temporal reasoning. In M. Gattis (Ed.), Spatial schemas in abstract
thought (pp. 203–222). Cambridge, MA: MIT Press.
Gentner, D. (2003). Why we’re so smart. In D. Gentner & S. Goldin-Meadow (Eds.), Language in mind:
Advances in the study of language and thought (pp. 195–235). Cambridge, MA: MIT Press.
Gentner, D. (2010). Bootstrapping the mind: Analogical processes and symbol systems. Cognitive Science, 34
(5), 752–775.
Gentner, D., Bowdle, B. F., Wolff, P., & Boronat, C. (2001). Metaphor is like analogy. In D. Gentner, K. J.
Holyoak, & B. N. Kokinov (Eds.), The analogical mind: Perspectives from cognitive science (pp. 199–
253). Cambridge, MA: MIT Press.
736
K. Cooperrider, S. Goldin-Meadow / Topics in Cognitive Science 9 (2017)
Gentner, D., & Clement, C. (1988). Evidence for relational selectivity in the interpretation of analogy and
metaphor. In G. H. Bower (Ed.), The psychology of learning and motivation: Advances in research and
theory (vol. 22, pp. 307–358). New York: Academic Press.
Gentner, D., & Jeziorski, M. (1993). The shift from metaphor to analogy in Western science. In A. Ortony
(Ed.), Metaphor and thought (pp. 447–480). Cambridge, UK: Cambridge University Press.
Gentner, D., & Markman, A. B. (1997). Structure mapping in analogy and similarity. American Psychologist,
52(1), 45–56.
Gentner, D., & Rattermann, M. J. (1991). Language and the career of similarity. In S. A. Gelman & J. P.
Byrnes (Eds.), Perspectives on thought and language: Interrelations in development (pp. 225–277).
London: Cambridge University Press.
Goldin-Meadow, S. (2003). Hearing gesture: How our hands help us think. Cambridge, MA: Harvard
University Press.
Goldin-Meadow, S., & Beilock, S. L. (2010). Action’s influence on thought: The case of gesture.
Perspectives on Psychological Science, 5(6), 664–674.
Goldin-Meadow, S., Cook, S. W., & Mitchell, Z. A. (2009). Gesturing gives children new ideas about math.
Psychological Science, 20(3), 267–272.
Goldstone, R. L., Medin, D. L., & Gentner, D. (1990). Similarity involving attributes and relations:
Judgments of similarity and difference are not inverses. Psychological Science, 1, 64–69.
Goldwater, M. B., & Gentner, D. (2015). On the acquisition of abstract knowledge: Structural alignment and
explication in learning causal system categories. Cognition, 137, 137–153.
Gunter, T. C., Weinbrenner, J. E. D., & Holle, H. (2015). Inconsistent use of gesture space during abstract
pointing impairs language comprehension. Frontiers in Psychology, 6(February), 1–10. https://doi.org/doi.
org/10.3389/fpsyg.2015.00080.
Hofstadter, D., & Sander, E. (2013). Surfaces and essences: Analogy as the fuel and fire of thinking. New
York: Basic Books.
Holyoak, K. J., & Thagard, P. (1995). Mental leaps: Analogy in creative thought. Cambridge, MA: MIT
Press.
Hostetter, A. B. (2011). When do gestures communicate? A meta-analysis. Psychological Bulletin, 137(2),
297–315.
Jamalian, A., & Tversky, B. (2012). Gestures alter thinking about time. In N. Miyake, D. Peebles, & R. P.
Cooper (Eds.), Proceedings of the 34th Annual Cognitive Science Society (pp. 503–508). Austin, TX:
Cognitive Science Society.
Jee, B. D., Gentner, D., Uttal, D. H., Sageman, B., Forbus, K., Manduca, C., Ormand, C., Shipley, T., &
Tikoff, B. (2014). Drawing on experience: How domain knowledge is reflected in sketches of scientific
structures and processes. Research in Science Education, 44(6), 859–883.
Kelly, S. D., Hirata, Y., Manansala, M., & Huang, J. (2014). Exploring the role of hand gestures in learning
novel phoneme contrasts and vocabulary in a second language. Frontiers in Psychology, 5(July), 1–11.
https://doi.org/10.3389/fpsyg.2014.00673.
Kendon, A. (2004). Gesture: Visible action as utterance. Cambridge, UK: Cambridge University Press.
Lakoff, G., & Johnson, M. (1980). Metaphors we live by. Chicago, IL: University of Chicago Press.
Lempert, M. (2011). Barack Obama, being sharp: Indexical order in the pragmatics of precision-grip gesture.
Gesture, 11(3), 241–270.
Liddell, S. K. (1995). Real, surrogate, and token space: Grammatical consequences in ASL. In K. Emmorey
& J. S. Reilly (Eds.), Language, gesture, and space (pp. 19–41). Hillsdale, NJ: Lawrence Erlbaum.
Markman, A. B. (1997). Constraints on analogical inference. Cognitive Science, 21(4), 373–418.
McNeill, D. (1992). Hand and mind: What gestures reveal about thought. Chicago, IL: Chicago University
Press.
McNeill, D. (2005). Gesture and thought. Chicago, IL: Chicago University Press.
Meir, I. (2010). Iconicity and metaphor: Constraints on metaphorical extension of iconic forms. Language, 86
(4), 865–896.
K. Cooperrider, S. Goldin-Meadow / Topics in Cognitive Science 9 (2017)
737
Mittelberg, I. (2008). Peircean semiotics meets conceptual metaphor: Iconic modes in gestural representations
of grammar. In A. Cienki & C. M€uller (Eds.), Metaphor and gesture (pp. 115–154). Philadelphia, PA:
John Benjamins.
Morett, L. M., & Chang, L.-Y. (2014). Emphasising sound and meaning: Pitch gestures enhance Mandarin
lexical tone acquisition. Language, Cognition and Neuroscience, 30(3), 347–353.
Novack, M. A., Congdon, E. L., Hemani-Lopez, N., & Goldin-Meadow, S. (2014). From action to
abstraction: Using the hands to learn math. Psychological Science, 25(4), 903–910.
Parrill, F., & Sweetser, E. (2004). What we mean by meaning: Conceptual integration in gesture analysis and
transcription. Gesture, 4(2), 197–219.
€ urek, A. (2015). Visible cohesion: A Comparison of reference tracking in sign, speech, and
Perniss, P., & Ozy€
co-speech gesture. Topics in Cognitive Science, 7(1), 36–60.
Post, L. S., Van Gog, T., Paas, F., & Zwaan, R. A. (2013). Effects of simultaneously observing and making
gestures while studying grammar animations on cognitive load and learning. Computers in Human
Behavior, 29(4), 1450–1455.
Rottman, B. M., Gentner, D., & Goldwater, M. B. (2012). Causal systems categories: Differences in novice
and expert categorization of causal phenomena. Cognitive Science, 36(5), 919–932.
Smith, W. G., & Kam, C. L. H. (2015). Children’s use of gesture in ambiguous pronoun interpretation.
Journal of Child Language, 42(3), 591–617.
So, W. C., Kita, S., & Goldin-Meadow, S. (2009). Using the hands to identify who does what to whom:
Gesture and speech go hand-in-hand. Cognitive Science, 33(1), 115.
Taub, S. (2001). Language from the body. New York: Cambridge University Press.
Thibodeau, P., & Durgin, F. H. (2008). Productive figurative communication: Conventional metaphors
facilitate the comprehension of related novel metaphors. Journal of Memory and Language, 58, 521–540.
Tversky, B. (2011). Visualizing thought. Topics in Cognitive Science, 3(3), 499–535.
Winter, B., Perlman, M., & Matlock, T. (2013). Using space to talk and gesture about numbers: Evidence
from the TV News Archive. Gesture, 13(3), 377–408.
Appendix A: Links to gesture clips
Fig. 1. Sean Carroll in a lecture (see 0:15 to 0:29): https://www.youtube.com/watch?v=
rEr-t17m2Fo
Fig. 2A. Christine Lagarde in a television interview: https://archive.org/details/BLOOM
BERG_20140412_000000_Charlie_Rose#start/1500/end/1560
Fig. 2B. James Steinberg in a television interview: https://archive.org/details/KRCB_
20110625_060000_Charlie_Rose#start/448.5/end/470.1
Fig. 3A–C. Peter Slen in a television interview: https://archive.org/details/CSPAN2_
20141013_053500_Book_Discussion#start/550/end/560.7
Fig. 3D–E. Steven Pinker in a lecture: https://archive.org/details/CSPAN2_20141225_
160600_Key_Capitol_Hill_Hearings#start/237.7/end/248
Hands as molecules: Representational
gestures used for developing theory
in a scientific laboratory
L. AMAYA BECVAR, JAMES HOLLAN, and EDWIN HUTCHINS
Abstract
This research examines how representational gestures (Kita 2000) made by
scientists during collaborative discussions in a biochemistry lab, are used as
cognitive artifacts for formulating scientific theory. By analyzing digital
video of lab meetings and interviews with scientists, we find that representational gestures serve to reference and animate portions of existing material
structure such as models, diagrams, and graphs. Representational gestures
appear to play a significant role in how the scientific group both conceptualize and communicate theories about molecules. We believe that representational gestures operate as instantiations of essential spatiodynamic features that are not e‰ciently conveyed in other modalities, like language
and graphical representations, and as such, are vital resources for shaping
theoretical understandings in collaborative, face-to-face scientific activity.
Gestures may also serve to package theoretical conceptions into semiotic
entities that can be used symbolically in the ‘community of practice’ (Lave
1991). We use a theoretical framework provided by distributed cognition
and embodied cognition to examine communal representational gestures as
cognitive artifacts produced and modified by the biochemistry lab community during instances where theories are being negotiated.
1. Introduction
Seeing Dalton’s wooden models of molecules
in the science museum put me into a sort
of rapture, conceiving that the chemical reactions one saw macroscopically in the lab . . .
were the consequence of activities infinitely
small — single atoms, with distinctive weights
and characters, combining with each other
one by one. Until then, there had only been a
Semiotica 156–1/4 (2005), 89–112
0037–1998/05/0156–0089
6 Walter de Gruyter
90
L. Amaya Becvar et al.
vague, mysterious sense of an invisible microworld; Dalton’s models gave the imagination
something concrete to chew on, made this tangible and real.
— Oliver Sacks (1999: 122)
Throughout its history, chemistry has been a remarkably visual science.
However, the physics of the molecular world is fundamentally di¤erent
than the physics of the macromolecular world where we live and operate.
In reality humans have very few perceptual resources for directly relating
to molecules. Even the images produced by the most powerful scanning
tunneling electron microscopes give a misleading impression of molecules.
Molecular models allow chemists to use their perceptual resources to help
generate and test chemical theories.
Molecular models are essential cognitive tools for chemists. These
models range from physical ‘tinker-toy’ modeling kits, to skeletal structural drawings, to three-dimensional computer models. Historical analysis indicates that the materiality of molecular models has played a central
role in the development of chemistry (Francoeur 2000; Brown 2003). In a
similar vein, a large body of research on the use of graphical representations in scientific practice in general indicates that these physical images
are integral to the production of scientific facts (Fleck 1979; Latour
1990; Lynch and Woolgar 1990; Ochs et al. 1991; Goodwin 1995). That
is, theoretical and empirical results are not only communicated, but also
enabled by the visible, tangible molecular models that they construct and
manipulate.
Consider the following segment of discourse, taken from the lab meeting of a biochemistry research group in which a year-long ethnographic
research study was conducted1:
1.1. (00:37:04;00)
B: fAnd so (0.3) our new
theory is (0.5) that (1.1)
1.2. (00:37:08;00)
thrombomodulin
does something like this (0.5)
1.3. (00:37:11;30)
or like} {this (1.5) okay
(0.3).}
Representational gestures in a scientific laboratory
91
In this segment, the researcher B is describing a theoretical account of
how a protein, thrombin, moves when it binds to another protein, thrombomodulin. She describes this motion (‘something like this’) through gesture. Note how in this moment, the researcher’s own hand has become a
model of the shape and movement of the molecule, thrombin. (This segment will be analyzed more thoroughly in section 1.2).
In this paper, we examine the role of representations of molecules
as cognitive artifacts used during real time collaborative interactions of
biochemists. In our analysis, we extend the concept of ‘cognitive artifact’
to include other representational forms besides physical objects and
graphical representations. In particular, we focus on how representational
gestures, coupled to molecular models, are used in scientific activity to develop theories about molecules in a biochemistry lab. Exploring gesturein-interaction permits a glimpse into the schematic ways scientists in this
lab understand the behavior of protein molecules. We use cognitive ethnography to document and examine the interactions of a scientific ‘community of practice’ (Lave 1991). These interactions involve biochemists
during laboratory meetings and interviews. As scientists talk and gesture
about theoretical entities like proteins, molecules, and chemical bonds, we
can attempt to understand the conceptual structures that underlie their interactions, and track their development through time, documenting the
role played by gesture and embodied physical experience.
This research calls upon the theoretical frameworks provided by distributed cognition (Hutchins and Palen 1997; Hutchins 1995) and embodied cognition (Johnson 1987; Clark 1997; Nunez 1999; Varela et al. 1991).
Distributed cognition expands the unit of analysis for cognition beyond
individual brains to include bodies, material structures, and social contexts of cognitive activity, and provides a framework for examining the
propagation of information through representational forms, such as
spoken language, gesture, graphical models, and text.
A theory of embodied cognition (Johnson 1987; Clark 1997; Nunez
1999; Varela et al. 1991) holds that our conceptual understandings, derived through our bodily experience of the world, both enable and constrain our ability to think about and represent abstract phenomena (in
this case, how scientists understand entities that cannot be directly perceived, like molecules). We use the term ‘embodied’ to refer to representational forms (such as gestures, utterances, and inscriptions), and highlight
how these representations are organized by conceptual systems grounded
in the experienced world.
92
L. Amaya Becvar et al.
1.1.
Cognitive artifacts and representations
Cognitive artifacts are tools used for aiding, enhancing, or improving human cognition (Hutchins 1996). Close investigation of artifacts and their
use can provide insight into the meaningful cognitive practices of a community. Cognitive artifacts support reasoning processes and transform
cognitive tasks. By using cognitive artifacts we can abstract and represent
information in such a way as to replace cognitively challenging tasks, such
as mental arithmetic, memorization, complex simulations, by ‘cognitively
robust perceptual processes’ (Hutchins 1996: 1). We can represent perceptions, experiences, and thoughts in media other than which they originally
occurred (Norman 1990), and perform complex transformations in media
that allow us to make full use of our powerful perceptual skills. Research
has shown that the nature of the representation determines how a problem might be conceptualized, and that certain types of representations
render problems more amenable to human cognitive abilities (Rumelhart
1980; Kirsh 1995; Goldstone and Barsalou 1998). Di¤erent representations of a problem can have a dramatic impact on problem di‰culty
even if the formal structures are the same (Zhang and Norman 1994). In
this light, we should look at the role of representational artifacts when
attempting to understand the reasoning processes of scientists as they
analyze empirical data and produce theories. This analysis should include
not only static representations, such as data tables, diagrams, and graphs,
but also the more ‘ephemeral’ representational forms expressed in human
interaction (i.e., gesture and speech).
1.2.
Gestures
Human conversation involves a rich interaction among multiple modalities of verbal and non-verbal communication. Recently, there has been
a growing interest in gesture studies (McNeill 1992; Kendon and Muller
2001), brought about in large part by a change in the theoretical foundations of many of disciplines that traditionally viewed gesture as peripheral
and incidental to spoken language. Recent work on gesture has begun to
elucidate the role played by bodily engagement with the world through
basic practical actions, which provide structure to cognition and conceptual development (see Roth 2000, for a review). This work is largely influenced by the theory of embodied cognition. For example, it appears that
many gestures are derived from manipulating real objects and making
practical actions. However, gestures are also tied to verbal utterances
that express very abstract ideas. Analysis of gestural expressions promises
Representational gestures in a scientific laboratory
93
to tell us much about the way cognition is grounded in our engagement
with the physical environment (Nunez 1999; McNeill 2000; Parrill 2000).
Our research focuses on the use of representational gestures in collaborative scientific discourse. Representational gestures (Kita 2000)
bear meaningful relationship to the semantic content of the speech they
accompany.
2. Methods
2.1.
Cognitive ethnography
Over the past decade in the field of cognitive science, we have seen an increasing interest in the role of the material and social world in cognitive
processes. We are now beginning to recognize that patterns in the material and social world play a crucial role in the construction of cognitive
task demands, and that many cognitive processes extend beyond the individual and are played out in interaction with the environment (Goodwin
1994). Currently, such questions are being explored in the fields of distributed cognition (Hutchins 1995) and embodied cognition (Clark 1997).
These approaches emphasize the importance of examining human cognition in naturalistic settings (i.e., ‘in the wild’) (Hutchins 1995).
Through our work we are developing a method that we call cognitive
ethnography. A primary goal of this research is to better understand realworld cognitive processes, which include interactions between people as
well as interactions with resources in the environment. As a result, the human body and the material world take on central, rather than peripheral
roles in the cognitive system. Fine-grained analysis of real-world behavior
allows us to glean evidence about the nature of cognitive processes as
they are enacted in naturally-situated activity. A cognitive ethnographer
typically makes recordings (audio, video, photographic) of ongoing activity. The widespread availability of inexpensive digital recording and storage devices now enables for us to go about easily collecting ethnographic
data on real-world activity for review and analysis.
2.2.
Ethnographic context
To provide domain-specific background to the videotaped interactions
analyzed in this research, here we summarize some of the scientific concepts studied in the biochemistry lab field site.
94
L. Amaya Becvar et al.
Figure 1. Ribbon diagram of a protein, from Banner (2000)
2.2.1. Proteins. Proteins are large molecules that fold into intricate
three-dimensional structures, which are unique to each protein. It is this
three-dimensional structure that a¤ords proteins many of their diverse
functions. In biochemistry, typically an intimate structure-function relationship is assumed, and so research done to reveal protein structure often
is used to speculate about the way proteins function in living systems. The
laboratory in which this ethnographic analysis was performed also believes that to understand protein function, they must understand protein
dynamics as well. Much of their research is aimed at characterizing not
only the structure of proteins, but the internal motion of proteins.
Biochemists have devised a number of methods to show the threedimensional structure of proteins. In order to fully explore protein structure in detail, scientists use di¤erent types of molecular models, including
‘ribbon diagrams,’ ‘cartoon’ views, and three-dimensional computational
models. These representations range from complex models, in which
every atom of the structure is shown, to the simpler ‘ribbon diagrams’
that trace the position of the amino acids in three-dimensional space
(Figure 1). A ribbon diagram is a skeletal model of a protein. More schematic representations like the ribbon diagram omit detail in order to
highlight specific aspects of structure such as helical components and
loops.
2.2.2. The biochemistry of blood clotting. The biochemistry group participating in this study primarily does research on the protein thrombin,
which is involved in shutting down the process of blood clotting (see Banner 2000 for a review). Thrombin is an enzyme, one of a family of proteins that chemically cleaves other proteins. This cleavage takes place in
the enzyme’s active site, a cavity exclusively structured for particular substrates to fit.
Representational gestures in a scientific laboratory
95
One of thrombin’s roles is to activate the protein fibrinogen, which
forms mesh-like clumps that make up blood clots. When a body forms
aberrant blood clots, it produces another protein, called thrombomodulin.
Thrombomodulin binds to thrombin in a site distal to the active site.
When this happens, a change occurs in thrombin that causes thrombin
to stop binding fibrinogen into its active site and instead bind another
protein, Protein C.
a)
b)
Thrombin þ thrombomodulin ! active Thrombin
active Thrombin þ Protein C ! active Protein C
When activated, Protein C breaks down blood clots. So depending on
whether thrombin accepts fibrinogen or Protein C into its active site determines whether blood clots are being formed or broken down in the
body. One aim of this research group is to determine how the binding of
thrombomodulin to thrombin causes thrombin to accept Protein C, and
not fibrinogen, into its active site. This research is aimed towards designing therapeutic drugs for heart attacks and strokes, diseases that are
caused by aberrant blood clots. The laboratory is particularly interested
in what happens between thrombomodulin and thrombin, which causes
thrombin to bind Protein C.
3. Results and analysis
For the biochemistry research group, understanding the internal dynamic
properties of protein molecules is essential for characterizing how proteins interact with each other, and what sorts of changes occur when
they do interact. The example presented here demonstrates how representational gestures, acting as instantiations of embodied schematic understanding, may play an essential role in how scientists both represent and
conceptualize molecules and molecular dynamics, and how these understandings are symbolized in a cognitive system. These gestures have
a complex, interdependent relationship with static molecular models,
spoken language, and conceptual structure.
The following data was obtained during one lab meeting in October
2002, and in a follow up interview that took place in April 2003. During
the lab meeting, ‘C,’ an advanced graduate student, presented at the lab
meeting. ‘B’ is the research advisor of the lab. ‘S’ and ‘J’ are graduate students in the lab. During this meeting, C showed new data that several
members of the lab had not yet seen. The following microanalysis was
taken from a six minute section of video from the lab meeting, and a
three second section of the follow-up interview. Only select sections are
96
L. Amaya Becvar et al.
discussed and presented in the accompanying transcripts.2 Though the incident we describe is limited in time and scope, the processes we describe
should be considered to be much more ubiquitous in both scientific domains, and human interaction in general.
3.1.
Embodying a molecule
2.1. (37:10;00)
B: 5See how (0.8)
you have 5(0.3)5
all these little loops5(0.3)
2.4. (37:20;10)
in three-dimensional
space they’re like this
2.2. (37:14;10)
this loop55this loop5 (0.6)
5
this loop5 and 5this loop5
(0.8)
5
2.3. (37:18;20)
{all kin’ of: (0.3)
2.5. (37:21;00)
(1.1) an’ that’s the 5active site
(0.3)5}
2.6. Model of thrombin
on overhead projector
3.1.1. Segment 2. The segment of video depicted in Segment 2 was
taken near the end of a lab meeting during which C, a graduate student
had presented experimental measurements taken on thrombin that show
di¤erences before and after thrombomodulin binds to it. After C presented the data, the PI of the lab, B rose from her chair. In Segment 2, B
stands beside an overhead projector showing a graphical model of thrombin (frame 2.6).
Using a combination of indexical language and gesture (frames 2.1 and
2.2), B points out four distinct ‘loops’ surrounding the active site of
thrombin. She uses the projected shadow of her left hand over the overhead projector to annotate the graphical model. After highlighting four
loops, B cups her left palm with fingers rigidly extended, and lays it next
to the protein model on the overhead. Onscreen, the shadow of her fingers and thumb lie near to the previously referenced ‘four loops.’ B twists
Representational gestures in a scientific laboratory
97
her wrist in an awkward angle, preserving both the orientation (pointing
towards her left) and morphological similarity between her fingers and
the four loops (rounded projections pointing away from a globular central body). The mapping between fingers and loops is drawn through
both iconic similarity and the hand’s close physical proximity to the
model.
After indexing the model and laying her hand next to it, B virtually
‘lifts’ the thrombin model o¤ the transparency, and by embodying it
in her hand, she presents it, palm outward to the group, stating (frame
2.4), ‘In three-dimensional space they’re like this.’ B’s utterance draws
attention to one of the essential features of her hand that is not a feature
of the graphical projection of thrombin: namely, her hand’s ‘threedimensionality.’ In this moment, we see the transformation from indexical gesture (hand pointing to referent in meaningful space) to iconic gesture (hand as referent). At the same time B transforms a two-dimensional
representation of the molecule thrombin into a three-dimensional metarepresentation (Norman 1990).
In the next frame (2.5), B uses both speech and gesture to index another feature of the thrombin model embodied in her left hand — the
active site. By concurrently saying ‘and that’s the active site,’ while pointing at the palm of her left hand, B indexes a portion of the thrombin
model relative to her hand through both deictic noun (‘‘that’s’’) and coincidental indexical gesture (right hand pointing). Note how the use of
‘that,’ commonly glossed as ‘distal relative to speaker’ (as opposed to
‘this’ which is proximate), is used to refer to her own hand. By using
‘that’ and directing the palm outward towards the group, B appears to
be characterizing herself as in an equivalent position vis-à-vis the target
as her recipients, making the ‘hand-as-molecule’ an object to which everyone has equal access (Sidnell 2004, personal communication).
B further builds upon the iconic mapping between the thrombin model
and her hand. The mapping relates essential morphological features of
both the active site of thrombin and her concave palm, both of which
are cavities surrounded on all sides by loop/finger projections. Notice
how the gesture preserves not only the topology and shape between both
the loops and fingers, and the active site and palm, but also the relative
orientations of the loops and active site (fingers surrounding palm, projecting outwards). B’s hands remain configured in a fashion that can be, in
this context, recognized as an index to the model of thrombin. The ‘virtual’ presence of the molecule can be jointly inferred by those in the
room, and it appears that B almost ‘wears’ the model on her hand. Indeed
she holds the hand-as-model sti¿y up and away from her body, engaging
musculature through her entire left shoulder, arm, wrist, and hand.
98
L. Amaya Becvar et al.
Because of the structural a¤ordances of the ‘thrombin hand’ model, B
is able to demonstrate to the group the orientation of the loops in threedimensional space, and their orientation with respect to the active site.
The assignment of relative spatial position in three-dimensions accomplished via B’s gesture is not possible with the flat, two-dimensional projection of the protein model.
3.1.2. Segment 1. (The following discussion refers to data in Segment
1, shown in the Introduction). B prompts the introduction of a theoretical
conjecture by saying ‘our new theory is (0.5) that . . .’ By using the thirdperson possessive noun ‘our,’ she draws attention to the mutual importance of the ‘new theory,’ a theory jointly possessed by all competent
practitioners in the lab. What follows next is a pregnant pause in spoken
discourse (1.1 seconds), while B squeezes her fingers, representing the
loops around thrombin’s active site, slowly in and out. B directs mutual
attention toward the gestural model through a variety of bodily cues.
For instance, her overall body posture — crouched position, head tilted
down while maintaining an intense gaze over the group her orientation
of the gesture — seem to convey the significance of the gesture. She orients her gesturing hand outward towards the group, and shifts her eye
gaze to her hand. These bodily cues, such as orientation and eye gaze,
can function deictically (Scheflen 1976; Gullberg and Holmqvist 1999).
The process that occurs here is comparable to how a state of mutual orientation is negotiated prior to the production of a coherent sentence in
conversation (Goodwin 2000). Furthermore, B uses a slow, intentional
manner when moving her fingers. Levy and Fowler (2000) propose that
gestural intensity, or energy, can indicate that a gesture is carrying new,
significant information content into the stream of discourse.
After producing the squeezing gesture for 1.1 seconds, B states,
‘Thrombomodulin does something like this,’ as she continues the squeezing gesture. Then she says, ‘or like this’ while twisting her fingers around
a central pivot. This is a very complex statement spoken to a community
of experts, and requires a good deal of domain-specific knowledge in order to ‘unpack.’ Thrombomodulin is a binding partner of thrombin, and
one of the aims of this research group is to determine how the binding of
thrombomodulin to thrombin causes thrombin to accept Protein C, and
not fibrinogen, into its active site. In the statement in frames 1.2 and 1.3,
based on domain-specific knowledge gained through ethnographic experience in the laboratory, it appears that what B is implying is that, ‘[The
binding of ] thrombomodulin [to Thrombin] does something like this or
like this [to the active site of thrombin].’
Representational gestures in a scientific laboratory
99
It is interesting that B uses verbal language to refer to elements of the
theory (thrombomodulin) that are not directly indexed in her gesture.
Thrombin, its loops and active site, all symbolized by her hand, by a previous indexicalization, are not nominally referenced by the utterance in
Segment 2, whereas the binding of thrombomodulin does not have a gestural analogue, and is referenced through spoken dialog only. Both verbal
and visual modalities come together in this instant to convey more than
either mode could alone.
Through this gesture, B describes to the group her conception of the
nature of the structural and dynamic changes that occur in thrombin.
Her gesture is no longer an iconic (referencing salient features of referent)
or a meta-representation of the structure of thrombin, as it was during
Segment 2, but instead becomes an instantiation of her conception of the
dynamic state change of the protein, which is in alignment with experimental measurements made in her laboratory. The gesture conveys new
spatiodynamic features not present in the static structural model. B’s prior
iconic mapping of component parts, set up in Segment 2, serves to connect the movement of the loops around the active site of thrombin with
the movement of her fingers in one of two motions, either rotation, or
squeezing inwards. By transforming the static graphical representation
into a hand-model she can do things that cannot be done with the graphic
display because of the specific dynamic and spatial a¤ordances (Norman
1990) of her hand. The gesture relies upon certain visuospatial correspondences between the model and the hand, while at the same time
o¤ering new features to the model. Her gesture introduces spatiodynamic properties, in line with indirect experimental measurements that B
is using to draw inferences about biochemical system.
B’s gestural conception draws upon more than a superficial analogy between the hand and the molecular model. The theoretical inferences B
makes in light of the data that C has presented rely upon a conception
of loops in proteins moving in ways analogous to tangible, dynamic,
three-dimensional objects like fingers. Therefore, B draws on the inferential structure provided by her embodied experiences with tangible objects
in order to formulate conjectures about the dynamic nature of proteins.
The ‘thrombin hand’ gesture exists as a stabilizing structure that juxtaposes indirect experimental measurements, which are numerical, the
graphical model, which is static, and embodied schematic structure derived from the spatiodynamics of tangible objects and hands. Interestingly, drawing inferences in the tangible world allows B to theorize about
what’s going on in the molecular world.
100
L. Amaya Becvar et al.
3.2.
Formation of a gestural symbol
3.1. (00:38:18;00)
3.2. (00:38:19;30)
B: {but when you
{suddenly that Protein C
bind thrombomo::dulin
Inhibitor is in there (0.5) a:
to the back side of thrombin} THOUSAND} FOLD
FASTER: (0.5)
3.3. (00:38:29;10)
{So there’s a lo:tta evidence
suggesting} {that so:mething
like this is going on. (2.0)}
3.2.1. Segment 3. Here B is referring to the experimental evidence from
another research group that corroborates her theory, and the evidence C
presented earlier in the lab meeting, which supports the theory that
thrombomodulin’s binding to thrombin causes a conformational or dynamics change in the loops around the active site of thrombin. It is interesting how B reinstantiates the ‘thrombin hand’ model in her left hand, in
order to describe the molecular details of the experimental findings of the
other research group. She uses her right hand as thrombin’s binding partner, thrombomodulin and later, the Protein C inhibitor molecule, against
the backdrop provided by the ‘thrombin hand’ shape, held stationary for
a full 20 seconds.
In frame 3.1, B then uses the ‘thrombin hand’ gesture in a fundamentally di¤erent way than she has used it previously. Now, the hand shape
and dynamics are used as a symbol, when she says, ‘So there’s a lot of evidence that something like this is going on.’ The deictic noun ‘this’ references the co-speech gesture, which is a blend between the ‘thrombin hand’
squeezing and ‘thrombin hand’ twisting gestures. At this point, speech
provides the syntax needed to support a gesture-based lexical item. The
gesture functions in a form of discourse deixis (Levinson 1983), pointing
back to the moment in prior discourse when B presented the theory of the
dynamic changes in thrombin’s loops. Standing behind the gesture is an
astonishing representational cascade, made up of experimental data,
structural models, and over two minutes of discourse to which the gesture
is anchored.
According to LeBaron and Streeck (2000), a paradigmatic instance of
symbol formation is the situated creation of a form-meaning pair that
Representational gestures in a scientific laboratory
101
embodies a nexus of locally produced, shared knowledge. B’s raised hand,
presented with a recognizable shape and motion, denotes a complex of
actions, theories, objects, and inscriptions, and has become a socially
shared symbol. LeBaron and Streeck (2000) also contend that shared
knowledge of communities grows through the creation, reuse, and transformation of symbolic forms. In the following passages, we see how the
‘thrombin hand’ gesture is now available to be used in various communicative purposes, syntactic contexts, and semantic roles.
3.3.
Propagation of a symbolic gesture
4.1. (00:39:45;00)
C: I mean so: so {there’s two things
going on (0.1)
4.2. (00:39:47;30)
There’s this sorta dynamics (0.3), or
conformational change right in the loops-}
3.3.1. Segment 4. B is not the only lab member who uses the symbolic
gesture in the ongoing discourse to reference the theoretical account of
thrombin. A few minutes later, C, a graduate student, invokes a similar
gesture to B’s in order to elaborate on the theory. C holds her right hand
up facing the group, mimicking the squeezing loop theory gesture as she
speaks ‘there’s some sort of dynamics or (0.3)/conformational change’
(frame 4.1). She uses the indexical speech term ‘this’ to bring the gestural
semiotic term grammatically into the spoken dialog.
C’s gesture mimics many features of B’s gesture. C supports her left elbow with her right hand similar to how B propped her left elbow up on
the conference table. C’s palm faces outwards, towards the center of the
102
L. Amaya Becvar et al.
conference room, and she squeezes her fingers in and out. Still, it is also
interesting how C’s gesture is slightly di¤erent from B’s gesture. C’s palm
faces slightly upwards, while B’s palm faced towards the group. Also,
though C moves her hand in similar way as B has done, her fingers
move with a less structured motion with a quicker frequency. Note the
relative ‘limpness’ of her hand in contrast to B’s left hand in Segment 2.
Levy and Fowler (2000) note that gestures and speech referring to topics
that have already been established are marked by patterns of lesser energy. Despite these di¤erences the gesture is rendered intelligible through
the surrounding context and co-gesture speech.
5.1. (00:42:51;00)
S: {Are these e¤ects, you know
This: thing yer doing (0.3)}
5.2. (00:42:55;30)
Is this- (0.2)
{You think it’s all those steric e¤ects}
3.3.2. Segment 5. In Segment 5, a postdoctoral student in the lab, S,
indexes the loop theory three times (‘these e¤ects’, ‘thi:s (0.5) thing yer
dewing’, ‘those steric e¤ects’); however, he only explicitly uses the symbolic gesture once, making the gesture with his right hand as he speaks
‘you know thi:s (0.5) thing yer dewing.’ It is interesting how he uses the
gesture to self-repair his statement from line 2, ‘Are these e¤ects.’ The
gesture allows him to refer back to a previous moment in the discourse
in a way perhaps more descriptive than he could convey with speech
alone.
Figure 2 is a representation showing the evolution of repeated gestural
forms relating to the theory about the protein thrombin through time in
order to demonstrate the ubiquitous use of the ‘thrombin hand’ gesture
form and its modifications.
Representational gestures in a scientific laboratory
103
Figure 2. Timeline showing the temporal distribution of repeated gesture forms relating to
thrombin for three speakers in a segment of discourse (B, C, and S). Left and right hands
(L.H. and R.H.) are indicated separately. Thick bars represent the presence of repeated gesture forms and are segmented at gestural transitions. Grey bars indicate when the speaker is
invoking the ‘thrombin hand’ gesture form. Stippled bars indicate the opposite hand entering
the palm of the ‘thrombin hand.’ Black bars indicate the opposite hand hitting the backside of
the ‘thrombin hand.’ Speech is indicated by a bold black line above each speaker’s row in the
timeline.
Through the recycling the ‘thrombin hand’ gesture, both the gesture itself and the accompanying indexical dialog have become intersubjectively
recognized discourse elements. The symbolic gesture appeals to community knowledge, knowledge that may have been acquired over the course
of the current situation, and also in a cultural and physical world that is
shared by members of the community of practice. The prior discourse
activity situates this symbolic gesture, imbuing it with meaning and communicative power well beyond what is explicitly conveyed. B’s hand is
just a hand in the absence of the surrounding aural, visual, and social
context. The gesture packages a complex theory into a simple, easily manipulable form. Recognizing a gesture as a meaningful display involves
not just orientation to someone’s moving hand, but also to the ongoing
creation and mutual alignment of disparate information forms emerging
through time and interaction (Goodwin 2000). The elements required to
104
L. Amaya Becvar et al.
assemble the meaning of a gesture are distributed, composed of di¤erent
media (graphical representations, moving hands, and speech) and also, in
this case, the activities of several participants.
3.4.
Robustness of a symbolic gesture in a community of practice
6.1. (00:28:48;00)
J: an’ we were looking at how (0.2)
{if you (0.1) bind thrombomodulin
6.2. (00:28:51;10)
what’s going on in the rest of the thrombin}
3.4.1. Segment 6. The data in this segment is taken from a follow-up
interview with J, a graduate student in the lab, who has done experiments
comparing the structural and dynamic di¤erences between thrombin with
and without thrombomodulin bound. This interview was completed six
months after the laboratory meeting from which the video Segments 1–5
were extracted. In the passage shown in Segment 6, J is summarizing the
research she did, which has recently been submitted for publication. As
shown in Figures 3 and 4, J’s gestures are remarkably similar to B’s
when she discusses the outcome of new experimental measurements of
thrombin.
J calls upon the very same ‘thrombin hand’ gesture that B used roughly
six months earlier as she expresses her conceptualization of the structural
changes that occur in thrombin in light of new empirical data. Here we
see evidence the ‘robustness’ of the gesture over time as a meaningful
and conceptually useful representation, as J reinstantiates the very same
hand shape and dynamic motion in order to discuss similar concepts in
light of new experimental data that corroborates B’s initial conjecture.
Representational gestures in a scientific laboratory
105
Figures 3 and 4. B (left) and J (six months later) as they discuss a molecular-level model of
changes taking place in the active site of thrombin when thrombomodulin binds. Subtle di¤erences are due to the di¤ering camera angle. However, note how in both cases, the iconic mappings of the gesture are the same (palm ¼ active site, fingers ¼ loops around active site, and so
on).
Though J’s speech and word choice are di¤erent than B’s were, her gestures are not. Moreover, neither J’s speech nor her gestures are arbitrary:
they preserve key elements of conceptual structure that are essential for
doing reasoning and drawing inferences in alignment with new empirical
measurements and prior research findings. In Figure 2, evidence was presented of the propagation of the ‘thrombin hand’ gesture, extending over
several moments of discourse, and being used to support a number of theoretical inferences about the nature of the thrombin-thrombomodulin
interaction. In Segment 6 and Figures 3 and 4, it appears that this gesture form has been ‘conventionalized’ through conceptual and discursive
practices taking place over the ensuing six months between the initial
lab meeting when the ‘thrombin hand’ gesture was conceived, and the
follow-up interview.
4. Discussion
By examining video segments of scientists engaged in collaborative activity, we have studied the development of a scientific theory through various representational forms, graphical representations, language, and gesture, ultimately to come to reside in a symbolic gesture. In this case study,
we first observed how a researcher used gesture and speech to transform
106
L. Amaya Becvar et al.
her left hand into the static molecular model of a protein, which was projected on the wall of the conference room. By constructing the hand
model, she was then able to adapt and modify the molecular model of
thrombin, animate it in a way that was consistent with the new theory,
and bring it into the ongoing discourse stream. The group members reintroduced and built upon the hand model through gestural imitation and
indexicalization practices (LeBaron and Streeck 2000), such as spoken
language and gaze shifts.
We have focused on how representational gestures, coupled with
molecular models, are used in scientific activity to develop theories about
molecules in a biochemistry lab. Exploring gesture-in-interaction has
permitted a glimpse into the schematic ways scientists in this lab build
meaningful understandings about the behavior of protein molecules.
And as the end product of scientific activity is theory (or scientific facts
[Fleck 1979]), this process may play a special role in scientific communities.
Nevertheless, although the examples presented in this chapter describe in
detail an isolated instance of a symbolic gesture, the phenomenon we describe is much more general, and we would expect to see a similar process
in myriad communities, scientific and non-.
The construction of knowledge is a shared process of forming symbols
that ‘embody experiences that have emerged in situated action’ (LeBaron
and Streeck 2000). In the scientific laboratory, when the activity is theory
negotiation, shared symbols are often representations of scientific phenomena that may be conceived and instantiated in many di¤erent media,
such as inscriptions (Latour 1979, 1990), scientific language (Halliday
1993), and in this case, gesture. In the scientific laboratory, where the
most significant product of activity is the scientific theory itself (and representations of theories), symbols play an integral role in the evolution of
bodies of knowledge.
Symbol formation is a powerful process in the fabrication of shared
knowledge because it allows participants to re-invoke shared experiences
(LeBaron and Streeck 2000), which in this case, are conceptualizations of
molecular action. Also particularly significant is that this gestural symbol
not only packages a scientific concept into an easily manipulable sign, but
that its form may actually shape the way this theory is conceptualized,
i.e., as a protein with loops that move like fingers do. The ‘thrombin
hand’ gesture both indexes elements of the surrounding material environment and introduces new spatiodynamic properties, which allows the
community to further develop the conceptual theory regarding the biochemical system of study.
This is significant because it indicates that representational gestures can
be built upon, and referred back to, during a stream of discourse, and
Representational gestures in a scientific laboratory
107
moreover, stabilized. That is, seemingly ‘spontaneous’ gestures (McNeill
1992) are historically contingent. This is important because it suggests
that certain ges…