(Salle F08)
Alac, Morana (University of California, San Diego)
Seeing scientific representations through body engagement
To study multimodal interaction I observe, videotape, and analyze work of two groups of cognitive scientists. The first group uses functional magnetic imaging technology (fMRI) to produce images of brain function. If the image is obtained while an experimental participant is engaged in a particular cognitive task, the image can indicate which parts of the brain are most active in that task. The second group designs computer programs to model human social behavior.
The main interests are the emergence of social attention and defi cits in shared attention in autism. The modeling group experiments with computer animated models of caregivers and infants.
Here I analyze two instances of practice where scientists interact with each other and with computer screens. I discuss how the interpretation of scientific images is produced through the interaction of various semiotic modalities, and how a virtual world acquires meaning through its embodied enactments in the social world of action.
Particular attention is directed toward the human body. The body
is not just an envelope for internal, abstract thought, but a critical component of meaning-making. In addition to semiotic functions of body orientation, positioning, and gesture, I focus on how the body enacts scientific representations. For example, an fMRI practitioner
while observing digital brain images can use her own body as a semiotic tool that comments on behavior of the experimental subject’s body, or a general human body. At the same time the practitioner can also use her body as a signifi er for such bodies. Similarly, while observing the modeling group I noticed a pervasive pattern where practitioners engage their own bodies to model the behavior of virtual caregivers and infants.
Employed in this fashion, the body is not a stable sign, but a trace able to quickly assume very different and multiple semiotic roles. In addition to this extreme fl exibility, the body enhances collaborative viewing, and a first person understanding of the experimental data.
Furthermore, the analysis focuses on how such embodied models
interact with various scientific representations such as hand drawn charts, graphs, and digital models that co-exist in the environment of practice.
