Poster
Ikeda, Tomoko
Markman, Kris M.
Mehus, Siri
The University of Texas, Austin
Representing multiple modalities : A survey and critique
In research on gesture and embodied action, representations of data
such as transcripts, drawings, diagrams, photographs, and video
clips serve two major functions : they illuminate interactive phenomena
and provide evidence for claims. In the presentation of fi ndings,
evidence for the existence and importance of the phenomena discussed
is to be found in the accompanying transcripts and illustrations.
Researchers who study spoken discourse have investigated the
practice of transcribing talk in its own right (e.g., Dressler & Kreuz,
2000 ; Du Bois, 1991 ; Edwards & Lampert, 1993 ; Ochs, 1979 ; Roberts
& Robinson, 2004). However, there has been little systematic
research on transcription systems for embodied actions.
The growing recognition of the multimodal nature of human communication
has led to an increased need to represent multiple aspects
of interaction in our written work. This shift in focus to multiple
phenomena means that one transcript may no longer be suffi cient to
represent everything of interest. As a result, different transcription
and data representation practices have evolved to serve the needs of
different disciplines and research foci.
Our purpose here is to survey a selection of practices used to represent
gesture and multimodal interaction, and to offer some recommendations
for the presentation of multimodal transcripts.
