TEXT-IMAGE RELATIONS IN CARTOONS. A CASE STUDY OF IMAGE SCHEMATIC METAPHORS
Volume 27 Issue 4 Nov , pp. Volume 26 Issue 4 Nov , pp.
Volume 25 Issue 4 Nov , pp. Volume 24 Issue 4 Nov , pp. Volume 23 Issue 4 Nov , pp. Volume 22 Issue 4 Nov , pp.
Metaphor and Second Language Learning: The State of the Field
Volume 21 Issue 4 Nov , pp. Volume 20 Issue 4 Oct , pp. Volume 19 Issue 4 Nov , pp. Volume 18 Issue 4 Nov , pp. Volume 17 Issue 4 Dec , pp. Volume 16 Issue 4 Dec , pp. Volume 15 Issue 4 Sep , pp. Volume 14 Issue 4 Oct , pp. When textures combined fluctuations and an upward direction i. This could result from physical difficulties to combine two types of gestures although it does not seem difficult to make this combination. In turn, for stationary textures, tonal textures were depicted with slow simple gestures, and noisy or granular textures were depicted with rapid complex gestures. In this case, the gestures focused only the fluctuation of the textures.
- Patterns of Imagery in Language?
- Aeschylus: The Oresteia (Landmarks of World Literature (New))?
- Metaphor and Gesture.
It is possible that imitators chose to focus their gestures on only one property of the referent sounds because they were rendering the other properties with their voice. But, as discussed below, a symmetrical selectivity was not observed for the vocalizations. As noted by Eitan and colleagues, the different metaphors used to represent sounds interact and are not symmetrical [ 38 ].
In contrast, vocalizations successfully combined the effects of tonalness and profiles of the referent sounds: vocalizations were noisy unvoiced in response to noisy textures and tonal voiced in response to tonal textures irrespectively of the direction of the textures , and their pitch or spectral centroid matched the direction of the referent sounds irrespectively of the tonalness of the referent sounds. Our measures were only able to show a modest effect of the granularity of the referent sounds, only for the tonal referent sounds: the pitch of the vocalization was more jittery in response to granular than to tonal sounds.
It is possible that our measure was insensitive to the relevant vocal features for the granular noisy textures. Another possibility is that imitators did not have any strategy to add more fluctuation to their already fluctuating unvoiced vocalization. However, informal listening tests suggested that it is possible to perceive a specific modulation added to the imitations of the granular textures.
Overall, this suggests a different role for the imitative vocalizations and gestures. Whereas the vocalizations reproduce several features of the referent sounds as faithfully as vocally possible, the gestures emphasize one salient feature with metaphors based on auditory-visual correspondances.
- Most Downloaded Articles.
- Passar bra ihop.
- Metaphor and Gesture!
- Edited By Marek Kuźniak, Agnieszka Libura and Michał Szawerna.
- Metaphor and Gesture | Edited by Alan Cienki and Cornelia Müller.
- The Role of Gamma Oscillations During Integration of Metaphoric Gestures and Abstract Speech.
Admittedly, the referent sounds in the experimental study were simple, the feature values had been chosen to be easily vocalized, and our previous work had shown that imitators actually use a variety of vocal strategies that depart from faithful reproduction to convey more complex acoustic features less amenable to vocal imitation [ 22 , 23 ]. Nevertheless, the emphasizing role of the gestures is striking. As such, it could potentially be decoded by receivers to focus their attention of some aspects of the vocalizations.
Further work is needed to explore such hypotheses. From a methodological point of view, these results were confirmed by the combination of two complementary studies: an observational study with ecologically valid and complex referent sounds, and an experimental study with simple, artificial, but controlled sounds that enabled us to use quantitive measures and inferential statistics. Instead of comparing the trajectories of sound and gesture features as in other sound tracing studies [ 52 , 58 ], our analyses sought to identify different types of correspondences between auditory features of the referent sounds and visual features of the gestures.
Initial observations and discussions with non-expert participants suggested that such a concept was foreign to them, as also observed by Caramiaux and colleagues [ 52 ]. Whereas the spatial metaphor of pitch could be considered as some kind of parameter tracing the position of the hands follows the evolution of pitch in time , the rustling metaphor appears to be of a more categorical nature: we were not able to observe more rapid and complex gestures when we combined noisiness and granularity.
Here again, further work is needed to inquire the categorical nature of such behavior. The origin of the metaphors is somewhat puzzling. In principle, they could be based on a pure convention e. Despite its ubiquity in the literature, there is no clear answer as to the origin of the spatial metaphor of pitch. Many languages use spatial analogies to describe pitch e.
But Dolscheid and colleagues have shown that these terms are not purely conventional for example they cannot be reversed and therefore must rely on more deeply engrained associations [ 48 ]. Lakoff and Johnson have argued that most Western concepts are organized in terms of spatialization metapors [ 36 ]: happy is up and sad is down, health is up and sickness is down, good is up and bad is down, rational is up and emotional is down.
Such metaphors are rooted in strong cultural and physical experiences. For example, when you pile up physical objects, level increases. Similarly, the spatial metaphor of pitch could be based on cultural or physical experience. For example, the Western musical notation system also uses such a spatial representation of pitch, and pitch models in music psychology also represent pitch height along the vertical dimension [ 82 ]. Another common interpretation is that this metaphor relies on the physical production of sounds by musical instruments.
Low notes are mapped to leftmost positions on a piano keyboard [ 44 ] and on the neck of guitars and luths, but the are mapped to the rightmost keyholes on flutes. They are mapped to the farthest positions of the fingers on the neck of a violin and of the slide of a trombone, to the highest positions on upright basses and violoncellos, but to the lowest keyholes on woodwind instruments the mapping refer here to the position of the fingers or the hands to produce a given note, from the point of view of the player.
So if listeners represent pitch with a spatial metaphor based on the physical production of sounds on musical instruments, they can potentially use different mappings. The most common mapping of pitch to spatial position seems to be that lower pitches are localized on a lower, more left, and closer position with respect to the listener than higher pitches that are localized on a higher, more right, and farther position.
One possibility is that participants were not able to hear correctly the pitch direction i. Self-confrontations interviews showed however that most of them voluntarily associated low pitches with higher positions and vice versa. Whereas the spatial representation of pitch is overwhelming, the precise mapping between pitch and spatial position varies a lot across subjects. The origin of the rapidly shaking gestures is even more intriguing. The self-confrontation interviews suggested a few potential origins.
But the most common interpretation was either causal or acoustical. In the causal interpretation, the participants mentioned and pantomimed natural events such as bubbles, raindrops, or rustling leaves. As such, participants were embodying the physical actions causing the sounds. In the acoustical interpretation, participants mentioned that their rapidly shaking gestures were illustrating the density of small events composing the referent sounds.
As such, the shaking metaphor could be seen as an extension of spatial metaphors, wherein small spatial trajectories illustrate the fluctuations of pitch or loudness which is a rather accurate definition of a noise or a granular sound. Though none of the imitators was a signer, the applause sign may have originated from a similar metaphor, depicting the complex acoustic pattern of individual claps composing a crowd applause. It should be noted however, that our measurements did not capture more subtle effects.
For example, in the latter acoustical interpretation, many participants indicated that they used gestures to indicate density of events because they could not do it with their voice. Some metaphors also seem to have been used by only a few participants e. The shared use of such metaphors across participants nevertheless offers a number of interesting questions.
A first question is whether receivers can actually decode the gestures as they do with gestures accompanying speech [ 14 ]. For example, the idea that gestures actually emphasize one salient aspect of the vocalizations remains to be tested. Another question is whether these gestures can be combined with a syntax and thus take a linguistic form, as observed in spontaneous sign languages [ 85 ]? Goldin-Meadow has suggested that gestures take a linguistic form only when they are used without speech, such as in sign language [ 14 ]. In our initial observations [ 1 ], depicting gestures always accompanied speech or non-speech vocalizations.
It remains to be studied whether spontaneous depicting gestures can communicate sensory experience by themselves, or whether they need to be combined with non-speech vocalizations. For example, some theories argue that gestures are well suited to communicate sensory experience and describe tool manipulation without vocalization, and advance that gestures could have been precursors to the apparition of vocal language in humans [ 15 , 16 , 86 — 88 ].
In addition to human-human interactions, these results also offer perspectives for the design of new human-computer interactions.