Visual Metaphors in Street Art

 

A collaboration with the Cognitive Semiotics group at Lund University (Sweden). Georgios Stampoulidis is a Greek PhD fellow working on visual rhetoric in street art, and focusing on the political graffiti that appeared in Athens in the past years (examples are shown in the image below). His PhD project is supervised by Professor Goran Sonesson and Professor Jordan Zlatev, at Lund University. Georgios invited Dr Marianna Bolognesi to Lund for 5 days in November, to work together on the development of an identification procedure and analytical framework that can be reliably applied to the identification and analysis of visual metaphors in street art. The collaboration is ongoing, and data sessions are scheduled for the upcoming 3 months. Stay tuned!

   

Posted November 28, 2017

New experiment developed in collaboration with GReSCA

 

Together with Lorena Bort-Mir, Marianna Bolognesi is developing a cross-linguistic and cross-cultural experiment to test the processing of metaphorical short films. The FilMIP procedure (filmic metaphor identification procedure) developed by Lorena Bort Mir within her PhD project, on Marianna and Lorena worked last summer at Metaphor Geeks, will be applied to select the stimuli for this experiment. Then, in a Think Aloud task, participants with different linguistic and cultural background will be submitted to watch the metaphorical video and verbalize out loud their thoughts. The transcribed protocols will be analyzed in a formal content analysis, and the results will arguably inform the analysts about the steps that viewers undertake when they process the metaphorical structures embedded in these videos. The data will be collected with English, Italian, and Spanish native speakers. The collaboration involves the GReSCA group for Contrastive and Applied Semantics, at Universitat Jaume I (Castello’ de la Plana, Spain), which is directed by Professor Ignasi Navarro as the director; Metaphor Geeks in Siena, Italy, coordinated and directed by Dr. Marianna Bolognesi, and the Creative Multilingualism programme at the University of Oxford, UK, directed by Professor Katrin Kohl. Stay tuned!

 

Example of metaphorical short video used for the experiment.

Posted November 28, 2017

Creative Multilingualism

Starting from October 1st, dr. Marianna Bolognesi will be officially involved as Postdoctoral Research Associate within the Creative Multilingualism program, led by the University of Oxford, and funded by the UK Arts and Humanities Research Council.

Marianna’s research focus will be on Metaphor  (strand 1 of the creative Multilingualism program) and she will be advised directly by Katrin Kohl, PI of Creative Multilingualism.

 

Posted October 19, 2017

Applying the FilMIP procedure

 

Lorena Bort-Mir is visiting the Metaphor Geeks Lab to work with Marianna Bolognesi on the FilMIP procedure, a procedure aimed at identifying filmic metaphors in the wild. Currently the procedure has been applied to various instances of filmic advertisements. FILMIP relates to and derives from other procedures that have been developed within the field of metaphor studies, for the specific identification of metaphors in various modalities. The original MIPVU procedure was developed by the Pragglejaz group and then improved by Steen et al (2010), and it is used for the identification of linguistic metaphors in corpora of texts. More recently, the VisMIP procedure has been proposed (Sorm, Steen, forth.), for the identification of metaphors in still images.

FilMIP will be illustrated at the Metaphor Festival Amsterdam (31st August – 2nd September) in a parallel session, as well as in a dedicated workshop, which will be held on Saturday September 2nd, within the Festival program. Stay tuned and register for THE Festival!!

 

Posted June 28, 2017

Visiting PhD fellow at Metaphor Geeks

Lorena Bort-Mir, lecturer at the University Jaume I (Spain) will be collaborating with Dr. Bolognesi for the implementation of a metaphor identification procedure that can support scholars in cognitive linguistic, media studies and metaphor studies, in reliably identifying filmic metaphors. The collaboration is sponsored by Universitat Jaume I (UJI): Lorena was awarded with a 2500 euros prize to support her research stay at Metaphor Geeks. Congratulations!

 

Posted May 30, 2017

Upcoming Conference on Multimodal Communication

The Metaphor Lab Amsterdam in collaboration with Metaphor Geeks will be at the International Conference on Multimodal Communication, in Osnabruek. Conference website here.

Posted May 13, 2017

High-Concept Low Downs: Workshop in Dublin

On March 29th Marianna Bolognesi was invited to give a talk at a one-day multidisciplinary workshop titled High-Concept Low-Downs. Invited speakers were tied together by their common interests in creativity (cognitive, linguistic, social, etc.). The workshop organizer and coordinator was prof. Tony Veale, principal investigator of the Creative Language System Group at the University College of Dublin, and world expert in computational creativity.

Great group of international scholars! Looking forward to concretize a collaboration!

Posted April 13, 2017

COGVIM project finalized

 

1st February 2017: The COGVIM project officially ends. We had a final dinner with the Metaphor Lab Amsterdam, to celebrate the successful completion of the 2-year Marie Curie awarded COGVIM project.

The project scientific findings are hereby summarized:

.

Project summary

.

The COGVIM project (Cognitive Grounding of Visual Metaphor, URL: https://cogvim.org/) explored the differences between visual and linguistic metaphors, that is, metaphors expressed through still images (such as advertisements, political cartoons and artworks) and metaphors expressed through words (for example in everyday conversation, in academic discourse, or in the news). An example of visual metaphor is the political cartoon displayed here:

 

In this image, the author (cartoonist Angel Boligan) displays an ATM embedded in a church confessional. The image stimulates the viewer to associate, metaphorically, the ATM machine with the confessional, and find possible similarities between the two entities. Conversely, an example of a linguistic metaphor is the linguistic statement “forget the fiscal cliff” (COCA corpus). Here, the word cliff is used in a metaphorical way, in relation to the economic crisis domain.

.
We predicted that visual and linguistic metaphors would construct metaphorical comparisons relying on different aspects of the meaning of the concepts aligned in the metaphor. In particular, based on extensive literature reviews, we distinguished between three types of conceptual structures, which could be operationalized into three different types of semantic representation by means of vector spaces. Based on the three different semantic representations, it was possible to investigate three different types of similarity between pairs of concepts that are aligned in visual or linguistic metaphors): 1) attributional similarity, that is, the similarity between metaphor concepts based on the entity-related semantic features that are shared by the two concepts; 2) relational similarity, that is, the similarity between metaphor concepts based on the shared contextual structures that characterize each pair of metaphor concepts; 3) language-based similarity, that is, the similarity between metaphor concepts based on the linguistic contexts that are shared by two metaphor concepts.

 

Project findings

 

The metaphor similarity modeled across the three different distributional spaces in WP1, WP2, and WP3 shows different patterns for, respectively, visual and linguistic metaphors. Therefore, the results support our hypothesis that visual and linguistic metaphors are constructed and represented on the basis of different types of semantic information, which are exploited to cue metaphorical comparisons between two aligned concepts (Bolognesi 2016a; Bolognesi 2016b; Bolognesi, Aina 2017).

.
In particular, in WP1 we found that the attributional similarity in visual metaphors is significantly higher than for linguistic metaphors. This similarity is based on shared features between metaphor terms that express entity-related properties, such as for example perceptual and systemic properties, or components of a given concept (e.g. CONFESSIONAL and ATM both have flat surfaces, while there are no reported entity-related properties, shared between SUBJECT and AREA).

.
In WP2 we found that the experience-based relational similarity in visual metaphors is significantly higher than for linguistic metaphors. This similarity is based on shared features between metaphor terms that express experience-based related properties, such as for example locations in which the concepts appear, or objects and participants that populate these environments (e.g. CONFESSIONAL and ATM, both typically appear together with human participants that interact closely with them, while there are no reported experience-based properties shared between SUBJECT and AREA).

.
Conversely, in WP3 we found that the language-based similarity is significantly higher for linguistic metaphors than for visual ones. This similarity is based on shared linguistic contexts between metaphor terms, such as for example grammatical and lexical structures in which two metaphor terms are typically used (e.g. for SUBJECT and AREA: a debated subject/area; subject/area of study, while for CONFESSIONAL and ATM there are significantly less shared linguistic contexts in which the two words are used).

.
A closer inspection of the data suggested, in addition, that visual metaphors tend to be on average more creative (i.e. less conventionalized) than linguistic metaphors, and they tend to involve on average more concrete concepts, because these have to be graphically depicted. In particular, it was observed that on one hand linguistic metaphors used in natural language tend to be highly conventionalized (e.g., “I seewhat you mean”, where see means ‘understand’), while visual metaphors are typically constructed as ad-hoc comparisons, to serve a specific communicative goal within a given genre. As a result, they are typically more creative (e.g., an anti-smoking campaign showing a cigarette in the shape of a coffin). Conversely, visual metaphors tend to involve more concrete concepts, because they have to be graphically depicted.

.
These two additional variables, Conventionality and Concreteness, were therefore investigated, to check whether they could also explain the different similarity patters obtained in the three work packages. A series of regression analyses showed that only the independent variable ‘Modality of expression’ (visual or linguistic) could significantly predict the amount of similarity between metaphors terms across the three distributional semantic spaces, while ‘Concreteness’ and ‘Conventionality’ did not (Bolognesi, in preparation).

.
These findings all together support multimodal accounts of cognition, where multiple modality-specific semantic representations contribute to shape our conceptual system. This is a crucial topic within the cognitive linguistic framework and within the field of metaphor studies, where, instead, it is often assumed that there is a one-to-one correspondence between words and concepts, and that the linguistic system matches and exhausts the semantic information retrieved from perceptual experiences.

.
Finally, we observed that in order to achieve a full interpretation of the metaphorical images, the viewer often needs to perform additional cognitive operations (besides identifying the shared features between two aligned concepts). Such complex cognitive operations explain how abstract concepts can emerge from concrete, depicted instances, and include combinations of metaphors with metonymies, and feature projection from one domain onto the other, which is used to create ad-hoc similarities, as opposed to pre-existing ones (Bolognesi, Steen, in preparation).

.
These final observations about the emergence of abstract concepts from concrete categories led the COGVIM team to organize an international and interdisciplinary symposium on the structure, processing and modeling of abstract concepts (https://abstractconceptsnet.wordpress.com/), which was attended by 100+ international delegates. The emerging debate will be collected in a special issue of the journal Topics in Cognitive Science (Eds. Bolognesi, Steen, in preparation) while a book proposal for the book series Human Cognitive Processing (Benjamins Publishers), currently under review, will collect a selection of papers based on the best presentations delivered at the symposium.

 .

 

References (COGVIM-related articles that have been published or accepted for publication)

 

Bolognesi M., Aina L. (2017) Similarity is Closeness: Using Distributional Semantic Spaces to model Similarity in Visual and Linguistic Metaphors. Corpus Linguistics and Linguistic Theory. DOI: https://doi.org/10.1515/cllt-2016-0061

Bolognesi M., Pilgram R., Van den Heerik R. (2016). Reliability in semantic categorization: the case of semantic features norms. Behavior Research Methods. DOI 10.3758/s13428-016-0838-6

Bolognesi M. (2016a). Using semantic features norms to investigate how the visual and verbal modes afford metaphor construction and expression. Language and Cognition. DOI: https://doi.org/10.1017/langcog.2016.27.

Bolognesi M. (2016b) Modeling semantic similarity between metaphor terms of visual vs linguistic metaphors through Flickr tag distributions. Frontiers in Communication, 1:9. DOI: https://doi.org/10.3389/fcomm.2016.00009.

Posted April 12, 2017

New collaboration!

 

Giorgia Tsouderos is a doctoral fellow in Comparative Studies in Greece. (B.A. degree in Philosophy, Univ. of Bologna and M.A. degree in Philosophy, Univ. of Crete). She is bilingual Italian-Greek, and lives between Bologna and Athens.

Her doctoral project addresses the following topic: Metaphor & Literature: a multi-modal approach. A summary can be found here: Metaphor Geeks – Giorgia Tsouderos.

Given the interdisciplinary nature of her project (between philosophy, cognitive linguistics, cognitive science, and literature) and the topic that she tackles (metaphors) Giorgia collaborates with Metaphor Geeks.

Posted December 9, 2015

COGVIM project kick off

 

The Marie Curie project COGVIM, the cognitive grounding of visual metaphor, just kicked off.

The project will be embedded in the Metaphor Lab, Amsterdam, and will lasts 2 years (2015-2017).

More info about COGVIM here.

Posted March 7, 2015