Representational Comparison of Figurative and Literal Meaning with Multivariate EEG

Abstract

Understanding the neural underpinnings of metaphor comprehension remains a fundamental challenge in cognitive neuroscience. This research plan proposes an investigation into the representational patterns of figurative meaning-making using multivariate pattern analysis on EEG data.

During data collection, participants are presented with AI-generated pictures depicting novel metaphors to ensure a higher cognitive demand necessary for the online processing of abstract stimuli, as opposed to conventional metaphors. The images are generated using a pre-normed list of novel linguistic metaphors. This linguistic base stimuli consist of adjective-and-noun word pairs in Hungarian. (e.g.: “tündöklő agy” = brilliant brain). We assign two images to each linguistic item. Pictures belonging to the first category of visual stimuli depict literalized meaning: both parts of the metaphor (vehicle and topic) are present, but the overall figurative meaning is missing. Pictures in the second category depict figurative meaning, merging the vehicle and the topic of the metaphoric expression. First in the experiment, 1) the linguistic source expression is shown to the participant, followed by 2) the literalized picture and 3) the metaphoric picture. This is followed by a choice task to assess preference between the two pictures.

This proposal employs innovative analysis techniques that combine computational models and activation data by creating and comparing representational distance matrices. Multivariate pattern analysis (MVPA) ensures thorough descriptiveness by working with an overall pattern of the data captured through electrodes instead of analyzing univariate recordings individually. Combined with the representational similarity analysis (RSA) technique, it promises insight into fine-grained patterns of neural activity [1].

During the analysis stage, EEG data packages collected during the presentation of 1) the linguistic expression, 2) the literalized picture, and 3) the metaphoric picture are converted into representational matrices and analyzed with multivariate RSA [2]. At this stage, the main focus is on comparing matrices to see whether between-category dissimilarity is significantly higher than within-category dissimilarity and whether a clear distinction can be made. A similar group comparison can be conducted based on outputs from the preference task. Using the resulting patterns, we can also check whether they have predictive value on unseen data. Another area of interest is detecting an abstractness effect: sustained frontal negativity presumably tied to figurative meaning processing [3].

The findings will contribute to models of figurative processing and offer insight into how the brain navigates between abstract and concrete conceptual domains. Results gained from this study may have implications for visual semantic processing and could inform the development of further investigations and methodology.

References

[1] K. Ashton et al., “Time-resolved multivariate pattern analysis of infant EEG data: A practical tutorial,” Developmental Cognitive Neuroscience, vol. 54, p. 101094, Apr. 2022. doi:10.1016/j.dcn.2022.101094

[2] N. Kriegeskorte and R. A. Kievit, “Representational geometry: Integrating Cognition, computation, and the brain,” Trends in Cognitive Sciences, vol. 17, no. 8, pp. 401–412, Aug. 2013. doi:10.1016/j.tics.2013.06.007

[3] B. Forgács, “An electrophysiological abstractness effect for metaphorical meaning making,” eNeuro, vol. 7, no. 5, Aug. 2020. doi:10.1523/eneuro.0052-20.2020