Action in Perception? An Investigation with the Cognitive Architecture Nengo

Abstract

Some theories of embodied cognition propose that perception is contingent upon the execution of sensorimotor representations [1]. This idea receives support from research on mirror neurons, which identifies overlapping neural activation between perceiving an other-performed action and its self-execution [1], [2].

The cognitive architecture SPAUN aims to create a biologically plausible model of cognition, with which particular theories of cognitive functions can be modelled from the bottom up, within a spiking neural network that expresses crucial aspects of biological neural systems [3]. Such aspects include, among many things, the principle of neuronal reuse that runs fundamental to the execution of sensorimotor representations in organismic perception.

This prompts consideration of whether sensorimotor representation will feature in SPAUN-based models of cognition. Current applications suggest this as a promising avenue for research. SPAUN models have been developed that demonstrate proficiency in both visual recognition and the execution of actions by manipulating simulated arm muscles, thus enabling an exploration of overlapping representations within the neural system.

This project aims to perform a replication study of such an existing computational model within Nengo, the practical interface of SPAUN, and then adapt it so that this principle of neuronal reuse can be explored. First, a neural system is built which possesses the relevant functionality for action execution and visual recognition. It is then trained to perform a simple set of actions and is subsequently trained in a visual recognition task on video images of those same actions. A statistical analysis can then reveal neuronal reuse between action execution and action recognition. Importantly, the inclusion of multiple actions adds robustness to the findings, as it allows to verify whether neuronal reuse is the greatest between the execution and recognition of the same action.

Therefore, this project furthers the understanding of this important principle in biological cognition by exploring its possible emergence within a neural system which is functionally and mechanistically explainable. This helps put to light the functional importance of action in perception, as well as the elementary components needed to express it.

References

[1] F. de Vignemont, “Introduction”, in Mind the Body: An Exploration of Bodily Self-Awareness. Oxford University Press, 2018, pp-1-10.

[2] L. Bonini, C. Rotunno, E. Arcuri, and V. Gallese, “Mirror neurons 30 years later: implications and applications,” Trends in Cognitive Sciences, vol. 26, no. 9, pp. 767–781, Sep. 2022

[3] C. Eliasmith, How to build a brain: A neural architecture for biological cognition. Oxford, 2013, Oxford University Press.