Audio-Tactile Short-Term Memory


  • Daniel Bermúdez University of Vienna



Perception works holistically: different perceptual inputs coming from distinct modalities are blended together in singular experiences. Normally, human beings do not discriminate the modalities and sources of their impressions unless an attentional focalization intervenes. Certainly, this immanent integration enhances the perception itself and the use of its contents in other cognitive processes. For example, what lacks or is confusing in the auditory sense can be compensated by an accurate information or processing in the tactile sense. But, simultaneously, the mutual influence between different perceptions can diminish or affect the processing. The aim of this project consists in investigating the role of STM in the integration of heard and felt impressions.


This project is based on existing cognitive computational models of tactile short-Term Memory [1] and arises from the Retrieved Context Theory [2]. In few words, it revolves around the idea that the links shaped by item information (i.e., frequency) and their contexts (e.g., time of presentation) can be used by the short-term memory to compare frequencies and make tactile judgments on the similarity of vibrations. Four mutually connected but different hypotheses are tested applying a computational model on data collected from N = 62 participants. First, the interferences in the memory about tactile frequencies emerge from a confusion of different items associated with similar contextual states. Secondly, the integration of different perceptual modalities is done in a contextual layer. Thirdly, the accuracy in the distinction of different auditory and tactile inputs comes from modality-specific tunings. Finally, the memory interference can be reduced by a dual-coding of context- item. The process for corroborating these hypotheses is divided in four laboratory experiments in which variants of a Delayed Match-to-Sample (DMS) tasks are implemented, i.e., matching and judging the similarity of a third vibration (the Probe) to two preceding vibrations), and Model- based simulations are utilized for the data analysis. So far, the first experiment has been conducted.

Partial Results

The main findings of the first experiment indicate that the observed data about the tactile perception follows a pattern that is consistent with the existence of a context layer, helping maintain item information of previously presented stimuli. Hence, the tactile perception seems to be supported by episodic memory structures, i.e., short-term traces of item and context information about tactile stimuli (Hypotheses 1 and 2). The efficiency of the computational model is demonstrated by its capacity to accurately predict the empirical data. Performance loss in the DMS task can be attributed to Item-Context binding errors.


[1] T. Bancroft, W. Hockley and P. Servos, “Can vibrotactile working memory store multiple items?,” Nerurosci. Lett., vol. 514, no. 1, pp. 31–34, Apr. 2012.

[2] M. Kahana, “Computational models of memory search,” Annu. Rev. Psychol., vol. 71, pp. 107–138, Jan. 2020.