Biologically Motivated Learning in Neural Networks with Convolutional Architectures

Abstract

Universal Bidirectional Activation-based learning (UBAL) is a novel learning algorithm for artificial neural networks [1], which is based on the workings of real biological neurons. Instead of propagating error derivatives it uses local activation values for updating its weights. It extends contrastive Hebbian learning, which uses presynaptic and postsynaptic activation values for the updates. It is also bidirectional for which it uses two different weight matrices for each activation direction. The ideas for UBAL comes from its predecessor algorithms recirculation and GeneRec [1].

UBAL has never before been implemented in a convolutional version, which is the main aim of the proposed master thesis. Convolutional neural networks are usually better suited for processing images, therefore our hypothesis is that convolutional UBAL will also yield better results in the image classification task compared to the current, fully connected version. We will implement convolutional UBAL and make experiments in the programming language Python, namely the Pytorch library [2].

We will first test the implementation on the famous MNIST dataset and possibly extend the experiments also to the Cifar10 dataset. Apart from the Convolutional UBAL, we will explore the influence of target encoding on the success of the UBAL network in terms of using representations of a more perceptual nature along with the perceptual symbol system hypothesis (PSSH) in cognitive science [3].

According to PSSH representations in our cognition are not arbitrary symbols (e.g. words) as originally assumed by classical computationalism. They do indeed resemble their referents in the world and are multi-modal in essence, including visual properties, affordances, memories, etc. In this sense a representation of a digit (e.g. MNIST dataset) in the neural network would have some resemblance to the digit as it relates to the process of its recognition, so instead of coding it as a position in a vector of zeros (one-hot) we use digitized image that would represent all different examples of that digit in the trained network. We hypothesize that this encoding together with the biological nature of the convolutional UBAL model will yield better results.

References

[1] K. Malinovská, L. Malinovský, P. Krsek, S. Kraus, and I. Farkaš, “UBAL: a Universal Bidirectional Activation-based Learning Rule for Neural Networks,” in Proceedings of the 2019 2nd International Conference on Computational Intelligence and Intelligent Systems, New York: Association for Computing Machinery, 2020, pp. 57–62. doi: 10.1145/3372422.3372443

[2] A. Paszke et al., “Pytorch: An imperative style, high-performance deep learning library,” arXiv.org, https://arxiv.org/abs/1912.01703 (accessed May 9, 2023)

[3] L. W. Barsalou, "Perceptual symbol systems“, Behav. Brain Sci., vol. 22, no. 4, pp. 577–660, August 1999. doi:10.1017/S0140525X99002149