Concept Learning in Logic Tensor Networks

Abstract

How is information represented in cognitive systems? How do these representations emerge? How are they processed? These are pivotal questions in cognitive modelling. Our study assumes that cognitive systems generate multiple kinds of representations that favor different processing styles. We propose that representations can be roughly organized in levels of increasing abstraction, and decreasing specificity: the sub-symbolic, the conceptual, and the symbolic levels. Our study aims at bridging the conceptual and symbolic levels using Logic Tensor Networks (LTN), a neuro-symbolic framework combining input-driven pattern extraction with rule-based reasoning [1]. Built on top of a neural network basis, Real Logic is a fully differentiable fuzzy first-order logic (FOL) where all symbols i.e., objects, functions, and predicates, are grounded into real-valued tensors, tensor operations, and standard fuzzy logic operators respectively [1]. 

Methods

We use a previously published data set on 13,978 movies based on highly processed, semantically interpretable geometric representations of review texts which we treat as conceptual space [2]. We apply the data set in a classification task with three types of labels: genres, keywords, and ratings. Each label has multiple classes (multi-class) and often movies belong to multiple classes in one category (multi-label). In the case of LTNs, the “classes” are used as FOL symbols which are grounded in the vector representations in conceptual space and refer to concepts. Simple logical rules of the type A implies B and A different B are approximated using conditional probability and difference between sets respectively. The LTN is optimized for satisfiability i.e., the network learns which vector representations of the symbols satisfy the logical rules most. 

Experiments

We explore whether LTNs can offer a competitive performance compared to standard ML models, whether they can benefit from background knowledge in the form of logical rules, and whether they can compensate for missing data. To investigate these questions, we train standard ML algorithms – kNN and ANN – on the classification task and compare the performance to the LTN that has additional learning constraints in the form of logical rules.

Conclusion

The main contribution of our study consists in exploring the potential of a neuro-symbolic framework with regards to performance and capability to generalize on a real-world data set.

References

[1] S. Badreddine, A. d’Avila Garcez, L. Serafini, and M. Spranger, “Logic Tensor Networks,” Artif. Intell., vol. 303, 2022, doi: 10.1016/j.artint.2021.103649.

[2] T. Ager, O. Kuzelka, and S. Schockaert, “Modelling Salient Features as Directions in Fine-Tuned Semantic Spaces,” in Proceedings of the 22nd Conference on Computational Natural Language Learning, 2018, pp. 530–540, doi: 10.18653/v1/K18-1051.