Gerth, beim Graben & Vasishth (Version 3)

edit: title changed

Tensor product models for language-related brain potentials

Sabrina Gerth (1), Peter beim Graben (2), & Shravan Vasishth (1)
(1) Institute for Linguistics, University of Potsdam, Germany
(2) School of Psychology and Clinical Language Sciences, University of Reading, UK

A central question in computational psycholinguistics is how symbolic processing capabilities such
as language are realized by neurodynamics in the human brain. An important online measure of language
processing is event-related brain potentials (ERPs). However, ERPs are currently interpreted purely phenomenologically,
and few computational accounts exist (see Hagoort 2005 for a recent proposal). We use tensor product representations of symbolic structures (Smolensky 1990; Smolensky and Legendre 2006; Mizraji 1989, 1992) to model syntactic parsing as nonlinear dynamics in neural activation space, and argue that ERP differences are due to trajectories which explore different regions in that space.

In an ERP experiment on the processing of German subject-object ambiguities, the sentence (1b) evoked a P600
ERP compared to (1a) reflecting an initial garden path interpretation. Starting (for simplicity) with a Government and Binding
formulation (Haegeman 1994) of the sentences (1a,b), we construct a locally ambiguous context-free grammar
from the phrase structure trees. This grammar is decomposed into its unambiguous parts representing the two
alternative processing strategies, namely subject preference against object preference in order to construct
two appropriate deterministic pushdown recognizers (Aho & Ullman 1972).

Using the tensor product representation, the syntactic categories of the disambiguated grammars are mapped
onto linearly independent filler vectors, while positions in a labeled binary tree are given as a basis of
three-dimensional space. In order to built a parallel processor (Lewis 1998) the two parses for the
subject-object sentence (regular vs. garden path) and the other two for the object-subject sentence
respectively were linearly superimposed in activation space. Then, model ERPs are obtained as the first
principal component.

Our model is able to describe, at least qualitatively, the obtained ERP results by trajectories that explore
functionally and causally different regions in activation space while pursuing different language processing
strategies. During its transient evolution, the trajectories of the model diverged exactly when the garden
path was encountered which shows remarkable resemblance with the P600 effect in the ERP.

(1a) Die Rednerin hat den Berater beim Kongress gesucht
The speaker [AMBIG] has [the advisor][SUB] at the congress sought
'The speaker has sought the advisor at the congress'

(1b) Die Rednerin hat der Berater beim Kongress gesucht
The speaker [AMBIG] has [the advisor][OBJ] at the congress sought
'The speaker has been sought by the advisor'

Haegeman L. (1994) Introduction to Government & Binding Theory, Blackwell Textbooks in Linguistics, vol1, 2nd
edn. Blackwell Publishers, Oxford, 1st edition 1991.
Hagoort P. (2005) On Broca, brain, and binding: a new framework. Trends in Cognitive Science 9(9):416 - 423.
Lewis RL (1998) Reanalysis and limited repair parsing: Leaping off the garden path. In: Fodor and Ferreira (1998), pp 147 - 285.
Mizraji E. (1989) Context-dependent association in linear distributed memories. Bulletin of Mathematical
Biology 51(2):195 - 205.
Smolensky P. (1990) Tensor product variable binding and the representation of symbolic structures in