Gerth, beim Graben & Vasishth (Version 4)

Thanks for the great comments!
edit: typo corrected
edit: changed Haegeman reference to Chomsky(1981)


Tensor product models for language-related brain potentials

Sabrina Gerth (1), Peter beim Graben (2), & Shravan Vasishth (1)
(1) Institute for Linguistics, University of Potsdam, Germany
(2) School of Psychology and Clinical Language Sciences, University of Reading, UK

A central question in computational psycholinguistics is how symbolic processing capabilities such as language are realized by neurodynamics in the human brain. An important online measure of language processing is event-related brain potentials (ERPs). However, ERPs are currently interpreted purely phenomenologically, and few computational accounts exist (see Hagoort 2005 for a recent proposal). We use tensor product representations of symbolic structures (Smolensky 1990; Smolensky and Legendre 2006; Mizraji 1989, 1992) to model syntactic parsing as nonlinear dynamics in neural activation space, and argue that ERP differences are due to trajectories which explore different regions in that space.

In an ERP experiment on the processing of German subject-object ambiguities, the sentence (1b) evoked a P600 ERP

compared to (1a) reflecting an initial garden path interpretation. Starting (for simplicity) with a Government and Binding formulation (Chomsky 1981) of the sentences (1a,b), we construct a locally ambiguous context-free grammar from the phrase structure trees. This grammar is decomposed into its unambiguous parts representing the two alternative processing strategies, namely subject preference against object preference in order to construct two appropriate deterministic pushdown recognizers (Aho & Ullman 1972).

Using the tensor product representation, the syntactic categories of the disambiguated grammars are mapped onto linearly independent filler vectors, while positions in a labeled binary tree are given as a basis of three-dimensional space. In order to build a parallel processor (Lewis 1998) the two parses for the subject-object sentence (regular vs. garden path) and the other two for the object-subject sentence respectively were linearly superimposed in activation space. Then, model ERPs are obtained as the first principal component.

Our model is able to describe, at least qualitatively, the obtained ERP results by trajectories that explore functionally and causally different regions in activation space while pursuing different language processing strategies. During its transient evolution, the trajectories of the model diverged exactly when the garden path was encountered which shows remarkable resemblance with the P600 effect in the ERP.

(1a) Die Rednerin hat den Berater beim Kongress gesucht
The speaker [AMBIG] has [the advisor][SUB] at the congress sought
'The speaker has sought the advisor at the congress'

(1b) Die Rednerin hat der Berater beim Kongress gesucht
The speaker [AMBIG] has [the advisor][OBJ] at the congress sought
'The speaker has been sought by the advisor'

Chomsky N. (1981) Lectures on Government and Binding: The Pisa Lectures. Mouton de Gruyter.
Hagoort P. (2005) On Broca, brain, and binding: a new framework. Trends in Cognitive Science 9(9):416 - 423.
Lewis RL (1998) Reanalysis and limited repair parsing: Leaping off the garden path. In: Fodor and Ferreira (1998), pp 147 - 285.
Mizraji E. (1989) Context-dependent association in linear distributed memories. Bulletin of Mathematical
Biology 51(2):195 - 205.
Smolensky P. (1990) Tensor product variable binding and the representation of symbolic structures in connectionist systems. Artificial Intelligence 46:159 - 216.