Blackboard Architecture simulation with simple biological networks captures the behavior of diverse neuroimaging measurements during language processing


1. Cognitive Neuroimaging Unit, CEA DSV/I2BM, INSERM, Université Paris-Sud, Université Paris-Saclay, NeuroSpin center, 91191 Gif/Yvette, France
2. Institute for Artificial Intelligence and Biological Systems. School of Computing. University of Leeds. LS2 9JT Leeds. United Kingdom

An important challenge in neurolinguistics is to understand the parsing process by which words are combined into larger constituents during sentence understanding. Few attempts have been made to model parsing with biological neural networks. The Neural Blackboard Architecture (NBA), proposed by Van der Velde and De Kamps(Velde 2006) is one of them. It was designed to answer many challenges in the neural modeling of sentence processing, including the ones detailed by Jackendoff(Jackendoff 2002). Illustration of the abstract circuit specification given by the NBA, to represent and parse utterance into tree structures, is shown in Figure \ref{Blackboard}. Here we expand on previous simulations of the Blackboard Architecture(Velde 2015) on leaky-integrage-and-fire (LIF) populations with population density techniques(de Kamps 2013) implemented in MIIND(Kamps 2008), to compare simulated time courses of neural activity associated to sentence parsing with functional magnetic resonance data (fMRI) and intracranial recordings (electro-corticography; ECOG).

Our simulations suggest that the neural dynamics of the simulated circuit of LIF neural populations, without tuning of the circuit parameters, already approximate the qualitative behavior of several neuroimaging measurements. In the case of Bold-fMRI measurements, we considered the work of Pallier et al(Pallier 2011). Manipulating the size of constituents in sequences of words presented visually to participants, they observed a sublinear increase of the amplitude of hemodynamic responses in language related regions as a function of constituent size. We confirmed that simulating the same phrases under a phrase grammar theory with a simple bottom-up parsing scheme leads to the mentioned pattern. WHAT ABOUT UNIVERSAL DEPENDENCIES In Figure \ref{pnas} we portray the simulated neural time courses and their respective hemodynamics. We also show how our obtained amplitudes compare with those in a region of the posterior superior temporal sulcus regions (pSTS). In the case of ECOG recordings, recent work from Nelson et al. (under review), provides evidence the the Local Field Potentials have increase with phrase constituent size and drop after binding of words into constituents. We confirm both qualitative properties from a preliminary simulation of right branched phrases of increasing number of words. In Figure \ref{nelson} we show the contrasted neural time courses of our simulation and a plot taken from Nelson et al.(under review). We also depict the LIF neural populations time courses inside a compartment circuit of the Neural Blackboard Architecture that originates the sudden drops of neural activity during constituent binding.

Currently we are developing a systematic link between the parameters of the implemented circuit and the measurements to be able to make quantitative predictions to investigate natural clusters of neural activity of phrases taken from a corpus. We hope to determine candidate sets of maximally different phrases under diverse grammar theories and parsing schemes to compare these theories in posterior experiments.

\label{Blackboard} Blackboard architecture.

A. Gating circuit that allows the implementation of conditional neural activity transfer between Neural assemblies X and Y through a gate assembly. The gate keeper assembly initially is also activated by the X assembly and then inhibits the gate assembly, so a control assembly has to inhibit the gate keeper assembly to let information flow through the gate assembly. B. Architecture of one compartment circuit of a connection matrix is shown. Six gating circuits are arranged such that conditional bidirectional neural activity flow is possible between two main assemblies. Control assemblies regulate the direction of information flow and allow the activation of sub assemblies. The two sub assemblies excite the working memory assembly which once activated encode the binding of the main assemblies and allow activation to flow between them if the controls allow it too. C. Each connection matrix contain n by m compartment circuits that encode the same relationship type between the same pair of assembly categories. There are m available assemblies for one category and n available assemblies for the complementary category and only one cell circuit can activate its working memory assembly to link two particular assemblies due to mutual row and column inhibition of cells in the connection matrix. The size of the connection matrix effectively represents memory limitations. A blackboard is composed of an arbitrary number of connection matrices that encode different relationship types for a pair of assembly categories. D. A blackboard is composed of multiple connection matrices, where each of them is defined by two node categories and a relationship type between them. E. Example of one possible tree structures out of the infinite that can be represented based on the specified connection matrices.

\label{pnas} Simulation of Pallier et al language stimuli.

A. The neural time courses normalized by the maximum firing rate of all phrases in the simulation and their corresponding hemodynamic responses. There are six conditions that correspond to 12 words presentation, where constituents of phrases in the word stream vary in size, from 1 to 12 word constituents. Horizontal black bars in the hemodynamics signal the maximum hemodynamic response for a condition and the vertical bars correspond to the time at which this maximum is achieved. B. The maximum amplitudes of the simulated hemodynamic responses for the different conditions in the same logarithmic scale of constituent size employed in the Pallier et al. plots. Contrasted against pSTS region as reported in Pallier et al.