Explainable Graph and Signal Processing methods for Multimodal EEG-NIRS Brain Network Modeling in Language Disorders

Abstract

This thesis presents a comprehensive methodological framework for the analysis of neural time series data through signal processing, complex network theory, and machine learning, with a particular focus on understanding and diagnosing language disorders, especially developmental dyslexia. Using non-invasive brain recording techniques, such as EEG and fNIRS, advanced signal processing methods were explored to extract meaningful information from neural time series. This information enables the measurement of functional coupling and connectivity, which are central to this thesis because they provide insights into the neural activity underlying cognitive processes. Subsequently, based on the measured neural coupling, these cognitive processes are modeled by means of complex networks, which allows analyzing their characteristics and topologies in order to find differential patterns. Beyond identifying differential patterns, this thesis emphasizes the importance of gaining a deeper understanding of the neural underpinnings of developmental dyslexia through explainable methods. In addition to graph-based modeling, a novel approach is introduced for transforming EEG signals into image sequences, thereby enabling the visualization of neural activity and coupling based on mechanisms such as cross-frequency coupling (CFC). Moreover, explainability techniques, including the development of brainSHAP maps, have been employed to enhance the interpretability of machine learning models and provide insights into the spatial contributions of different brain regions. Furthermore, a multimodal strategy is proposed that integrates EEG and fNIRS data based on neurovascular coupling, facilitated by the aforementioned image transformation. Finally, this thesis focuses on the development of methods for the analysis and exploration of neural time signals, aiming to identify patterns linked to cognitive processes while ensuring a high degree of explainability.