Decision Fusion of EEG and fNIRS Signals

Abstract

In this study, we investigated the use of multimodal functional neuroimaging in detecting mental stress on the prefrontal cortex (PFC). We recorded Electroencephalography (EEG) and functional Near Infrared Spectroscopy (fNIRS) simultaneously from 20-subjects performing mental arithmetic task under control and stress conditions. Stress was induced in this study based on two established stressors – time pressure and negative feedback about peer performance. We explored decision fusion by using support vector machine classifier for each modality, and optimizing the classifiers based on Receiver Operating Characteristic (ROC) curve values. Experiment results revealed significant reduction in alpha rhythm and mean change in concentration of oxygenated hemoglobin at PFC when stressed, p<0.001 and 0.0001 respectively. The decision fusion improved significantly the detection rate of mental stress by +7.76% and +10.57%, when compared to sole modality of EEG and fNIRS, respectively.