08M.S. Thesis · Biomedical AI

EEG–fMRI Fusion for Neural Decoding

A multimodal neuroimaging framework predicting visual stimuli by fusing 70-channel EEG with 3T fMRI (Wakeman–Henson dataset) using temporal convolutional networks.

University of Cincinnati · M.S. thesis · advisor Prof. Vikram Ravindra · 2022 — 2024

84.8%
within-subject accuracy
81.1%
cross-subject (LOSO)
0.93
ROC-AUC (within)
0.90
ROC-AUC (cross)

In short

  • SSS, ICA/wavelet-ICA, and DWT preprocessing; MNI-normalized fusiform/occipital ROIs.
  • Beat GRU, LSTM-CNN, and SVM baselines — vs 65.5% EEG-only and 74.6% fMRI-only.
01

Question

EEG has millisecond temporal resolution and poor spatial localization; fMRI is the reverse. Can a fusion model decode which visual stimulus a subject saw better than either modality alone?

02

Method

  • 70-channel EEG plus 3T fMRI from the Wakeman–Henson dataset.
  • Preprocessing with signal-space separation, ICA and wavelet-ICA artifact removal, and discrete wavelet transform features.
  • MNI-normalized fusiform and occipital ROIs for the fMRI stream.
  • Temporal convolutional networks over the fused representation.
03

Result

84.8% within-subject and 81.1% cross-subject (leave-one-subject-out) accuracy, ROC-AUC 0.93 and 0.90 — against 65.5% EEG-only and 74.6% fMRI-only, and ahead of GRU, LSTM-CNN, and SVM baselines.