Research Information System
IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS, vol.24, no.6, pp.1695-1702, 2020 (SCI-Expanded)
https://ieeexplore.ieee.org/document/8933102
In the present article, a novel emotional complexity
marker is proposed for classification of discrete emotions induced by affective
video film clips. Principal Component Analysis (PCA) is applied to full-band
specific phase space trajectory matrix (PSTM) extracted from short emotional
EEG segment of 6 s, then the first principal component is used to measure the
level of local neuronal complexity. As well, Phase Locking Value (PLV) between
right and left hemispheres is estimated for in order to observe the superiority
of local neuronal complexity estimation to regional neuro-cortical connectivity
measurements in clustering nine discrete emotions (fear, anger, happiness,
sadness, amusement, surprise, excitement, calmness, disgust) by using
Long-Short-Term-Memory Networks as deep learning applications. In tests, two
groups (healthy females and males aged between 22 and 33 years old) are
classified with the accuracy levels of 68.52% and 79.36% through the proposed
emotional complexity markers and and connectivity levels in terms of PLV in
amusement. The groups are found to be statistically different (p << 0.5)
in amusement with respect to both metrics, even if gender difference does not
lead to different neuro-cortical functions in any of the other discrete emotional
states. The high deep learning classification accuracy of 98.00% is commonly
obtained for discrimination of positive emotions from negative emotions through
the proposed new complexity markers. Besides, considerable useful
classification performance is obtained in discriminating mixed emotions from
each other through full-band connectivity features. The results reveal that
emotion formation is mostly influenced by individual experiences rather than
gender. In detail, local neuronal complexity is mostly sensitive to the
affective valance rating, while regional neuro-cortical connectivity levels are
mostly sensitive to the affective arousal ratings.
DISCUSSION
AND CONCLUSION
In the present study, a new emotional recognition
methodology has been presented. The close relationship between affective
stimulus parameters and neuro-cortical activities in young females and males in
nine discrete emotional states. For this purpose, PCA is applied to PSTM of
short EEG segments. The primary concept was to observe the gender effect on
emotional neuro-complexity levels, and then, the second concept was to observe
the usefulness of the proposed complexity markers for emotion recognition. In
all applications, EEG measurements were segmented into short epochs of 6 s and
12 s in order to investigate the influence of analysis interval for emotion
recognition. Conventional and deep learning networks were trained in not only
instant classification but also subject classification manners for both
segmentation steps. Regarding those main concepts, the results were compatible
with to each other: Females were differed from males in E4 (amusement) through
CL-4 for both shorter and larger segments. Thus, CL-4 provided the relatively
lower performance for discrimination of E4 from baseline in comparison to
recognition of the other emotional states for both segment statements. For
recognition of E4 (amusement) and E8 (calmness), classification performances
were increased when the subjects were classified through CL-4 instead of instant
classification. When the analysis interval was largened from 6 s to 12 s,
emotion recognition performances were decreased through all classifiers.
Considering emotional EEG complexity levels mediated by audio-visual affective
video films, gender differences became slightly un-avoidable when subjects were
classified instead of instant classification. Similarly, each emotional states
were differed from baseline with very high CA levels when subjects were
classified instead of instant classification. Among conventional and deep
learning networks, the most useful classifier was CL-4, i.e., CNN. Although,
SVM classifiers provided slightly better performance when the number of features
was low, deep learning algorithms, CNNs and LSTMNs were found to be better when
the large number of features were examined. In conclusion, mixed emotions
highly modulate the functional connectivity of the amygdala with the other
regions of the brain. In particular, regional PCPSTM estimations characterize
the dynamic signature of emotion formation depending on individual experiences
driven by ongoing perception and cognition processes. From EEG signal
processing point of view, primary extraction of PSTM reduces the background
EEG, i.e., increases the signal-to-noise ratio. Then, application of PCA on
PSTM highlights the main harmonics in association with audio-visual evoked
potentials embedded in short epochs. Due to ongoing combination of excitatory
and inhibitory post-synaptic potentials as well as Action Potentials (APs), EEG
series are time-varying psychophysiological signals. In particular, timevarying
audio-visual affective stimuli continuously cause both generation and
propagation of nerve APs at auditory, visual and cognitive cortices
simultaneously. Therefore, EEG segmentation and analysis of short
non-overlapped epochs are crucial pre-processes for emotion recognition. As
well, the most useful length of short epoch is found to be 6 s due to AP phases
lasting about 2 s. In recent neuroscience studies, it is highlighted that emotional
responsiveness of individuals can be clinical support in not only rare disease
so called pre-symptomatic Huntington’s disease [101] but also several important
and widespread psychiatric conditions such as unipolar depression [102],
Parkinson’s disease with lack of dementia [103] and autism spectrum [104] as
well as amygdalar lesions [105]. Full-band PSTMCs can be proposed as
single-channel emotion recognition system.