1. Introduction
Emotions have been found to have a significant influence on the psychological and physical well-being of human beings [
1]. The sharing of emotions by patients and how well they are received by therapists’ aid in the assessment of good treatment options. A huge amount of data is processed by therapists over a long period of time, which is quite a tumultuous task [
2]. Thus, if a speech-based emotion training process is made feasible, it would be highly beneficial to therapists, as it would save a lot of time and energy. For instance, voice samples portraying different emotions, such as sadness, surprise, joy, anger, and neutral mode, could be considered and used to train a neural network to allow for their recognition [
3]. Without the need for intrusive technology, spoken audio signals can be examined so that emotional information can be obtained. Using various applications, emotions have been grouped into text and audio based on how they evolve with time [
4]. For human communication, emotion serves as an utmost important skill so that interpersonal connections can be managed well. With the help of emotional inputs, multiple cognitive computing tasks are achieved, and they sometimes greatly aid in perception and rational thinking. Some versatile examples of emotion classification and recognition include banking, computer games, video and audio monitoring, and psychiatric diagnosis [
5]. Emotional speech recognition can be utilized for entertainment purposes, clinical investigations, online learning, and corporate applications. When voice signals are combined with emotions, it can create a holistic method called emotion recognition from speech. This method was chosen in this research as it has a negligible cost and is more appealing and useful than processing other biomedical signals [
6,
7]. In short, SER is widely used to trace and identify emotions in human speech signals. This technique has received a lot of attention in the last decade, as it bridges the gap and serves as an important link in human–computer interfaces [
8]. SER has been applied for drowsiness detection, emotional state assessment, sleep stage scoring, medical diagnosis, etc. In the past few years, various SER studies have been conducted, primarily based on acoustic features [
9]. They include assessments of voice quality, spectral features, prosodic features, etc., implemented for emotion recognition. Earlier studies used either one type of acoustic feature to identify emotions or multiple features to identify emotions, supported by classification with traditional machine learning or deep learning classifiers [
10]. Some of the common works carried out in the past few years are discussed below. In this work, experiments were conducted on four publicly available datasets, namely, EMOVO [
11], RAVDESS [
12], SAVEE [
13], and Berlin Emo-DB [
14], and results previously reported on these datasets are discussed specifically for performance comparison analyses.
Regarding the EMOVO dataset, the following techniques and results have been reported: speaker awareness for SER was analyzed, obtaining a classification accuracy of 68.50% [
15]; automatic feature selection techniques were applied for emotion recognition in low-resource settings, obtaining a classification accuracy of 41% [
16]; novel feature selection techniques were applied for entire recognition, obtaining a classification accuracy of 60.40% [
17]; and the transfer learning concept was applied to enhance SER where the knowledge gained through one task or dataset is utilized to improve the model performance on another related task or different dataset, obtaining a classification accuracy of 76.22% [
18].
Regarding the RAVDESS database, the following techniques and results have been reported: machine learning techniques were implemented, obtaining a classification accuracy of 80.21% [
19]; convolutional neural networks (CNNs) were implemented, obtaining a classification accuracy of 79.50% [
20]; multimodal SER using CNN was implemented, obtaining a classification accuracy of 78.20% [
21]; spiking neural networks were implemented, obtaining a classification accuracy of 83.60% [
22]; the capsule routing technique was implemented, obtaining a classification accuracy of 77.02% [
23]; bagged SVM was implemented, obtaining a classification accuracy of 75.69% [
24]; spectrogram-based multi-task audio classification was carried out, obtaining a classification accuracy of 64.48% [
25]; frequency cepstral coefficients with neural networks were applied, obtaining a classification accuracy of 79.80% [
26]; and continuous wavelet transform (CWT) was applied, obtaining a classification accuracy of 60.10% [
27].
Regarding the SAVEE dataset, the following techniques and results have been reported: a hierarchical classifier was used for SER, obtaining a classification accuracy of 83.78% [
28]; joint deep cross-domain transfer learning was used, obtaining a classification accuracy of 69.00% [
29]; the concept of negative emotion recognition was used, obtaining a classification accuracy of 65.83% [
30]; 3D-CNN-based SER was carried out using K-means clustering and spectrograms, obtaining a classification accuracy of 81.05% [
31]; recurrence dynamics were integrated for SER, obtaining a classification accuracy of 80.20% [
32]; a comparison of cepstral features for SER was carried out, obtaining a classification accuracy of 78.60% [
33]; and hybrid Particle Swarm Optimization (PSO)-based biogeography optimization was carried out for emotion and stress recognition, obtaining a classification accuracy of 78.44% [
34].
Regarding the Berlin Emo-DB database, the following techniques and results have been reported: a two-layer fuzzy multiple random forest concept obtained a classification accuracy of 87.85% [
35], a modified quantum-behaved PSO obtained a classification accuracy of 82.82% [
36], a wavelet packet analysis obtained a classification accuracy of 79.50% [
37], multi-time-scale convolution obtained a classification accuracy of 70.97% [
38], voting mechanisms obtained a classification accuracy of 64.52% [
39], the stacked generalization method obtained a classification accuracy of 82.45% [
40], and the divide-and-conquer-based ensemble technique obtained a classification accuracy of 82.00% [
41].
The main contributions of this work are as follows:
- (a)
After the basic pre-processing of the signals is carried out using an Independent Component Analysis (ICA), the pre-processed signals are subjected to six transforms, and then features are extracted.
- (b)
The extracted features are selected using three efficient techniques, namely, OIFS, HHO, and CSA, and they are finally fed into machine learning classifiers.
- (c)
While the techniques may already be established, the combinations with which the workflow proceeds are completely novel and interesting, as no previous works have reported the combination of transforms with optimization techniques, followed by classification with machine learning classifiers. A simplified version of a block diagram for a holistic understanding is presented in
Figure 1.
2. Implementation of Transforms
The following six important transforms were applied to pre-processed speech emotion signals: the synchrosqueezing transform, FST, KSTDIS, FAWT, chirplet transform, and superlet transform. Some of the advantages considered when choosing these transforms are that they should be computationally very fast and these transforms should be able to explore the fine and innate details of the signal clearly. Some of the transforms can also provide a good and simultaneous localization in both time and frequency domain. The implementation aspects should also be easy so that it could ease the burden of the researchers to a great extent. The implementation of these transforms is described below.
2.1. Synchrosqueezing Transform
The synchrosqueezing transform is highly dependent on the continuous wavelet transform (CWT) [
42]. This transform is highly useful for obtaining the localized time–frequency specifications of non-stationary signals and the oscillation components of a signal. The basic form of the signal
is indicated as follows:
where every component in
represents an oscillation function with time-varying frequency and amplitude, and the noise is specified by
. For every component in
, the intention is to obtain the amplitude
and the instantaneous phase
. Hilbert transforms are applied to the original
component, and the formation of the analytic signal
is obtained as follows:
For the multicomponent signal,
is extended by means of a vector construction to every analytic signal of the channel; therefore, a multivariate analytic signal is obtained as follows:
Every speech emotion is represented as a monocomponent signal, as shown in Equation (3). For the multicomponent signal
, the highly localized time–frequency indication is represented by the SST algorithm utilizing the instantaneous amplitude. By utilizing the CWT, the frequency information is extracted from the analytic signal. The oscillatory component of a particular signal is imbibed by the CWT algorithm via the use of time–frequency filters termed wavelets. The finite oscillation function is represented by the mother wavelet
, and it is usually convolved with the signal
. The computation of the CWT of
is carried out as follows:
Here, the wavelet coefficients for every scale–time pair
are indicated by
. The bandpass filter output consists of only the wavelet coefficients. In the frequency domain, the bandpass filter is scrolled by the scale factor
so that the bandwidth of the filter changes. The scale parameter
highly influences the frequency properties, while the time of interest is indicated by the shift parameter
. Therefore, at a particular frequency of
, the spreading of the energy of a sinusoidal wavelet transform occurs around a scale factor and is represented as
. Here,
indicates the wavelet’s center frequency, and the energy of the original frequency
is widely spread across
. The actual frequency
and the estimated frequency in the relevant scales are similar. For every scale–time pair
, the evaluation of the instantaneous frequency
is carried out as follows:
The wavelet coefficients are transformed from the scale–time domain to the time–frequency domain using the SST technique. Every point
is transformed into
. A scaling step is required, as
and
are discrete values. The computation of
for any
, where
, is carried out. When the domain transformations are carried out (from the scale–time domain to the time–frequency domain),
, the computation of the SST coefficient
is carried out at the center
of the frequency range
, with
. At a resolution of
, the frequency bias is indicated by
. The linear frequency scales are specified by
, and this acts as a constant value. The oscillations in the univariate mode are reconstructed using SST so that
. The reconstructed signal
and the synchrosqueezing coefficient are expressed as follows:
The time–frequency indication of the signal is represented by Equation (4) and is synchrosqueezed along with the frequency scale. In Equation (7), represents the normalization constant. For the mother wavelet , represents the Fourier transform. To maintain a consistent indication in the time–frequency domain, the coefficient of the CWT is reallocated using the SST algorithm so that instantaneous frequencies are automatically generated. The multivariate synchrosqueezing transform is explained as follows: The expressive indication in the time–frequency domain is provided using the modulated oscillation model. A multivariate extension is presented to the synchrosqueezing transform so that the common oscillations pertaining to the multiple data channels are recognized. For multivariate data such as speech emotion data, the procedure is started by applying the SST to every channel. The synchrosqueezing coefficients are initially obtained, and then the multiple channels are assigned depending on the rules. From a collection of multivariate signals, a group of monocomponent signals are identified, and the separation of the time–frequency domain components into frequency band components is carried out as . Assessments of the instantaneous frequencies and amplitude are conducted. The computation of the multivariate instantaneous frequency and amplitude is carried out across the channels, and, ultimately, an assessment of the multivariate synchrosqueezing coefficient is conducted.
2.2. Fractional Stockwell Transform (FST)
Among the prevalent joint time–frequency analysis tools, one of the most famous techniques is the FST, as it helps to improve the time–frequency concentration to a greater extent [
43]. Here, two widely used transforms are combined—the fractional Fourier transform (FrFT) and the Stockwell transform (ST). For the signal
, a
-order continuous and discrete FST is represented mathematically as follows:
and
where the kernel function of the FrFT is expressed as
, which is expressed as follows:
The scalable Gaussian window function is expressed as
; it is dependent on time
and the fractional Fourier frequency
, and it is represented as follows:
where
and
. There are two window adjustment blocks of parameters alongside the fractional order parameter
, namely,
and
, which help to mitigate the window shape. The energy of the signal is localized using these parameters so that the time–frequency resolution is improved. The calculation of the FST is carried out as follows:
In the context of chirp basis functions, the Gaussian-windowed signals are decomposed with respect to the phase and time, and they are highly influenced by different values. In the time–frequency plane, it is best to analyze the non-stationary signals, as they are more interpretable and robust. As it provides a good frequency and time resolution, the ST remains a very famous window-based technique among the joint time–frequency analysis tools. At lower frequencies, the frequency resolution is improved, and, at higher frequencies, a good time resolution is provided. Despite having such versatile features, it has also certain shortcomings. The time resolution is sometimes degraded, as the Gaussian window function is quite lengthy with tapering ends; therefore, the pertinent data present in the signal are suppressed. The window shape parameters in the ST cannot be adjusted, as they are highly dependent on the frequency. Therefore, to improve performance, these limitations must be overcome successfully, and, to this end, a fractional variation of the ST called the FST is used in this work. To eliminate noise, the fractional Fourier domain is quite effective when dealing with non-stationary signals. The noisy counterparts can be separated in the intermediate time–frequency domain using the fractional order of the transform, and a better resolution can be obtained using the ST. The extension of the ST in the fractional domain is aided as the signal localization is improved, thereby enhancing the flexibility of the entire spectrum of the signal. Thus, to process the noisy content of the signals, FST is utilized. A scalable fractional Gaussian window is utilized by this modified variant of the ST in the fractional domain with the help of the fractional parameter and two window adjustment parameters and . A good improvement can be seen in the resolution when using these two window adjustment parameters and fractional parameter. The rotation angle can also be changed so that even multi-resolution spectrograms can be achieved, which are sometimes large, thereby improving the flexibility.
2.3. K-Sine Transform-Dependent Integrated System (KSTDIS)
A K-sine transform-dependent integrated system is described below to investigate its effectiveness [
44]. The sinusoidal transformation exhibits a very powerful nonlinear dynamic behavior and it can be implemented to chaotic systems quite easily. With the help of the sine map, the chaos of a chaotic system can be enhanced as follows:
The K-sine map is dependent on the sine map, and it is expressed as follows:
The chaos of the function is enhanced by
with the mitigation of periodicity of the trigonometric function. If the value of
, then
becomes equivalent to
. The chaotic property of
is higher than that of
, and it is specified with the help of the Lyapunov exponent [
45] and sample entropy [
46]. The numerical features of adjacent trajectories are indicated by the Lyapunov exponent, and this aids in tracing the chaotic motion. To identify the chaos of a system, the LE is highly useful. For a variable chaotic system,
, and its Lyapunov exponent is expressed as follows:
The system progresses towards a chaotic state if
, and such a mapping is termed a chaotic mapping. If the
value is large, then the chaotic behavior is quite complex in nature. To assess the time series complexity, sample entropy is highly useful when compared with approximate entropy. The time series complexity is assessed using sample entropy, and this is carried out by specifically measuring the probability of a normal pattern in the time sequences. The complexity will be higher if the probability of the generated new patterns is high. The complexity will be lower if the sample entropy value is small and the time series self-similarity value is high. If the sample entropy value is large, then the time series will be more complex. Based on
, this study proposes the use of KSTDIS, and its mathematical model is represented as follows:
where
and
are two known 1D chaotic map seeds. The control parameters are
and
,
, and the absolute value is denoted by
. The coupling operation is performed on
and
. Then, sine transformation is performed on the coupling result with the help of
. The chaotic dynamics can be easily absorbed by the coupled operation. The merits of the nonlinear dynamic behavior of the sine transform are greatly taken care of, and, thus, the arbitrary selection of seed maps can be carried out so that novel chaotic maps larger in number can be generated. The effectiveness of the KSTDIS must be proven. Here,
is considered to have a value of 3, and the chosen seed chaos is a logistic map, a sine map, and a tent map.
Then, the control parameters for all the maps are the variables
and
. The mathematical model for the 3-KSTDIS is expressed as follows:
Here, the three seed mappings are not combined, as the chosen value of is only 3.
2.4. FAWT
To analyze speech emotion signals, the analytic wavelet transforms with flexible time–frequency covering is utilized [
47]. The Hilbert transform pairs of atoms are utilized by this transform so that a higher flexibility is provided, thereby controlling the dilation factor, redundancy, and Quality Factor (
QF). The signals can be easily analyzed by adjusting the parameters: the down sampling factor for the low-pass channel (l) and high-pass channel (h) and the up-sampling factor for the low-pass channel (u) and high-pass channel (p). The parameter that helps to ascertain and control the
QF is expressed as follows:
By implementing the iterative filter bank structure, the level of the FAWT is obtained. At every level, the decomposition in this technique provides dual channels and a single channel, which corresponds to high-pass channels and a low-pass channel, respectively.
The frequency response of the low-pass filter is expressed as follows:
The frequency response of the high-pass filter is expressed as follows:
where
is expressed as follows:
The following conditions should be fulfilled so that a proper reconstruction is allowed:
Without using the chirp parameters, the parameter of the FAWT is deduced. The value of is set to 0.5, and the QF is set to 2 in our experiment.
2.5. Chirplet Transform
For non-stationary signals, the time–frequency domain specifications are expressed well with the chirplet transform [
48]. Information on the amplitude variation with time is expressed by the time domain signal. Information on the amplitude variation with frequency is expressed by the frequency domain signal. With respect to both frequency and time, information on the variation in energy or amplitude is expressed by the CT. The generalized specification of the STFT and CWT is the chirplet transform, and it has numerous applications in the fields of biosignal processing, image processing, etc.
is a speech emotion signal with
, and the total number of samples present in the speech signal
is represented by
. Then, the chirplet transform of
is expressed as follows:
where
represents the analytic signal of
and is evaluated using the Hilbert transform. The analytic signal of
is expressed as
, where the Hilbert transform of the speech emotion
is represented by
. The complex conjugate of
is represented by the factor
. In the chirplet transform, the window function is
, and it is expressed as follows:
where the Gaussian window function is represented by the factor
and is expressed as follows:
By utilizing the chirplet transform of the speech emotion signal, the time–frequency matrix is obtained, and it is mathematically expressed as follows:
where the real part of the time–frequency matrix
is expressed as
, and the imaginary part of the time–frequency matrix
is expressed as
. For the time–frequency matrix, the magnitude is expressed as follows:
2.6. Superlet Transform
A collection of wavelets with a high bandwidth is used by the superlet transform [
49]. The excellent temporal resolution of the wavelength is combined geometrically so that a high frequency resolution is attained. A time–frequency analysis of the signal is performed using the STFT, and, hence, both time and frequency representations of the signals are specified. A quick localization of both the time and frequency is not attained by the generated features of the STFT because of the Heisenberg uncertainty principle. The drawbacks of the STFT are mitigated by the CWT, and, thus, at higher frequencies, a good temporal resolution is achieved [
50]. Thus, DWT is utilized where the specification of the frequencies is determined as a power of two. There are only a limited number of constraints in both the STFT and CWT. A good frequency resolution is provided by the STFT, and a good temporal resolution is provided by the CWT [
51]. At higher frequencies, a good frequency resolution with poor temporal resolution is provided by STFT, whereas a good relative temporal resolution is maintained by CWT throughout the spectrum but it can easily degrade in frequency resolution and becomes redundant with increasing frequency [
52]. To obtain a high time–frequency resolution, different wavelets are utilized by the superlet transform. A good temporal but low frequency resolution is provided by wavelets with a smaller number of cycles, and a high frequency resolution is obtained by wavelets with a greater number of cycles, but a degraded temporal resolution is obtained. To obtain a super-resolution, both high temporal resolutions and high frequency resolutions are combined by the superlet [
53]. For various applications such as biosignal processing, image processing, wireless communication, and electrical signal processing, superlets have been widely used. The Morlet is utilized as a mother wavelet by the superlet transform, which expresses a multi-resolution spectro-temporal specification. One of the most complex wavelets is the Morlet wavelet, and it is expressed as follows:
where the central frequency of the mother wavelet is expressed as
. The Morlet wavelet is almost like the Gabor transform in terms of its process. The scaling parameter helps to scale the window function in the Morlet transform. In the Gabor transform, the window size is already prefixed. The modified Morlet implemented in the superlet is expressed as follows:
The displacement parameter is expressed as
, where the time variance of the wavelet is regulated as follows:
It is generally inversely proportional to the frequency. A broad frequency response is obtained if the value of
is small. A narrow frequency response is obtained if the value of
is large.
is adjusted so that the wave covers the full cycles within the standard deviation
of the Gaussian envelope. Different wavelets are used by the superlet with
so that a better time–frequency representation is obtained. The mathematical specification of the superlet is as follows:
The order of the superlet that manages the wavelet is expressed by
. For every wavelet, the number of cycles is represented by
in the superlet. In the wavelets of the superlet transforms, the number of cycles is selected additively or multiplicatively. The number of cycles is utilized and selected by making use of the multiplicity concept as follows:
Here,
. As far as the individual wavelet response is concerned, the response of the superlet to the speech emotion signal
is specified by the geometric mean, and it is expressed as follows:
Here, the response of the wavelet of is represented as .
For the Morlet, it is expressed as follows:
The speech emotion signal is represented by , and the complex convolution is represented by . Superlet transforms are almost like CWT, except for the fact that the superlet transforms utilize superlets instead of wavelets, which are utilized in the CWT. A CWT is explained as a superlet that includes superlets of order 1. Superlets with a higher order provide a good indication of the signal. For a wide frequency, adaptive superlets are preferred. If the adaptive superlet is used with an order higher than 1, then a good improvement in the time–frequency domain of the signal is obtained.
3. Feature Selection Approaches
An important step in the machine learning pipeline is automatic feature selection. It involves selecting the most important features while eliminating the least important ones. To enhance the performance of any machine learning model and to reduce the computational overhead, automatic feature selection techniques are utilized. Novel feature selection techniques utilize new ideas to extract these relevant features and can employ a plethora of techniques to do it successfully. Using three various approaches—two filter approaches and a wrapper approach—feature selection can be carried out [
54]. Filter algorithms employ the inherent properties of the dataset to estimate the priority of features. Wrapper algorithms assess the performance of a particular classifier with respect to the chosen features. Hybrid methods use the techniques of both the wrapper and filter algorithms; therefore, the number of candidate features is reduced. The general wrapper algorithm is implemented by using an optimization algorithm with a classifier. Depending on the training results of the classifier, the feature combination elements are fine tuned. The training phase and the optimization algorithm are time consuming, so wrapper algorithms sometimes seem to be less suited for high-dimensional datasets. Hybrid techniques that involve the combination of both the filter and wrapper algorithms are also time consuming and can be computationally expensive. When comparing wrapper and hybrid techniques, filter algorithms have several advantages, such as easy implementation and high efficiency; thus, in this work, initially, Overlapping Information Feature Selection (OIFS) is used to choose the features depending on the overlapping of samples concept to improve the feature selection efficiency [
55]. Geometrical separation is ensured for every pair of categories in at least one feature dimension. Using an approximate equation, the computation of the overlapping matrices of every feature is analyzed. Global features are projected as the features that have the lowest overlapping rate. Then, the pair of categories that cannot be split by the global features is identified, and the pair with the lowest overlapping rate is considered. Following OIFS, the HHO algorithm and CSA algorithm are also utilized for the efficient selection of features in this work.
3.1. OIFS Method
For every class, the boundaries must be identified, and this serves as the main objective of a classification problem. When there is a geometrical distribution among the various boundaries of the classes, the problem becomes a straightforward one. Here, the overlapping rate of the sample is considered an important factor for the feature selection method [
55]. If one feature is used to geometrically separate two classes, then a low overlap between these two classes is implied, thereby facilitating the easy discrimination of the features. For each pair of classes, if an appropriate feature satisfies the above condition, then the complexity is reduced. The approximation of the overlapping rate between two classes is expressed as follows:
where
and
represent the sample means,
and
represent the standard deviation of class
and class
, respectively. For the
class classification issue, the overlapping matrix of feature
is expressed as follows:
The overlapping matrix is considered symmetric in nature, and, using Equation (45), all the values of every element can be derived. It is not necessary to compute the diagonal elements, as they specify the overlapping rate of every class by itself. A feature set comprises
. Depending on these matrices, the features are selected. To distinguish every pair of classes, at least one feature is required. Overlapping matrices generally have a
structure. Training the overlapping matrices is the best method for choosing a feature for every pair of classes. The corresponding feature of the overlapping matrix is then selected. Ultimately, in the final feature combination,
features are chosen. Some redundant features may exist in the chosen combination. There are two important phases of OIFS, and they aim to mitigate the redundancy of feature combination. The first phase consists of choosing the global features so that multiple pairs of classes can be clearly discriminated against and computational efficiency is easily improved. This process can also aid in easy classification. Due to the inherent properties, some classes may show blurred boundaries, so local features must be used. The next phase in OIFS concentrates on choosing the local features so that the classification of two ambiguous classes can be very well designed and facilitated. To choose the global features, the procedure is as follows: The average overlapping rate is computed for every overlapping matrix, and, for the
feature, it is represented as follows:
The average value of the overlapping matrix is computed using Equation (47). Depending on the average overlapping rates, the features are sorted in ascending order. The global features consist of only the top features selected from this procedure. The number of candidates chosen as global features is identified as . The input is converted to an integer using the function . specifies the element of vector . Using the feature, the pair of classes can be easily discriminated. For global features, the overlapping limit is indicated by . Here, the parameters and are considered tuning parameters, and their values are assigned as follows: and . The values of and are increased if the number of features is reduced; thus, the global features are increased, and the local features are decreased. The pair of classes that is not identified by the global features is traced out. Local features are assigned to these pair of classes, and the minimum overlapping rate is computed. In between the two related classes, the determination of a feature with the ability to assign itself to a high overlapping rate or low overlapping rate is ascertained. In the feature set, the minimum overlapping rate between the two classes is recorded so that the local features can be selected. The overlapping rate limit is ascertained and checked to ensure that it always satisfies the criterion. Several iterations are required to choose only the efficient features, thereby eliminating the redundant ones.
3.2. Harris Hawks Optimization
In nature, the associated and relative behavior of Harris Hawks influences the HHO algorithm [
56]. Due to its versatility, it has been proven to be a promising search technique, and it is used to address a plethora of optimization issues. The HHO algorithm has two exploration stages and four exploitation stages. To improve the quality of the results, different intelligent schemes are employed, which utilize a greedy scheme.
3.2.1. Stages of Initialization
In this stage, both the search spaces and fitness are presented. The basic chaotic opposition-dependent initialization technique is used, and all the parameters are assigned values.
3.2.2. Stages of Exploration
In the HHO algorithm, Harris Hawks is tested as a candidate solution. Using two strategies, the fitness is computed depending on the planned prey, and this stage is expressed as follows:
where the new position in the second iteration is expressed as
, and the current position is expressed as
. The random position of the hawk is indicated as
, and the optimal position of the intended rabbit is expressed as
.
and
specify the random numbers within [0, 1], and this is updated in every iteration. The lower bound is represented by LB, and the upper bound is represented by UB. The average position of the
solution is represented by
, and it is expressed as follows:
where the total number of hawks is denoted as
, and the position in iteration
of every hawk is represented as
.
3.2.3. Exploration Stage to Exploitation Stage
In this stage, when the escaping action occurs, the energy of the prey is mitigated; therefore, the exploration stage leads to the exploitation stage, and this is expressed as follows:
where the escape energy of the prey is indicated by
, the starting stage of the energy is represented by
, and the maximum number of iterations is represented by
. The position of the prey is observed by the hawk in the case of
, and this indicates the exploration phase of the HHO. In the case of
, the hawk is in the exploitation phase.
3.2.4. Stages of Exploitation
This stage comprises four important steps: soft surrounding, hard surrounding, hard surrounding, hard surrounding, hard surrounding, hard surrounding with rapid dives, and soft surrounding with rapid dives. A summary of the steps is expressed as follows: A soft surrounding strategy occurs when
and
. The position of the hawk is updated as follows:
where the difference between the position vector of the rabbit and the present position in iteration
is expressed as
. The random jumping power of the rabbit is expressed by
, where the random variable is expressed as
. The hard surrounding strategy is considered if
and
. In this case, the position of the hawk is updated as follows:
A soft surrounding strategy with rapid dives occurs when
and
. In this stage, the prey successfully escapes, and the hawks must make a collection decision; this is expressed as follows:
To model this strategy, the best Levy flight motion-based patterns are utilized by HHO, and they are defined as follows:
where the dimension of the solution is represented by
, a random number vector of size
is represented by
, and the Levy flight motion is denoted by
LF.
LF is expressed as follows:
where
is a random value in the range of [0, 1].
The constant is defined by
and is assigned a value of 2 in our work. The hawk’s position is updated as follows:
where
and
are utilized by using (54) and (55), and both equations refer to the locations of the next new iteration. A hard surrounding strategy with rapid dives is implemented if
and
, and this is expressed as follows:
where
3.2.5. Initialization Based on Chaotic Opposition
The candidate solutions are randomly initialized, and the search process is started with the traditional optimization technique. The solution space of the population is initialized by the implementation of the chaotic map strategy. The convergence speed must be increased, so the chaotic opposition-dependent learning population technique replaces the random initialization solution. An initial state is generated by the chaotic opposition to improve the solution diversity, and this is achieved by utilizing a chaotic map to introduce randomness. Thus, the algorithm is prevented from prematurely converging to a local optimum, and the global search ability is greatly improved.
3.2.6. Simulated Annealing
Simulated annealing is a very famous local search technique and is analyzed as a single heuristic dependent on solid annealing [
57]. The issue of stagnation in the local economy can be surmounted by the application of this approach. With a specific probability, even a worse solution can be accepted by SA. The Boltzmann probability
controls the worst solution/worst neighbor, where the difference between the best solution and generated neighbor fitness is expressed by
. During the search process, the temperature is indicated by
, and this is periodically decreased. The starting temperature is set to
, where the number of attributes for every data point is expressed by
, and the cooling time is computed appropriately.
3.2.7. Operators of Mutation and Crossover
Every Harris Hawk is placed in a random location in the exploration of HHO so that the location of the rabbit can be determined. The exploration stage of the feature space is improved, and the actual location updated point is detected so that the crossover and mutation can be well prevented as described below. The mutation rate is expressed as follows:
where the maximum number is denoted by
, and the current iteration is expressed as
. As the total number of iterations increases, there is a decrease in mutation
, and this happens in a linear manner from 0.9 to 0. During the iteration procedure, the crossover is added between the current solution
and the resultant value of the mutation, and this operation is expressed as follows:
where the value resulting from the mutation is expressed as
, a random number in the range [0, 1] is denoted as
, and the freshly generated solution is denoted by
. The
size in
is denoted as
.
3.2.8. Tournament Choosing
One useful selection strategy is tournament choosing, and it accesses a specific tournament among randomly chosen individuals from a specific object. There are four main phases in this strategy. The population size and tournament are utilized as input values; then, a random number is generated in the range of [0, 1]. This random number is compared with that of the selection probability to adjust the selection pressure. If the fitness value is high, then the best solution is obtained if the tournament size is great; otherwise, the weak solution is selected. Ultimately, diversity is preserved if the tournament choosing strategy is utilized.
3.2.9. KNN Classifier
A versatile classification technique is KNN [
58]. The main concept of this technique is that a particular sample refers to a specific group and the characteristics quite common to the samples present in the group are exhibited. If a high majority of K samples belong to that category, then classification occurs. This technique is used in signal processing, image processing, text classification, financial risk level assessment, etc., as it is quite straightforward, easy to understand, and simple to implement. The distance chosen in this study is the Euclidean distance, and the mathematical equation is as follows:
where the dimension of the sample is represented by
. The sample in the training set is indicated by
, and the sample in the test set is indicated by
. The feature selection method utilizing the optimization algorithm and KNN is shown in a flowchart in
Figure 2.
3.2.10. Parameter Settings of HHO
Experiments were carried out on a Window 10 computer with an i5 core processor and 8 GB memory. When using the KNN classifier, the Euclidean distance metrics’ K value was assigned 8 in our experiment. The results of various optimization algorithms were run over 50 times. The size of the population was set to 25, and the maximum number of iterations was set to 100. The value of was set to 4, and the number of hawks was set to 10. The value was fixed at 0.001, and the value was fixed at 0.95. For the SA algorithm, the particle number was set to 20. The final temperature was set to 0.2, and the cooling plan was set to 0.5.
3.3. Standard Chameleon Swarm Algorithm (CSA)
The foraging behavior of the chameleons is characterized by the mathematical model [
59] described below.
3.3.1. Population Initialization
With a collection of
potential solutions representing the number of chameleons, the position of chameleons
in the search space is randomly initialized, where a possible solution is specified by every chameleon. At a specific iteration
, the position of the
chameleon in the search space is expressed as follows:
where
, the current iteration is represented by
, and the dimension is represented by
. The position of chameleon
in a particular dimension
is represented by
. In the search space, the initial population of the Chameleon Swarm Algorithm is randomly generated depending on the total number of chameleons and the given problem dimension as expressed below:
where the initial vector of chameleon
is represented by
. In dimension
, the lower and upper limits of the search space are specified by
and
, respectively. The random value, which is uniformly created, is represented by
and is in the range of [0, 1]. Predetermined fitness criteria are used to determine the new position solution for every chameleon. For every chameleon, the present position is updated if a solution with a better quality than the current position solution is identified. When the CSA algorithm is simulated, if the current solution quality is high and better than that of the new position, then the chameleon stays in its original position.
3.3.2. Position Update
When searching for the prey, the positions of the chameleons are updated with the help of the position update strategy. The foraging conduct of the chameleon is represented as follows:
where the normal position of chameleon
at distance
and iteration
is represented by
. The present position of chameleon
in dimension
and iteration
is represented by
.
traces the highest and best position achieved by chameleon
in dimension
.
represents the global best position achieved by the chameleon in dimension
.
and
are considered two positive values that help in assessing the exploration behavior. In the interval [0, 1], the random values are denoted
r1,
r2, and
r3. The random value present in index
is denoted by
and is in the range of [0, 1].
denotes the discerning prey of the chameleon, and
is either −0.5 or +0.5 and can highly influence the exploitation/exploration phase. The function of the iterations is denoted as
and is expressed as follows:
where
represents the current number of iterations, and
indicates the maximum number of iterations.
are constant values, with values of 1, 1.5, and 2, respectively, and they help to mitigate the exploitation behavior. Parameter
indicates that the present candidate solution is highly similar to the optimal solution. The probability of a chameleon managing the prey in the environment is represented by parameter
, and
. This condition is fixed so that the present position can be altered based on the prey observation in the search space. When searching for the prey, the locations in the search space are altered by the chameleons in a random manner. To identify prey, the search space is randomly explored in various areas and directions so that a high potential is achieved; thus, the optimal goals can be easily represented.
3.3.3. Update Model Based on the Eye Rotation of the Chameleon
The chameleon possesses an innate ability to perceive the position of prey, as they are able to rotate their eyes to detect prey within a 360-degree range; this gives them special access and power. A chameleon can easily spin and move towards the prey’s position, and the novel position is represented as follows:
where the updated position of the chameleon is represented by
, and the rotation matrix of the chameleon is represented by
. The current position of the chameleon is represented by
, and the center of the chameleon’s present position is represented as
, which is determined as follows:
where the orthonormal vectors are represented as
and
, and they have a size of
. The rotation matrices are indicated by
, and the cycle of rotation of the eyes is denoted by
, which can be expressed as follows:
where
denotes the rotation angle from 0 to 180 degrees, which is in an interval of [0, 1].
is either −1 or +1, and it denotes the rotation direction.
3.3.4. Velocity Update Model
The chameleon terminates the stalking process once the prey is assaulted and it is not far from the chameleon’s position. This chameleon is considered the best among all the chameleons. The position of this chameleon is updated, as it can extend its tongue to twice its length. This helps the chameleons catch the prey efficiently, as it allows them to exploit the search area quickly. In this algorithm, the chameleon’s tongue moves towards the prey rapidly, and the velocity of this can be modeled as follows:
In dimension
, iteration
, and
, the following and current velocities of chameleon
is represented by
and
. The present position of chameleon
is indicated by
. The two random values are denoted by
r1 and
r2 in an interval of [0, 1]. To manage the effects of
and
on the chameleon’s tongue, the position values
c1 and
c2 are considered important. The inertia weight is indicated by
and is determined as follows:
where, to mitigate the exploitation features, the positive values are utilized as
.
When moving towards the prey, the position of the chameleon’s tongue is computed as follows:
where
denotes the former velocity of chameleon
and dimension
. The rate of acceleration is denoted by
, and it reaches an approximate value of 2600 m/s, as shown below:
By randomly generating the chameleon position, the CSA performs optimization and feature selection. Within every iteration, the positions of all the chameleons are consecutively updated. The chameleons are returned to the boundary if they exist in the search space. With the help of fitness functions, solutions are assigned so that the fittest chameleon is identified. In every loop, the steps are reiterated until the iteration condition is met. The chameleon incessantly explores and exploits the search space so that they can move towards the prey and catch it with their tongue. The mathematical model provided above helps to address the optimization issue, even with very large search spaces. The simplified implementation of the CSA algorithm is explained in Algorithm 1.
Algorithm 1: Implementation of CSA algorithm |
The Key parameters of CSA are assigned. |
The position for all chameleons is analyzed using and , then initialized. |
The velocity of falling chameleon’s tongue is assigned. |
Evaluate the starting position of all chameleons |
While (t < T) do |
Update the position of chameleons based on Equation (67) |
Update the position of chameleons depending on their eyes turn merit using Equation (69) |
Update the tongue velocity of the chameleon using Equation (72) |
Update the tongue position of the chameleon using Equation (74) |
Integrate the position of chameleons using and of the problem variables. |
Update the position of every chameleon |
t = t + 1 |
end while |
Return the global best position of the chameleons. |
3.3.5. Parameter Settings of CSA
We conducted experiments on a Window 10 computer with an i5 core processor and 8 GB memory. The fitness was evaluated using fitness-dependent KNN with Euclidean distance metrics, and K was assigned a value of 8. The results of various optimization algorithms were run over 50 times. The size of the population was set to 25, and the maximum number of iterations was set to 100. The value was fixed at 0.99, and the value was fixed at 0.05.
4. Classification Using Machine Learning Algorithms
In this study, ten machine learning algorithms were employed, as described below.
4.1. Random Forest (RF)
Multiple decision trees are constructed randomly by means of sampling with replacement. The bagging technique is adopted by random forest, where many weak learners are integrated to form a strong learner. Weak learners are generated in an independent manner and based on the majority votes of the prediction of the weak learners, the final prediction is obtained.
4.2. Support Vector Machine (SVM)
SVM is a famous machine learning algorithm utilized for classification and regression problems. The margin is maximized around the separation bounds of two classes in a classification problem using a plane, line, or hyperplane to split the data.
4.3. Extreme Gradient Boosting (XGB)
XGB is quite an efficient, portable, and flexible parallel tree boosting system. The generalization issue is overcome by XGB, and the modal complexity is greatly reduced by the addition of a regularization parameter to the objective function.
4.4. Naïve Bayesian Classifier (NBC)
NBC depends on Bayes’ theorem learning technique. The conditional probability of every point is computed based on the conditions of every class. The class of the test data is like a class with a high conditional probability. Every feature has a strong independence, and there is no correlation between them in the NBC.
4.5. Gradient Boosting
Gradient boosting is a famous gradient descent-dependent boosting technique. Here, gradient descent is used so that the errors of the individuals are minimized. In an iterative manner, base learners are constructed by means of reweighting the misclassified instances. In every training observation, the negative partial derivatives of the loss function are operated by the gradient boosting to determine the weights. To obtain a high-performance model, weak learners are combined sequentially.
4.6. K Nearest Neighbor (KNN)
The KNN algorithm is a famous non-parametric technique. The K-nearest training points are used in the test so that the class value of the test points can be easily predicted. KNN is suitable for a high amount of training data, and it is not influenced by the noise of the training dataset.
4.7. Adaptive Boosting (AB)
Adaptive boosting (AB) is a famous boosting classifier, and, here, multiple base decision tree classifiers are combined in order to create a robust classifier with a good classification accuracy. Using weighted data, a sequence of weak classifiers is trained iteratively, and an ensemble is constructed by focusing on prior misclassified cases. Depending on the weighted summation of all weak classifiers, the final boosted classifiers are determined.
4.8. Decision Trees (DTs)
DT comprises a non-parametric tree-like structure with a root, leaf, and internal nodes. The training dataset is divided into many non-overlapping data subsets through the Gini Index. Depending on the attribute value, a partition is made by the root node, features are indicated by the internal nodes, and the outcome is specified by the leaf node.
4.9. Extreme Learning Machine (ELM)
For the training set
, and the number of hidden nodes is represented by
. The following optimization is solved using the standard ELM:
This is subject to
which denotes the output vector of the hidden layer for input
, where all weights
and biases
are randomly selected. A user-defined parameter is expressed as
so that a tradeoff is achieved between the regularization and the empirical error. The optimization problem of the ELM is expressed as follows:
For the output weight vector
, the optimal value is expressed as follows:
where the Moore–Penrose generalized inverse is expressed as
, and the least norm least squares solution is obtained [
60]. For the binary classification issue, the output function of the ELM considering the optimal weight
is analyzed as follows:
In the ELM optimization issue, the least squares loss function is used so that it is made sensitive to noise in the data. When working with imbalanced datasets, poor performance is obtained when using the standard ELM. To manage class-imbalanced datasets efficiently, many variants of the ELM model have been developed. Various schemes are utilized by these variants so that the training points are weighed based on their corresponding class representation. In the past decade, deep learning has also been coupled with the ELM, as the iterative tuning of weights is not required. Some famous examples include the residual ELM, stacked ELM autoencoder, and local receptive field ELM.
4.10. Twin ELM
A twin ELM is used to obtain a better classification accuracy. A pair of quadratic programming problems are solved by the TELM, and two non-parallel hyperplanes are obtained in a random feature space [
61].
The following pairs of QPPs are solved using the TELM model:
This is subject to
which is subject to
The user-defined parameters are and .
The corresponding Wolfe dual problems are obtained to derive efficient solutions to the primal problems, and they are expressed as follows:
This is subject to
and
which is subject to
. The vectors of the Lagrange multiplier are denoted as
and
.
5. Results and Discussion
The proposed work is tested on four robust datasets: the EMOVO [
11], RAVDESS [
12], SAVEE [
13], and Berlin Emo-DB datasets [
14]. EMOVO is an Italian speech database, and Berlin Emo-DB contains speeches made by German actors. The language of the SAVEE and RAVDESS datasets is English. There are seven emotions in the EMOVO, SAVEE, and Emo-DB datasets, while the RAVDESS speech dataset contains eight emotions. The EMOVO dataset consists of six actors (three males and three females) and about 588 observations, with a sampling rate of approximately 48 KHz. The RAVDESS dataset consists of 24 actors (12 males and 12 females) and about 1440 observations, with a sampling rate of approximately 48 KHz. The SAVEE dataset consists of four males and 480 observations, with a sampling rate of approximately 44.1 KHz. The EMO-DB dataset consists of 10 actors (5 males and 5 females) and 535 observations, with a sampling rate of approximately 16 KHz. A 10-fold cross validation method is used in our experiment. The proposed techniques are implemented and the results are obtained as described below.
Table 1 shows a performance analysis of the classifiers with the transforms for the OIFS technique on the EMOVO dataset. A high classification accuracy of 78.65% is obtained for the FAWT with the twin ELM classification method.
Table 2 shows a performance analysis of the classifiers with the transforms for the HHO feature selection technique on the EMOVO dataset. A high classification accuracy of 79.98% is obtained for the superlet transform with the twin ELM classification method.
Table 3 shows a performance analysis of the classifiers with the transforms for the CSA feature selection technique on the EMOVO dataset. A high classification accuracy of 80.63% is obtained for the chirplet transform with the twin ELM classification method.
Table 4 shows a performance analysis of the classifiers with the transforms for the OIFS technique on the RAVDESS dataset. A high classification accuracy of 84.19% is obtained for the FAWT with the twin ELM classification method.
Table 5 shows a performance analysis of the classifiers with the transforms for the HHO feature selection technique on the RAVDESS dataset. A high classification accuracy of 85.76% is obtained for the FAWT with the twin ELM classification method.
Table 6 shows a performance analysis of the classifiers with the transforms for the CSA feature selection technique on the RAVDESS dataset. A high classification accuracy of 85.54% is obtained for the chirplet transform with the twin ELM classification method.
Table 7 shows a performance analysis of the classifiers with the transforms for the OIFS technique on the SAVEE dataset. A high classification accuracy of 83.94% is obtained for the chirplet transform with the twin ELM classification method.
Table 8 shows a performance analysis of the classifiers with the transforms for the HHO feature selection technique on the SAVEE dataset. A high classification accuracy of 83.21% is obtained for the chirplet transform with the twin ELM classification method.
Table 9 shows a performance analysis of the classifiers with the transforms for the CSA feature selection technique on the SAVEE dataset. A high classification accuracy of 83.11% is obtained for the KSTDIS transform with the twin ELM classification method.
Table 10 shows a performance analysis of the classifiers with the transforms for the OIFS technique on the Berlin Emo-DB dataset. A high classification accuracy of 87.46% is obtained for the superlet transform with the twin ELM classification method.
Table 11 shows a performance analysis of the classifiers with the transforms for the HHO feature selection technique on the Berlin Emo-DB dataset. A high classification accuracy of 87.57% is obtained for the KSTDIS transform with the twin ELM classification method.
Table 12 shows a performance analysis of the classifiers with the transforms for the CSA feature selection technique on the Berlin Emo-DB dataset. A high classification accuracy of 89.77% is obtained for the KSTDIS transform with the twin ELM classification method.
Figure 3 displays a performance analysis of the classifiers with the transforms for the OIFS technique on the EMOVO dataset.
Figure 3 shows that a low classification accuracy of 60.23% is obtained when the FST is implemented with the GB classifier.
Figure 4 displays a performance analysis of the classifiers with the transforms for the OIFS technique on the RAVDESS dataset.
Figure 4 shows that a low classification accuracy of 69.76% is obtained when the synchrosqueezing transform is implemented with the GB classifier.
Figure 5 displays a performance analysis of the classifiers with the transforms for the HHO feature selection technique on the SAVEE dataset.
Figure 5 shows that a low classification accuracy of 70.84% is obtained when the synchrosqueezing transform is implemented with the GB classifier.
Figure 6 displays a performance analysis of the classifiers with the transforms for the CSA feature selection technique on the Berlin Emo-DB dataset.
Figure 6 shows that a low classification accuracy of 83.12% is obtained when the FST is implemented with the DT classifier.
Comparison with Previous Works
The results obtained here are compared with previously obtained results for all four databases, and the results are tabulated in
Table 13,
Table 14,
Table 15 and
Table 16.
An analysis of
Table 13 shows that the proposed works produced very good results when compared with the previous results. The best classification accuracy result of 80.63% was obtained for the Chirplet + CSA + TELM combination on the EMOVO dataset. An analysis of
Table 14 shows that the proposed works once again produced very good results when compared with the previous works. A high classification accuracy of 85.76% was obtained for the FAWT + HHO + TELM combination on the RAVDESS dataset. An analysis of
Table 15 shows that a high classification accuracy of 83.94% was obtained for the Chirplet + OIFS + TELM combination on the SAVEE dataset, which is good when compared with previous works. An analysis of
Table 16 shows that a high classification accuracy of 89.77% was obtained for the KSTDIS + CSA + TELM combination on the Berlin Emo-DB dataset, surpassing the results of previous works.