In this review, we outline our current understanding of the mechanisms involved in the absorption, storage, and transport of dietary vitamin A to the eye, the trafficking of rod opsin protein to the photoreceptor outer segments, which encompasses the logistical backbone required for visual function. Two key mechanisms of this process are emphasized in this manuscript: Ocular and systemic vitamin A membrane transporters, and rhodopsin transporters. Understanding the complementary mechanisms responsible for the generation and proper transport of the retinylidene opsin to the photoreceptor outer segment will eventually shed light on the importance of genes encoded by these proteins, and their relationship on normal visual function and in the pathophysiology of retinal degenerative diseases.

Detecting and understanding emotions is critical for our daily activities. As emotion recognition (ER) systems develop, we start looking at more difficult cases than just acted adult audio-visual speech. In this work, we investigate automatic classification of audio-visual emotional speech of children. Our interest is, specifically, in the improvement of the utilization of the cross-modal relationships between the selected modalities: video and audio. To underscore the importance of developing ER systems for the real-world environment, we present a corpus of children’s emotional audio-visual speech that we collected. We select a state-of-the-art model as a baseline for the purposes of comparison and present several modifications focused on a deeper learning of the cross-modal relationships. By conducting experiments with our proposed approach and the selected baseline model, we observe a relative improvement in performance by 2%. Finally, we conclude that focusing more on the cross-modal relationships may be beneficial for building ER systems for child-machine communications and the environments where qualified professionals work with children.

Characters are one of the most important elements in composing digital animation. The appear-ance and voice of a character should be designed to express the personality and values of the character. However, it is not easy for animation producers to harmoniously match the appear-ance and voice of a character. Advances in deep learning technology have made it possible to overcome this limitation. To achieve this, firstly, an audio-visual dataset of characters is required. In this study, we construct and verify a Korean audio-visual dataset consisting of frontal face im-ages of various characters and short voice clips. We developed an application that can automati-cally extract the frontal face image and a short voice clip of a character by collecting videos up-loaded to YouTube. Through this, a dataset consisting of a total of 1,522 face images and a total of 7,999 seconds of voice clips was built based on 490 characters. Furthermore, we automatically la-bel characters by gender and age to validate the dataset. The dataset built in this study is expected to be used in various deep learning fields, such as classification, generative adversarial networks, and speech synthesis.

Deep-learning-based speech enhancement techniques have been recently grown in interest, since their impressive performance can potentially benefit a wide variety of digital voice communication systems. However, such performance has been evaluated mostly in offline audio processing scenarios (i.e. feeding the model, in one go, a complete audio recording, which may extend several seconds). It is of great interest to evaluate and characterize the current state-of-the-art in applications that process audio online (i.e. feeding the model a sequence of segments of audio data, concatenating the results at the output end). Although evaluations and comparisons between speech enhancement techniques have been carried out before, as far as the author knows, the work presented here is the first that evaluates the performance of such techniques in relation to their online applicability. Meaning, this work measures how the output signal-to-interference ratio (as a separation metric), the response time and memory usage (as online metrics) are impacted by the input length (the size of audio segments), in addition to the amount of noise, amount and number of interferences, and amount of reverberation. Three popular models were evaluated, given their availability on public repositories and online viability: MetricGAN+, Spectral Feature Mapping with Mimic Loss, and Demucs-Denoiser. The characterization was carried out using a systematic evaluation protocol based on the Speechbrain framework. Several intuitions are presented and discussed, and some recommendations for future work are proposed.

With the rise of piano teaching in recent years, many people have joined the team of piano learners. However, the expensive cost of manual instruction and the unique one-on-one teaching model have made piano learning an extravagant event. Most existing approaches based on the audio modality aim to evaluate piano players' skills. Unfortunately, these methods ignored the information contained in the video, which led to a one-sided and simplistic evaluation of the piano player's skills. More recently, multimodal-based methods are proposed to assess the skill level of piano players using both video and audio information. However, existing multimodal approaches use shallow networks to extract video and audio features, which are deficient in extracting complex spatio-temporal and time-frequency features from piano performance. Furthermore, the fingering and the pitch-rhythm information of the piano performance is contained in the spatio-temporal and time-frequency features, respectively. In the paper, we propose a ResNet-based audio-visual fusion model that combines video and audio features to assess the skill level of piano players. Firstly, ResNet18-3D is used as the backbone network for our visual branches, which can extract feature information from the video data. Then, we consider ResNet18-2D as the backbone network of the aural branch and extract the feature information from the audio data. The extracted video features are fused with the audio features to generate multimodal features for the final piano skill evaluation. The experimental results on the PISA dataset show that our proposed audio-visual fusion model, with a validation accuracy of 70.80%, outperforms the state-of-the-art methods in both performance and efficiency. Then, we also explore the impact of different layers of ResNet on model performance, and the experimental results show that the audio-visual fusion model dealing with the piano skill assessment problem can make full use of both feature information when the number of video features is close to the number of audio features.

The focus of this study was to investigate how individuals with bipolar disorder integrate auditory and visual speech information compared to non-disordered individuals and whether there were any differences in auditory and visual speech integration in the manic and depressive episodes in bipolar disorder patients. It was hypothesized that bipolar groups’ auditory-visual speech integration would be less robust than the control group. Further, it was predicted that those in the manic phase of bipolar disorder would integrate visual speech information more than their depressive phase counterparts. To examine these, the McGurk effect paradigm was used with typical auditory-visual speech (AV) as well as auditory-only (AO) speech perception on visual-only (VO) stimuli. Results. Results showed that the disordered and non-disordered groups did not differ on auditory-visual speech (AV) integration and auditory-only (AO) speech perception but on visual-only (VO) stimuli. The results are interpreted to pave the way for further research whereby both behavioural and physiological data are collected simultaneously. This will allow us understand the full dynamics of how, actually, the auditory and visual (relatively impoverished in bipolar disorder) speech information are integrated in people with bipolar disorder.

This research study focuses on addressing practical optimization challenges in audio-visual content using advanced computational intelligence methods, with a specific emphasis on privacy-preserving issues. The research will be presented as a proposal and can be conducted based on the provided ideas. Computational intelligence encompasses various approaches, including evolutionary algorithms, learning methods, fuzzy systems, and neural networks, which have become essential tools for tackling complex engineering optimization problems due to computational expenses, dimensional complexities, and interdisciplinary integration. The research objectives include a thorough investigation of optimization challenges in audio-visual content, the delineation of problem characteristics, and the development of relevant methodologies based on these specifications. The main goals of the study, titled "Advanced Computational Intelligence Assisted Less-Expensive Privacy-Preserving Optimization," encompass the development of a metamodel-based learning algorithm integrated with evolutionary computation for privacy-preserving optimization, less-expensive signal watermarking optimization using hybrid metamodeling and evolutionary algorithms, and a closed-loop robust intelligent PID control system applied in real-time dynamic signal watermarking. Additional objectives may be defined as the research progresses and further background review is conducted.

The emergence of Artificial Intelligence (AI) driven Keyword Spotting (KWS) technologies has revolutionized human to machine interaction. Yet, the challenge of end-to-end energy efficiency, memory footprint and system complexity of current Neural Network (NN) powered AI-KWS pipelines has remained ever present. This paper evaluates KWS utilizing a learning automata powered machine learning algorithm called the Tsetlin Machine (TM). Through significant reduction in parameter requirements and choosing logic over arithmetic based processing, the TM offers new opportunities for low-power KWS while maintaining high learning efficacy. In this paper we explore a TM based keyword spotting (KWS) pipeline to demonstrate low complexity with faster rate of convergence compared to NNs. Further, we investigate the scalability with increasing keywords and explore the potential for enabling low-power on-chip KWS.

In this paper, we propose an imagined speech-based brain wave pattern recognition using deep learning. Multiple features were extracted concurrently from eight-channel Electroencephalography (EEG) signals. To obtain classifiable EEG data with fewer number of sensors, we placed the EEG sensors on carefully selected spots on the scalp. To decrease the dimensions and complexity of the EEG dataset and to avoid overfitting during the deep learning algorithm, we utilized the wavelet scattering transformation. A low-cost 8-channel EEG headset was used with MATLAB 2023a to acquire the EEG data. The Long-Short Term Memory Recurrent Neural Network (LSTM-RNN) was used to decode the identified EEG signals into four audio commands: Up, Down, Left, and Right. Wavelet scattering transformation was applied to extract the most stable features by passing the EEG dataset through a series of filtration processes. Filtration has been implemented for each individual command in the EEG datasets. The proposed imagined speech-based brain wave pattern recognition approach achieved a 92.50% overall classification accuracy. This accuracy is promising for designing a trustworthy imagined speech-based Brain-Computer Interface (BCI) future real-time systems. For better evaluation of the classification performance, other metrics were considered, and we obtained 92.74%, 92.50% and 92.62% for precision, recall, and F1-score, respectively.

This study aimed to evaluate the effects of Blazepod technology when incorporated with various training regimens on the essential elements of physical fitness in volleyball, such as agility, speed, visual response time, and vertical leap. Thirty-six participants (age :17.33±0.48 y, height: 183.78±6.34 cm, weight: 77.14±5.16 kg) were in-volved in this study. In randomized design, three equal groups with two experimental groups and one control group were involved in the study over a period of six weeks: (GE1) used Blazepod technology to workout at a low level; (GE2) used Blazepod technology to workout at a higher intensity; (CG) did not employ Blazepod technology, but it adhered to the same fundamental training curriculum. We conducted agility, speed, reaction time, and vertical jump tests before, midway and after the study to determine optimal training volume, intensity, and frequency. A significant main effect of time (P0.01), and time×group interaction (P>0.001) were found for all tests. Post hoc test showed that the two experimental groups gadget significantly improved in all performance variables (P

Chinese texts often feature a substantial volume of normative content, and their analysis has increasingly become a focal point in recent semantic analysis endeavors. Enhancing the effectiveness of text classification and clustering is pivotal in unveiling the intricate semantics within extensive textual data. Empirical research within the realm of computational social science has underscored the need for improved explanatory power when it comes to general text clustering methods applied to normative texts. This study introduces an innovative hierarchical clustering method that incorporates part-of-speech (POS) features and pioneers a novel semantic network model. It achieves this by amalgamating a POS-based feature weight calculation method with hierarchical clustering techniques. To assess its efficacy, the method is rigorously evaluated across three diverse text datasets, encompassing normative reports, news articles, and social media content. The experimental results unequivocally demonstrate the superior performance of the POS-HC method, surpassing the traditional TF and TF-IDF feature extraction methods in terms of clustering accuracy, and surpassing some existing clustering methods. Furthermore, its classification effectiveness exhibits notable advantages, particularly in the semantic classification of normative texts.

This paper propose an original approach of achieving a Cymatics based visual perception of isolated speech commands. The idea is to smartly combine the effective speech processing and analysis methods with the phenomena of Cymatics. In this context, an effective approach for automatic isolated speech based message recognition is proposed. The incoming speech segment is enhanced by applying the appropriate pre-emphasis filtering, noise thresholding and zero alignment operations. The Mel-Frequency Cepstral coefficients (MFCCs), Delta coefficients and Delta-Delta coefficients are extracted from the enhanced speech segment. Later on, the Dynamic Time Warping (DTW) technique is employed to compare these extracted features with the reference templates. The comparison outcomes are used to make the classification decision. The classification decision is transformed into a methodical excitation. Finally, this excitation is converted into the systematic visual perceptions via the phenomenon of Cymatics. The system functionality is tested with an experimental setup and results are presented. The approach is novel and can be employed in various applications like visual art, encryption, education, archeology, architecture, integration of impaired people, etc.

Brain-Computer Interface (BCI) systems can assist physically challenged people to interact with their surroundings and improve the quality of their lives. Decoding human thoughts is a powerful technique that can assist paralyzed people who have lost their speech production ability. Speaking is a combined process involving synchronizing the brain and the oral articulators. This paper proposed a high-accuracy brain wave pattern recognition based on inner speech using a novel feature extraction method. Only eight EEG electrodes were used in this study, and they were set on selected spots on the scalp. Support Vector Machine (SVM) was employed to decode the recorded EEG dataset into four internally spoken words which are: Up, Down, Left, and Right. The proposed approach achieved overall classification accuracy that ranges between 96.20% to 97.5%. In addition, more performance evaluation metrics were estimated to test the reliability of classifying the EEG-based inner speech data, and we obtained 97.61%, 97.50%, and 97.73% for F1-score, recall, and precision respectively. Furthermore, the Area Under Curve of the Receiver Operating Characteristic (AUC-ROC) proved the strength of the proposed approach for classifying the specified inner speech commands by achieving a macro-average amount of 99.32%. The inner speech classification method using electroencephalography proposed in this work can clinically help improve communication for patients with problems including speech disorder, mutism, cognitive development, executive function, and psychopathology.

This study addresses the critical challenge of Speech Enhancement (SE) in noisy environments, where the deployment of Deep Neural Networksolutions on microcontrollers is hindered by their extensive computational demands. Focusing on this gap, our research introduces a novel SE method optimized for MCUs, employing a 2-layer GRU model that capitalizes on perceptual speech properties and innovative training methodologies. By incorporating self-reference signals and a dual strategy of compression and recovery based on the Mel scale, we develop an efficient model tailored for low-latency applications. Our GRU-2L-128 model demonstrates a significant reduction in size and computational requirements, achieving a 14.2× decrease in model size and a 409.1× reduction in operations compared to traditional DNN methods like DCCRN, without compromising performance. This advancement offers a promising solution for enhancing speech intelligibility in resource-constrained devices, marking a pivotal step in SE research and application.

Domain generalization has led to remarkable achievements in Hyperspectral Image (HSI) classification. Inspired by contrastive language-image pre-training (CLIP), the language-aware domain generalization method has been explored to learn cross-domain-invariant representation. However, existing methods face some challenges: 1) The weak capacity to extract long-range contextual information and inter-class correlation. 2) Due to the inadequacies of the large-scale pre-training for HSI data, the spatial-spectral features of HSI and linguistic features can not be straightforwardly alignment. To address the above problems, a novel network has been proposed with a CLIP framework, which consists of an image encoder, based on an encoder-only transformer to obtain the global contextual information and inter-class correlation, a frozen text encoder, and a cross-attention mechanism, named Linguistic-Interact-with-Visual Engager (LIVE), enhances the interaction between two modalities. Extensive experiments demonstrating superior performance over state-of-the-art methods in HSI Domain Generalization with a CLIP framework.

The aim of this research is to investigate and use a variety of immersive multisensory media techniques in order to create convincing digital models of fossilised tree trunks for use in XR. This is made possible via the use of geospatial data derived from sources such as aerial imaging using UAS, terrestrial material using cameras and also include both the visual and audio element for better immersion, accessible and explorable in 6 Degrees of Freedom (6DoF). Immersiveness is a key factor in order to result in output that is especially engaging to the user. Both conventional and alternative methods are explored and compared, emphasising in the advantages made possible via the help of Machine Learning Computational Photography. Material is collected using both UAS and terrestrial camera devices, including a 3D-360º camera with 6 sensors, using stitched panoramas as sources for photogrammetry processing. Difficulties such as capturing large free standing objects using terrestrial means were overcome using practical solutions involving mounts and remote streaming solutions. Conclusions indicated that superior fidelity can be achieved by the help of Machine Learning Computational Photography processes and higher resolutions and technical specs of equipment not necessarily translating to superior outputs.

The pursuit of Artificial Intelligence (AI) that emulates human cognitive processes is a cornerstone of ethical AI development, ensuring that emerging technologies can integrate seamlessly into societal frameworks requiring nuanced understanding and decision-making. Zero-Shot Instance Retrieval (ZSIR) stands at the forefront of this endeavour, potentially providing a robust platform for AI systems, particularly large visual language models, to demonstrate and refine cognition-aligned learning without the need for direct experience. In this paper, we critically evaluate current cognition alignment methodologies within traditional zero-shot learning paradigms, using visual attributes and word embedding generated by large AI models. We propose a unified similarity function that quantifies the cognitive alignment level, bridging the gap between AI processes and human-like understanding. Through extensive experimentation, our findings illustrate that this similarity function can effectively mirror the visual-semantic gap, steering the model towards enhanced performance in zero-shot instance retrieval. This work not only benchmarks the cognition alignment of AI, but also sets a new precedent for the development of visual language models attuned to the complexities of human cognition.}

The oscillating heat pipe (OHP) is a new member in the family of heat pipes, and it has great potential applications in energy conservation. However, the fluid flow and heat transfer in the OHP as well as the fundamental effects of inner diameter on them have not been fully understood, which are essential to the design and optimization of the OHP in real applications. Therefore, by combining the high-speed visualization method and infrared thermal imaging technique, the fluid flow and thermal performance in the OHPs with inner diameters of 1, 2 and 3 mm are presented and analyzed. The results indicate that three fluid flow motions, including small oscillation, bulk oscillation and circulation, coexist or, respectively, exist alone with the increasing heating load under different inner diameters, with three flow patterns occurring in the OHPs, viz. bubbly flow, slug flow and annular flow. These fluid flow motions are closely correlated with the heat and mass transfer performance in the OHPs, which can be reflected by the characteristics of infrared thermal images of condensers. The decrease in the inner diameter increases the frictional flow resistance and capillary instability while restricting the nucleate boiling in OHPs, which leads to a smaller proportion of bubbly flow, a larger proportion of short slug flow, a poorer thermal performance, and easier dry-out of working fluid. In addition, when compared with the 2 mm OHP, the increasing role of gravity induces the thermosyphon effect and weakens the 'bubble pumping' action, which results in a little smaller and bigger thermal resistances of 3 mm OHP under small and bulk oscillation of working fluid, respectively.

In theory and applications, it is often inevitable to work with projectors onto convex sets, where a linear transform is involved. In this article, a novel projector is presented, which generalizes previous results in that it admits a broader family of linear transforms, but on the other hand it is limited to box-type convex sets in the transformed domain. The new projector has an explicit formula and it can be interpreted within the framework of proximal optimization. The benefit of the new projector is demonstrated on an example from signal processing, where it was possible to speed up the convergence of a signal declipping algorithm by a factor of more than two.

We describe a multi-resolution approach for audio classification and illustrate its application to the open data set for environmental sound classification. The proposed approach utilizes a multi-resolution based ensemble consisting of targeted feature extraction of approximation (coarse scale) and detail (fine scale) portions of the signal under the action of multiple transforms. This is paired with an automatic machine learning engine for algorithm and parameter selection and the LSTM algorithm, capable of mapping several sequences of features to a predicted class membership probability distribution. Initial results show an improvement in multi-class classification accuracy.

Structures inevitably suffer damages after an earthquake, with their severity ranging from minimal damages to non-structural elements to partial or even total collapse, possibly with loss of human lives. Thus, it is essential for engineers to understand the crucial factors that drive a structure towards suffering higher degrees of damage, so that preventative measures can be taken. In the present study, we focus on three well-known damage thresholds, namely Collapse Limit State, Ultimate Limit State and Serviceability Limit State and analyze which of the features obtained via Rapid Visual Screening, determine whether or not a given structure will cross that threshold. To this end, we use Machine Learning to perform binary classification for each damage threshold, as well as Explainability via SHAP values (acronym for SHapley Additive exPlanations) to quantify the effect of each parameter. The quantitative results obtained demonstrate the potential applicability of ML methods to re-calibrate the computation of structural vulnerability indices using data from recent earthquakes.

The growing awareness of the biological effects of artificial light on humans has stimulated ample research. It is now widely accepted that asynchrony between artificialand natural light-dark cycles can elicit severe detrimental health effects. New research has been devoted to lighting solutions that dynamically change their color to mimic spectral changes of daylight and to account for human needs. However, in some situations, the visual properties of light must be preserved: For example professional TV video editors and shift workers who must work under standardized lighting conditions to do color correction in post-production. We have investigated the possibility to tune circadian effects using white lights that are spectrally different but nonetheless have similar color coordinates thus appear as a similar white tone. Our simulation results indicate that it is possible to modulate circadian light effects by combining LEDs for neutral white (4000 K), a widely used white tone for indoor lighting in europe. The results also show that the solutions combining single-color LEDs do however not meet the quality criteria from the visual point of view because their color rendering ability decays to unacceptable low levels. Combining narrowband LEDs with a broadband white LED improves the color rendering quality and we show how far circadian light effects can be tuned according to common theoretical models. The aim is to reflect daylight situations with artificial lighting, thus having a high melatonin suppression in the morning and low melatonin suppression in the evening. As a consequence our aim is to show what maximum and minimum circadian effect is possible with the same set of LEDs.

This study delves into Personal Narratives (PN), which encompass both oral and written recounting of individual experiences, encompassing facts, events, people, and thoughts. Traditional emotion recognition and sentiment analysis primarily focus on broader categories like utterances or documents. Our research, however, centers on identifying Emotion Carriers (EC), which are specific segments within speech or text that elucidate the narrator's emotional state (e.g.,"losing a parent", "decision-making moments"). Extracting these ECs enriches the representation of a user's emotional state, thereby enhancing natural language understanding and the sophistication of dialogue models. While previous studies have utilized lexical attributes to identify ECs, we argue that incorporating spoken narratives offers a more nuanced view of the context and emotional state. This paper explores the integration of speech and textual embeddings at the word level, alongside both early and late fusion techniques, for improved EC detection in spoken narratives. We employ Residual Neural Networks (ResNet), initially pre-trained on diverse speech emotion datasets and subsequently fine-tuned for EC detection. Our experimental findings demonstrate that late fusion, in particular, significantly enhances EC detection capabilities.

The article considers an approach to the problem of assessing the quality of speech during speech rehabilitation as a classification problem. For this, a classifier is built on the basis of an LSTM neural network for dividing speech signals into two classes: before the operation and immediately after. At the same time, speech before the operation is the standard to which it is necessary to approach in the process of rehabilitation. The metric of belonging of the evaluated signal to the reference class acts as an assessment of speech. An experimental assessment of rehabilitation sessions and a comparison of the resulting assessments with expert assessments of phrasal intelligibility were carried out.

Purpose: This study aims to provide a foundational understanding of patient-centric perceptions of telemedicine implementation in rural Minnesota. Methods: This retrospective cohort study utilized a mail questionnaire to assess patient attitudes and behaviors regarding telemedicine during the emergency response to COVID-19. The target population was patients of a Primary Care clinic in eastern Minnesota. Descriptive statistics were used to assess respondents’ level of agreement with survey items scored on a 0-10 scale. Results: A majority of the respondents indicated that they were satisfied with using telemedicine (73.96%); that they gained an additional sense of control over their health condition due to the availability of telemedicine (52.08%); that they would support the use of telemedicine in the future (61.05%); that telemedicine increased access to care (65.63%); telemedicine was preferred over the clinic (61.46%), and that they would purchase new equipment to increase the utility of telemedicine services (54.17%). 51.04% of the respondents indicated a 10/10 level of satisfaction with telemedicine services. 48.96% indicated the strongest level of disagreement related to difficulty using telemedicine or a preference for in-person clinical appointments. Conclusions: Patients were found to have favorable perceptions of telemedicine overall. Though providers continue to recognize the benefit of face-to-face visits, their patients show an increased predilection for virtual care and telemedicine will likely remain an essential tool for providers to reach patients who would otherwise choose not to seek care.

The focus of this research is to analyze the results of encrypting audio using various authenticated encryption algorithms implemented in the Python cryptography library for ensuring authenticity and confidentiality of the original contents. The Advanced Encryption Standard (AES) is used as the underlying cryptographic primitive in conjunction with various modes including Galois Counter Mode (GCM), Counter with Cipher Block Chaining Message Authentication Code (CCM), and Cipher Block Chaining (CBC) with Keyed-Hashing for encrypting a relatively small audio file. The resulting encrypted audio shows similarity in the variance when encrypting using AES-GCM and AES-CCM. There is a noticeable reduction in variance of the performed encodings and an increase in the amount of time it takes to encrypt and decrypt the same audio file using AES-CBC with Keyed-Hashing. In addition, the corresponding encrypted using this mode audio spans a longer duration. As a result, AES should either have GCM or CCM for an efficient and reliable authenticated encryption integration within a workflow.

The software industry has adopted component-based software development (CBSD) to rapidly build and deploy large and complex software systems with significant savings at minimal engineering effort, cost, and time. However, CBSD encounters issues on security trust, mainly with respect to dependability attributes. A system is considered dependable when it can produce the outputs for which it was designed with no adverse effect on its intended environment. Dependability consists of several attributes that imply availability, confidentiality, integrity, reliability, safety, and maintainability. Dependability attributes must be embedded in a CBSD model to develop dependable component software. Motivated by the importance of these attributes, this paper pursues two objectives: to design a model for developing a dependable system that mitigates the vulnerabilities of software components, and to evaluate the proposed model. The model proposed in this study is labelled as developing dependable component-based software (2DCBS). To develop this model, the CBSD architectural phases and processes must be framed and the six dependability attributes embedded according to the best practice method. The expert opinion approach was applied to evaluate 2DCBS framing. In addition, the 2DCBS model was applied to the development of an information communication technology (ICT) portal through an empirical study method. Vulnerability assessment tools (VATs) were employed to verify the dependability attributes of the developed ICT portal. Results show that the 2DCBS model can be adopted to develop web application systems and to mitigate the vulnerabilities of the developed systems. This study contributes to CBSD and facilitates the specification and evaluation of dependability attributes throughout model development. Furthermore, the reliability of the dependable model can increase confidence in the use of CBSD for industries.

This article presents an overview of the first AI-human collaborated album, Hello World, by SKYGGE, which utilizes Sony’s Flow Machines technologies. This case study is situated within an overview of current and emerging uses of AI in popular music production, and connects those uses with myths and fears that have circulated in discourses concerning the use of AI in general, but also in music specifically. By proposing the concept of an audio uncanny valley in relation to AIPM (artificial intelligence popular music), this article offers a lens through which to examine the more novel and unusual melodies and harmonisations made possible through AI music generation, and questions how this content relates to wider speculations about posthumanism, sincerity, and authenticity in both popular music, and more wider assumptions of anthropocentric creativity. In its documentation of the emergence of a new era of popular music, the AI era, this article surveys: (1) the current landscape of artificial intelligence popular music focusing on the use of Markov models for generative purposes; (2) posthumanist creativity and the potential for an audio uncanny valley; and (3) issues of perceived authenticity in the technologically mediated “voice”.

Pure monocular 3D reconstruction is an ill-posed problem that has attracted the research community's interest due to the affordability and availability of RGB sensors. SLAM, VO, and SFM are disciplines formulated to solve the 3D reconstruction problem and estimate the camera’s ego-motion, so many methods have been proposed. However, most of these methods were not evaluated in large datasets, under various motion patterns, had not been tested under the same metrics, and most of them had not been evaluated following a taxonomy, making their comparison and selection difficult. In this research, we performed a comparison of ten publicly available SLAM and VO methods following a taxonomy, including one method for each category of the primary taxonomy, three machine learning-based methods, and two updates of the best methods, to identify the advantages and limitations of each category of the taxonomy and test if the addition of machine learning or the updates made on those methods improved them significantly. Thus, we evaluated each algorithm under the TUM-Mono benchmark and performed an inferential statistical analysis to identify significative differences through its metrics. Results determined that sparse-direct methods significantly outperformed the rest of the taxonomy, and fusing them with machine learning techniques significantly improves the performance of geometric-based methods from different perspectives.

The combination of neural network and beamforming has been proved to be very effective in multi-channel speech separation. But its performance faces a challenge in complex environment. In this paper, an iteratively refined multi-channel speech separation method is proposed for the challenge, where the proposed is composed of initial separation and iterative separation. In initial separation, the time-frequency domain dual-path recurrent neural network neural network (TFDPRNN), minimum variance distortionless response (MVDR) beamformer and post-separation (also TFDPRNN) are cascaded for obtaining the first additional input in iterative separation. In iterative separation, the MVDR beamformer and post-separation are iteratively used, where the output of the MVDR beamformer is used as an additional input of the post-separation network and the final output comes from post-separation module. This iteration of the beamformer and post-separation is fully employed for promoting their individual optimization, which ultimately improves the overall performance of speech separation in multi-speaker scenarios. Experiments on the spatialized version of the WSJ0-2mix corpus show that our proposed method is significantly better than the current popular methods. In addition, the method also has a good effect on the dereverberation task.

With the rapid development of big data, artificial intelligence, and Internet technologies, the human-human contact and human-machine interaction have produced an explosive growth of voice data. Rapidly identifying the speaker's identity and retrieving and managing his or her speech data in the massive amount of speech data has become a major challenge for intelligent speech applications in the field of information security. This research proposes a vocal recognition technique based on information adversarial training for speaker identity recognition in massive audio and video, and speaker identification when oriented to the information security domain. The experimental results show that the method projects data from different scene channels all onto the same space and dynamically generates interactive speaker representations. It solves the channel mismatch problem and effectively improves the recognition of speaker's voice patterns across channels and scenes. It is able to separate overlapping voices when multiple people speak at the same time and reduce speaker separation errors. It realizes speaker voice recognition for information security field and achieves 89% recall rate in massive database, which has practical application value for intelligent application field.

For the Remotely Piloted Aircraft Systems (RPAS) market to continue its current growth rate, cost-effective "Detect and Avoid" systems that enable safe beyond visual line of sight (BVLOS) operations are critical. We propose an audio-based "Detect and Avoid" system, composed of microphones and an embedded computer, which performs real-time inferences using a sound event detection (SED) deep learning model. Two state-of-the-art SED models, YAMNet and VGGish, are fine-tuned using our dataset of aircraft sounds and their performances are compared for a wide range of configurations. YAMNet, whose MobileNet architecture is designed for embedded applications, outperformed VGGish both in terms of aircraft detection and computational performance. YAMNet's optimal configuration, with > 70% true positive rate and precision, results from combining data augmentation and undersampling with the highest available inference frequency (i.e. 10 Hz). While our proposed "Detect and Avoid" system already allows the detection of small aircraft from sound in real time, additional testing using multiple aircraft types is required. Finally, a larger training dataset, sensor fusion, or remote computations on cloud-based services could further improve system performance.

Background: Visual acuity (VA) assessments are crucial in ophthalmology but traditionally rely on in-clinic evaluations. The emergence of telemedicine has spurred interest in creating dependable self-administered VA tests for use beyond standard clinical environments. This study evaluates the practicality and validity of a self-administered Near VA card test against traditional Snellen and Rosenbaum Pocket Vision Screener (RPVS) methods for home monitoring and enhancing clinical workflow. Methods: In a cross-sectional study a near VA card (Hadassah Self-Visual Acuity Screener-HSVA) was developed with written and videotaped instructions for self-use. Patients with a minimal best corrected VA (BCVA) of 1.0 LogMAR in at least one eye were recruited from Ophthalmology and Optometry clinics. Outcomes included mean BCVA difference between the self-administered and those obtained by the examiner, and correlations among BCVA values obtained by Snellen, RPVS, HSVA and previous distance BCVA according to the patient electronic medical records. Results: A total of 275 participants (mean age: 42.5±19.4 years, range: 18-89 years, 47% female) were included. Test-retest reliability analysis for the HSVA demonstrated very good correlation and repeatability (n=38 patients.; Rs=1.0, P

The need for cross-modal retrieval (CMR) among users is growing because of the swift advancements in computer vision and natural language processing. Promoting home operating systems is greatly aided by the openKylin home operating system. It is focused on enhancing the functionality of the operating system with safe and dependable operating technologies. Practical implications abound for integrating computer vision research into home operating systems. A multimodal retrieval approach based on openKylin is presented to improve the usability of text search graphs, audio, and video. The overview of pertinent domestic and international research is the main topic of this article. This technological advancement. It then offers details on the design, implementation procedure, and final implementation outcomes for demonstration. The works of natural operations show that this strategy has excellent accuracy and performance.

Artificial Intelligence is increasingly being used by social media platforms to tackle online hate speech. The sheer quantity of content, the speed at which is it developed and the enhanced pressure companies are facing by States to remove hate speech quickly from their platforms have led to a tricky situation. This commentary argues that automated mechanisms, which may have biased datasets and be unable to pick up on the nuances of language, should not be left unattended with hate speech as this can lead to issues of violating freedom of expression and the right to non-discrimination.

The speech-to-song illusion is a type of auditory illusion that the repetition of a part of a sentence would change people’s perception tendency from speech-like to song-like. The study aims to examine how pace, emotion, and language tonality affect people’s experience of the speech-to-song illusion. It uses a between-subject (Pace: fast, normal, vs. slow) and within-subject (Emotion: positive, negative, vs. neutral; language tonality: tonal language vs. non-tonal language) design. Sixty Hong Kong college students were randomly assigned to one of the three conditions characterized by pace. They listened to 12 audio stimuli, each with repetitions of a short excerpt, and rated their subjective perception of the presented phrase, whether it sounded like a speech or a song, on a five-point Likert-scale. Paired-sample t-tests and repeated measures ANOVAs were used to analyze the data. The findings reveal that a faster speech pace could strengthen the tendency of the speech-to-song illusion. Neither emotion nor language tonality show a statistically significant influence on the speech-to-song illusion. This study suggests that the perception of sound should be in a continuum and facilitates the understanding of song production in which speech can turn into music by having repetitive phrases and to be played in a relatively fast pace.

The speech entailed in human voice comprises essentially para-linguistic information used in many voice-recognition applications. Gender voice-recognition is considered one of the pivotal parts to be detected from a given voice, a task that involves certain complications. In order to distinguish gender from a voice signal, a set of techniques have been employed to determine relevant features to be utilized for building a model from a training set. This model is useful for determining the gender (i.e, male or female) from a voice signal. The contributions are involved in two folds: (i) providing analysis information about well-known voice signal features using a prominent dataset, (ii) studying various machine learning models of different theoretical families to classify the voice gender, and (iii) using three prominent feature selection algorithms to find promisingly optimal features for improving classification models. Experimental results show the importance of sub-features over others, which are vital for enhancing the efficiency of classification models performance. Experimentation reveals that the best recall value is equal to 99.97%; 99.7% of two models of Deep Learning (DL) and Support Vector Machine (SVM) and with feature selection the best recall value is 100% for SVM techniques.

Acoustic energy mapping provides the functionality to obtain characteristics of acoustic sources, such as: presence, localization, type and trajectory of sound sources. Several beamforming-based techniques can be used for this purpose, however, they rely on the difference of arrival times of the signal at each capture node (or microphone), so it is of major importance to have synchronized multi-channel recordings. A Wireless Acoustic Sensor Network (WASN) can be very practical to install when used for mapping the acoustic energy of a given acoustic environment. However, they are known for having low synchronization between the recordings from each node. The objective of this paper is to characterize the impact of current popular synchronization methodologies as part of the WASN to capture reliable data to be used for acoustic energy mapping. The two evaluated synchronization protocols are: Network Time Protocol (NTP) y Precision Time Protocol (PTP). Additionally, three different audio capture methodologies were proposed for the WASN to capture the acoustic signal: two of them, recording the data locally and one sending the data through a local wireless network. As a real-life evaluation scenario, a WASN was built using nodes conformed by a Raspberry Pi 4B+ with a single MEMS microphone. Experimental results demonstrate that the most reliable methodology is using the PTP synchronization protocol and audio recording locally.

Spectrally weighed irradiance on the corneal plane is often used to determine the visual and non-visual potency of light. Spectral irradiance or illuminance is measured with a diffuser in the so-called 2π geometry (i.e., a hemispheric measurement). Due to anatomical conditions, the human binocular field of view (FOV) does not correspond to a hemisphere but is occluded upwards and downwards. FOV-restricted measurements are scarce, however, and thus the error when using hemispheric measurements is yet to be determined. For our study, we designed and 3D printed FOV occlusions as attachments to spectral measurement devices. Measurements with and without the occlusion were taken in different lighting situations in the laboratory (light from different directions) and in real-world conditions (light typically from above). Our measurements show a reduction of visual and melanopic values due to the FOV occlusion ranging from negligible to over 60% for realistic scenarios. Further, the reduction was comparable for visual and melanopic parameters, as the distribution of light in the FOV was generally spectrally homogeneous. In the case of an exemplary and artificial laboratory situation, however, the melanopic daylight (D65) efficacy ratio changed by more than a factor of 2 only due to the FOV occlusion. Finally, head orientation showed a marked impact on all quantities. Our results reveal the potential for considerable error when just evaluating and comparing vertical, hemispheric measurements in experiments and non-visual lighting design projects. We thus encourage the (additional) use of a FOV occlusion for eye-level measurements in typical viewing directions and provide open-source 3D-print files.

With the increasingly online exposure of urban public spaces, the new concept of "Internet-celebrity cities" has emerged in China. The online internet-famous sites are the main characteristics of "Internet-celebrity cities". To fully understand online users' preferences in such kind of public spaces, this article took 27 typical riverfront internet-famous sites (RIFS) in Changsha city (China), as an example. Through the online images and review comments on social media platform "Xiaohongshu", the study analyzed users’ perception of RIFS specially on visual and emotional preferences. It revealed the following findings:(1) The popularity of RIFS had a significant head effect and there were far more positive emotions than neutral and negative emotions in review comments. (2) According to 5 different categories of RIFS: modern buildings with river views, restaurants with strong visual impact became the main attractions on commercial RIFS. The art exhibition RIFS attracted visitors to experience traditional culture and highlight their aesthetic pursuits. In the historical and cultural RIFS, people preferred to see the historical buildings with surrounding natural environment, as well as river and mountains views. Ecological Recreational RIFS showed the similar results with the uncultivated RIFS, for the main focus was on the ecological environment, a broader vision and the specific atmosphere. Moreover, the uncultivated RIFS showed more diversified attraction points. This study can provide a new perspective on the waterfront vitality and can offer a targeted and attractive manner for waterfront regeneration that are different from traditional methods.

The question of whether the early visual cortex (EVC) is involved in visual mental imagery remains a topic of debate. In this paper, I propose that the inconsistency in findings can be explained by the unique challenges associated with investigating EVC activity during imagery. During perception, the EVC processes low-level features which means that activity is highly sensitive to variation in visual details. If the EVC has the same role during visual mental imagery, any change in the visual details of the mental image would lead to corresponding changes in EVC activity. Within this context, the question should not be whether EVC is ‘active’ during imagery but how its activity relates to specific imagery properties. Studies using methods that are sensitive to variation in low-level features reveal that imagery can recruit the EVC in similar ways as perception. However, not all mental images contain a high level of visual details. Therefore, I end by considering a more nuanced view which states that imagery can recruit the EVC, but that does not mean that it always does so.

Audio recording in classrooms is a common practice in educational research, with applications ranging from detecting classroom activities to analysing student behaviour. Previous research has employed neural networks for classroom activity detection and speaker role identification. However, these recordings are often affected by background noise that can hinder further analysis, and the literature has only sought to identify noise with general filters and not specifically designed for classrooms. Although the use of high-end microphones and environmental monitoring can mitigate this problem, these solutions can be costly and potentially disruptive to the natural classroom environment. In this context, we propose the development of a neural network model that can specifically detect and filter out background noise in classroom recordings. This model would allow the use of lower quality recordings without compromising analysis capability, thus facilitating data collection in natural educational environments and reducing the costs associated with high-end recording equipment.

This study compared cortical responses to speech in preschoolers with typical language development (TLD) and Developmental Language Disorder (DLD), and in adults. We investigated whether top-down language effects modulate speech perception in an adult-like manner. We compared cortical mismatch responses (MMR) during passive perception of speech differing in linguistic complexity in three groups of participants: preschoolers with TLD (n=11), preschoolers with DLD (n=16) and adults (n=20). We also measured children’s phonological skills and investigated if they are associated with the cortical responses. MMR results indicated top-down language effects in adults, with enhanced cortical discrimination of lexical stimuli but not of non-words. In preschoolers, the TLD and DLD groups did not differ on the MMR measures and no top-down effects were detected. Moreover, we found no association between MMRs and phonological awareness skills, even though the DLD group’s phonological skills were significantly lower. Our findings suggest that top-down language modulations on speech discrimination may not be present during early childhood, and that children with DLD may not exhibit cortical speech perception deficits. The lack of association between phonological and MMR measures indicates that further research is needed to understand the link between language skills and cortical activity in preschoolers.

(1) The incidence of ankyloglossia ranges from 0.02 to 10.7%. The literature describes the effect of ankyloglossia on selected dysfunctions of the stomatognathic system, however no studies could be found reporting the influence of ankyloglossia on the occurrence of several disorders in a group of subjects. The aim of the present study was to assess the effect of lingual frenulum on swallowing, speech, occlusion, and periodontal status; (2) Methods: The subjects were 172 patients, 86 with ankyloglossia (study group) and 86 with normal tongue frenulum (control group). In all subjects, the length of tongue frenulum, the type of swallowing, tongue mobility, occlusion, periodontal status and speech abnormalities were assessed; (3) Results: All subjects from the control group and all those with mild ankyloglossia showed normal tongue mobility. A limited tongue mobility was found in 29.4% subject with moderate and in 70.6% subjects with severe ankyloglossia. Rhotacism was observed in 21.3% subjects with normal frenulum, in 2.1% with mild, 38.3% with moderate, and 38.3% with severe ankyloglossia. Malocclusion or crowding was diagnosed in subjects with mild, moderate and severe ankyloglossia in 7.4%, 33.9% and 20.7% subjects (total 62%), respectively, whereas in the control group - in 21.6% subjects. No abnormalities in the periodontium in the area of the lingual surfaces of the crowns of the lower central incisors were found in any of the examined persons. Among patients with infantile type of swallowing 24.4% had a normal length of the tongue frenulum, 11.1% - mild, 28.9% - moderate, and 35.6 - severe ankyloglossia. Among patients presenting a mature type of swallowing 58.7% had a normal length of the frenulum; (4) Conclusions: 1.A shortened tongue frenulum correlates with “infantile swallowing pattern”. 2. Moderate or severe ankyloglossia significantly limits tongue mobility. 3. Short tongue frenulum is related to speech disorders.

herein, we have compared the performance of SVM and MLP in emotion recognition using speech and song channels of the RAVDESS dataset. We have undertaken a journey to extract various audio features, identify optimal scaling strategy and hyperparameter for our models. To increase sample size, we have performed audio data augmentation and addressed data imbalance using SMOTE. Our data indicate that optimised SVM outperforms MLP with an accuracy of 82 compared to 75%. Following data augmentation, the performance of both algorithms was identical at ~79%, however, overfitting was evident for the SVM. Our final exploration indicated that the performance of both SVM and MLP were similar in which both resulted in lower accuracy for the speech channel compared to the song channel. Our findings suggest that both SVM and MLP are powerful classifiers for emotion recognition in a vocal-dependent manner.

The inaccurate estimates of linear prediction coefficient (LPC) and noise variance introduce bias in Kalman filter (KF) gain and degrades speech enhancement performance. The existing methods proposed a tuning of the biased Kalman gain particularly in stationary noise condition. This paper introduces a tuning of the KF gain for speech enhancement in real-life noise conditions. First, we estimate noise from each noisy speech frame using a speech presence probability (SPP) method to compute the noise variance. Then construct a whitening filter (with its coefficients computed from the estimated noise) and employed to the noisy speech, yielding a pre-whitened speech, from where the speech LPC parameters are computed. Then construct KF with the estimated parameters, where the robustness metric offsets the bias in Kalman gain during speech absence to that of the sensitivity metric during speech presence to achieve better noise reduction. Where the noise variance and the speech model parameters are adopted as a speech activity detector. The reduced-biased Kalman gain enables the KF to minimize the noise effect significantly, yielding the enhanced speech. Objective and subjective scores on NOIZEUS corpus demonstrates that the enhanced speech produced by the proposed method exhibits higher quality and intelligibility than some benchmark methods.

Noninvasive brain stimulation technique can be used for modulation of motor behaviors such as reaction time (RT). The main aim of this study was to investigate whether RT during a sequential visual isometric pinch task (SVIPT) can be affected by anodal transcranial direct current stimulation (a-tDCS) over three different stimulation sites such as dorsolateral prefrontal cortex (DLPFC), primary motor cortex (M1) or posterior parietal cortex (PPC). We also aimed to assess if stimulation of these cortical sites affect the transfer of learning during SVIPT. A total of 48 right-handed healthy participants were randomly assigned to one of the four a-tDCS groups: 1) left M1, 2) left DLPFC, 3) left PPC and 4) sham. A-tDCS was applied concurrent with SVIPT in which the participants were precisely controlled their forces to reach seven different target forces from 10 to 40% of maximum voluntary contraction (MVC) presenting on a computer screen with right dominant hand. Four test blocks were randomly performed at baseline and 15 min after the intervention including sequence and random blocks with either hand. Our results showed significant differences between a-tDCS groups on ratio RT in the sequence blocks of both the right trained (P = .041) and left untrained hands (P = .023) for target forces of 15% and 30 % of MVC, respectively. In sequence blocks, M1 showed significant elongation in ratio RT for the target force of 15% and 30 % MVC compared to PPC and Sham groups in the right hand. In sequence blocks with left hand, this negative effect were found between M1-DLPFC and M1-Sham groups. In random blocks, no significant differences were found among four groups at each target forces for the left hand while there were significant differences for 15% (P =. 04) and 40% MVC (P =. 035) in the right hand. Our findings suggest that three different stimulation sites (DLPFC, M1 or PPC) were differentially affected by a single session of a-tDCS at some target forces during SVIPT. The effects of a-tDCS on ratio RT were transferred in the left untrained hand for sequence blocks but not random blocks. Further research is needed to understand fundamental aspects of these main areas of the brain on temporal processing in a precision control task such as SVIPT.

People of all ages and skill levels can benefit from phonemic awareness activities due to their accessibility and adaptability. People can improve their pronunciation and communicate more clearly and confidently by developing their capacity to perceive and manipulate sounds. This combined emphasis on improving speech and reducing anxiety draws attention to the many benefits of incorporating phonemic awareness interventions into speech therapy plans and educational courses. The aim of this article review is to gather and collect relevant literature that solidifies the idea that phonemic awareness is one of the solutions to speech anxiety problems. Through the consistent practice of diverse phonemic exercises, people can become less sensitive to the anxiety-inducing elements of verbal communication, thus cultivating a sense of mastery and control over their speech. Through gradual exposure, people can gradually develop resilience and confidence, which is an essential component in the effective management of speech anxiety. In order to enable people to overcome speech anxiety, this conclusion emphasizes the importance of including phonemic awareness interventions within educational and therapeutic settings.

Online speech enhancement is of great interest, because of its diverse areas of application. The Demucs architecture has been recently shown to achieve a high level of performance, even with small audio segments, while requiring a relatively small amount of computational resources. However, an issue that it bears, as well as most of state-of-the-art speech enhancement techniques, is that of target speech selection: it is assumed that only one speech source is present. However, in real-life scenarios, more than one speech source may be present, in it is uncertain which speech source within the fed mixture should be enhanced. Thus, it is of interest to complement speech enhancement techniques (such as Demucs-based) with a target selection scheme, so as to ensure that only the source of interest is enhanced. In this work, two target selection strategies are explored: 1) an embedding-based strategy, using a codified sample of the target speech, and 2) a location-based strategy, using a beamforming-based pre-filter to select the target that is in front of a 2-microphone array. It is shown that while both strategies improve the performance of the Demucs-based technique when one or more speech interferences are present, they both have their pros and cons. Specifically, while the beamforming-based strategy achieves overall a better performance compared to the embedding-based strategy, it is sensitive against the location variation of the target speech source which the embedding-based strategy does not suffer from.

Data augmentation techniques recently gained more adoption in speech processing, including speech emotion recognition. Although more data tends to be more effective, there may be a trade-off in which more data will not provide a better model. This paper reports experiments on investigating the effects of data augmentation in speech emotion recognition. The investigation aims at finding the most useful type of data augmentation and the number of data augmentations for speech emotion recognition. The experiments are conducted on the Japanese Twitter-based emotional speech corpus. The results show that for speaker-independent data, two data augmentations with glottal source extraction and silence removal exhibited the best performance among others, even with more data augmentation techniques. For the text-independent data (including speaker and text-independent), more data augmentations tend to improve speech emotion recognition performances. The results highlight the trade-off between the number of data augmentation and the performance of speech emotion recognition showing the necessity to choose a proper data augmentation technique for a specific application.

Users on the social networking platform have the freedom to express themselves freely. Towards the same time, this has created a forum for disagreement and hate directed at someone, society, racism, sexual orientation, and so on. Identifying hate online is a challenging task. Researchers from all around the world have contributed major methods for detecting hate speech, but owing to the issue's complexity, there are still many unresolved issues. In this research, we offer a multi-model learning strategy for detecting hate speech on Twitter. We utilised the Kaggle TwitterHate dataset, which had 31962 tweets categorised as binary hate or non-hate, to evaluate our technique. The suggested method is tested using commonly used machine learning classifiers with multi-model technique. Using TF-IDF features, we acquired detection results of 96.29 %, precision of 96%, recall of 96%, and f1-score of 96%.

Hateful and abusive speech presents a major challenge for all online social media platforms. Recent advances in Natural Language Processing and Natural Language Understanding allow more accurate detection of hate speech in textual streams. This study presents a multimodal approach to hate speech detection by combining Computer Vision and Natural Language processing models for abusive context detection. Our study focuses on Twitter messages and, more specifically, on hateful, xenophobic and racist speech in Greek aimed at refugees and migrants. In our approach we combine transfer learning and fine-tuning of Bidirectional Encoder Representations from Transformers (BERT) and Residual Neural Networks (Resnet). Our contribution includes the development of a new dataset for hate speech classification, consisting of tweet ids, along with the code to obtain their visual appearance, as they would have been rendered in a web browser. We have also released a pre-trained Language Model trained on Greek tweets, which has been used in our experiments. We report a consistently high level of accuracy (accuracy score=0.970, f1-score=0.947 in our best model) in racist and xenophobic speech detection.

Background: Patients with allergic rhinitis (AR) have bothersome symptoms that may assess using the visual analog scale (VAS). Asthma frequently is associated with AR. The present study investigated the role of VAS in children with newly-diagnosed AR. Methods: Parameters were VAS for nasal and asthma symptoms, and doctor’s perception of these diseases, medication use, and asthma comorbidity. Results: 67% of AR children had asthma. Severity of perceived nasal symptoms was associated with more intense asthma symptoms and doctor’s perception of disease severity (p<0.001 and p=0.02, re-spectively. Conclusions: Allergic rhinitis frequently is associated with asthma in child-hood. Symptom perception assessment using VAS is a simple tool for assessing their intensity. Moreover, the severity of nasal symptoms is associated with asthma symptom severity. Therefore, asthma should be carefully considered in AR children and require appropriate symptom perception assessment.

In recent times, BERT based transformer models have become an inseparable part of the 'tech stack' of text processing models. Similar progress is being observed in the speech domain with a multitude of models observing state-of-the-art results by using audio transformer models to encode speech. This begs the question of what are these audio transformer models learning. Moreover, although the standard methodology is to choose the last layer embedding for any downstream task, but is it the optimal choice? We try to answer these questions for the two recent audio transformer models, Mockingjay and wave2vec2.0. We compare them on a comprehensive set of language delivery and structure features including audio, fluency and pronunciation features. Additionally, we probe the audio models' understanding of textual surface, syntax, and semantic features and compare them to BERT. We do this over exhaustive settings for native, non-native, synthetic, read and spontaneous speech datasets

Hate Speech Detection in Arabic presents a multifaceted challenge due to the broad and diverse linguistic terrain. With its multiple dialects and rich cultural subtleties, Arabic requires particular measures to address hate speech online successfully. To address this issue, academics and developers have used natural language processing (NLP) methods and machine learning algorithms adapted to the complexities of Arabic text. However, many proposed methods were hampered by a lack of a comprehensive dataset/corpus of Arabic hate speech. In this research, we propose a novel multi-class public Arabic dataset comprised of 403,688 annotated tweets categorized as extremely positive, positive, neutral, or negative based on the presence of hate speech. Using our developed dataset, we additionally characterize the performance of multiple machine learning models for Hate speech identification in Arabic Jordanian dialect tweets. Specifically, the Word2Vec, TF-IDF, and AraBert text representation models have been applied to produce word vectors. With the help of these models, we can provide classification models with vectors representing text. After that, seven Machine learning classifiers have been evaluated: Support Vector Machine (SVM), Logistic Regression (LR), Naive Bays (NB), Random Forest (RF), AdaBoost (Ada), XGBoost (XGB), and CatBoost (CatB). In light of this, the experimental evaluation revealed that, in this challenging and unstructured setting, our gathered and annotated datasets were rather efficient and generated encouraging assessment outcomes. This will enable academics to delve further into this crucial field of study.

The exponential increase in the use of the Internet and social media over the last two decades has changed human interaction. This has led to many positive outcomes, but at the same time it has brought risks and harms. While the volume of harmful content online, such as hate speech, is not manageable by humans, interest in the academic community to investigate automated means for hate speech detection has increased. In this study, we analyse six publicly available datasets by combining them into a single homogeneous dataset and classify them into three classes, abusive, hateful or neither. We create a baseline model and we improve model performance scores using various optimisation techniques. After attaining a competitive performance score, we create a tool which identifies and scores a page with effective metric in near-real time and uses the same as feedback to re-train our model. We prove the competitive performance of our multilingual model on two langauges, English and Hindi, leading to comparable or superior performance to most monolingual models.

This study aims to automate the optimization of a Full-Range speaker in an SUV's audio system according to the equal loudness contours principle. The input signal, and frequency responses of the amplifier and speaker were transferred to Matlab. Using ideal filter parameters, ten parametric equalizer models were created, and the speaker output was obtained using the convolution technique. The same filter settings were applied to the vehicle multimedia system, and experimental results were obtained. The simulation and experimental results were compared, showing high similarity with a Pearson correlation coefficient of 0.9295 and an RMSE of 2.29. Results were compared with the ideal contour. The Pearson correlation coefficients for simulation and experimental results were 0.6341 and 0.6715, with RMSE values of 4.88 and 2.57, showing low similarity. Consequently, each parametric equalizer filter's parameters were optimized using a genetic algorithm. The optimized parameters were applied to the vehicle multimedia system, and results were compared with the ideal contour. The Pearson correlation coefficients for simulation and experimental results were 0.965 and 0.9661, with RMSE values of 2.12 and 1.48. These results indicate that optimization aligns the speaker output closer to the ideal contour, enhancing in-vehicle audio system performance and increasing user satisfaction.

Evolution and geometry generate complexity in similar ways. Evolution drives natural selection while geometry may capture the logic of this selection and express it visually, in terms of specific generic properties representing some kind of advantage. Geometry is ideally suited for expressing the logic of evolutionary selection for symmetry, which is found in the shape curves of vein systems and other natural objects such as leaves, cell membranes, or tunnel systems built by ants. The topology and geometry of symmetry is controlled by numerical parameters, which act in analogy with a biological organism's DNA. The introductory part of this paper reviews findings from experiments illustrating the critical role of two-dimensional design parameters and shape symmetry for visual or tactile shape sensation, and for perception-based decision making in populations of experts and non-experts. Thereafter, results from a pilot study on the effects of fractal symmetry, referred to herein as the symmetry of things in a thing, on aesthetic judgments and visual preference are presented. In a first experiment (psychophysical scaling procedure), non-expert observers had to rate (scale from 0 to 10) the perceived beauty of a random series of 2D fractal trees with varying degrees of fractal symmetry. In a second experiment (two-alternative forced choice procedure), they had to express their preference for one of two shapes from the series. The shape pairs were presented successively in random order. Results show that the smallest possible fractal deviation from "symmetry of things in a thing" significantly reduces the perceived attractiveness of such shapes. The potential of future studies where different levels of complexity of fractal patterns are weighed against different degrees of symmetry is pointed out in the conclusion.

Only languages with abundant resources are given the opportunity to build speech recognition systems, whereas low resource languages are disregarded. This project attempts to create a system that can normally convert speech to text in the Myanmar (Burmese) language, a language part of low resource languages. The Burmese language is extremely intricate and is made up of vowels, consonants, tones, and stress. There are 10 numbers, 12 vowels, and 33 letters in the Myanmar alphabet. Deep learning allows for the evolution of numerous algorithms while eradicating ignorance and even low-resource languages may be turned into speech-to-text systems with a small quantity of data. The procedures required to construct a successful speech recognition system that can assist both the corporate sector and society will be covered in this project, including data collecting, preprocessing, modeling, prototyping, assessment, and ultimately deployment.

This study introduces an original comprehensive system centered on identifying specific terms that indicate a user's position, particularly the discrete values representing latitude and longitude. This system not only detects these terms but also retrieves the corresponding numerical data for accurate and efficient determination of locations. The importance of this study can be applied various fields, notably aiding offline operations of military personnel, who often lack internet access. In such scenarios, precise awareness of location is vital for strategic manoeuvres, rescue operations, and navigating unfamiliar landscapes. The system allows these personnel by allowing them to extract exact location coordinates from spoken terms, thereby enhancing their awareness even in challenging surroundings. Apart from its military utility, the project holds broader significance. Teams responding to emergencies, personnel involved in disaster management, and exploratory missions can all gain from this technology during disruptions in communication infrastructure. Furthermore, travelers, adventurers, and outdoor enthusiasts can utilize this system to accurately determine their positions in remote areas without relying on online maps. We used offline speech recognition techniques to precisely transcribe spoken terms, achieving an accuracy of over 91.3% and a word error rate of 4.2%. For sound recognition, the OpenAI Whisper model was used, and a conversion process from SpeechRecognition to AudioSegmentation was implemented, followed by transforming the audio into .wav format, we have also developed the interface of the app to use it efficiently using Streamlit. This was done to ensure seamless compatibility with the Whisper model and uninterrupted audio input. By training the system to identify specific linguistic linked to location, it achieves robust detection and extraction of relevant terms. This approach eliminates the necessity for constant internet connectivity, rendering it exceptionally useful in remote, offline, and resource-limited situations.

Automatic Voice Query Service can extremely reduce the artificial cost, which could improve the response efficiency for users. The automatic speech recognition (ASR) is one of the important component in AVQS. However, many dialect areas in China make the AVQS have to response the multi-accented Mandarin users by single acoustic model in ASR. This problem severely limits the accuracy of ASR for multi-accented speech in the AVQS. In this paper, a new framework for AVQS is proposed to improve the accuracy of response. Firstly, the fusion feature including iVector and filterbank acoustic features is used to train the Transformer-CTC model. Secondly, the transformer-CTC model is used to construct the end-to-end ASR. Finally, key words matching algorithm for AVQS based on fuzzy mathematic theory is proposed to further improve the accuracy of response. The results show that the final accuracy in our proposed framework for AVQS arrives at 91.5%. The proposed framework for AVQS can satisfy the service requirement of different areas in mainland of China. This research has a great significance for exploring the application value of artificial intelligence in the real scene.

Medical records contain many terms which are difficult to process. Our aim in this study is to allow the visual exploration of the information in medical databases where the texts presents a large number of syntactic variations and abbreviations, through an interface which facilitates content identification, navigation and information retrieval. We propose the use of multi-term tag clouds as content representation tools and as assistants for the browsing and querying tasks. The tag cloud generation is achieved through a novelty mathematical method that allows related terms to remain grouped together within the tags To evaluate this proposal, we have used a database with 24,481 records. 23 expert users in the medical field were tasked to complete a survey to evaluate the generated tag clouds properties and we obtained a precision of 0.990, a recall of 0.870 and a F1score of 0.904 in the evaluation of the tag cloud as an information retrieval tool. The main contribution of this approach is that we automatically generate a visual interface over the text capable of capturing the semantics of the information and facilitating access to medical records.

Audio classification using deep learning models, essential for applications like voice assistants and music analysis, faces challenges when deployed on edge devices due to their limited computational resources and memory. Achieving a balance between performance, efficiency, and accuracy is a significant obstacle in optimizing these models for such constrained environments. In this investigation, we evaluate diverse deep learning architectures, including Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM), for audio classification tasks on the ESC 50, UrbanSound8k, and Audio Set datasets. Our empirical findings indicate that Mel Spectrograms outperform raw audio data, attributing this enhancement to their synergistic alignment with advanced image classification algorithms and their congruence with human auditory perception. To address the constraints of model size, we apply model compression techniques, notably magnitude pruning, Taylor Pruning, and 8-bit quantization. The research demonstrates that a hybrid pruned model achieves a commendable accuracy rate of 89 percent, which, although marginally lower than the 92 percent accuracy of the uncompressed CNN, strikingly illustrates an equilibrium between efficiency and performance. Subsequently, we deploy the optimized model on Raspberry Pi 4 and NVIDIA Jetson Nano platforms for audio classification tasks. These findings highlight the significant potential of model compression strategies in enabling effective deep learning applications on resource-limited devices, with minimal compromise on accuracy.

Environmental pollution by noise is one of the most serious health threats nowadays. The impact of noise on the human body depends not only on the sound level but also on its spectral distribution. Reliable measurements of the environmental noise spectrum are often hampered by the very high price of top quality measuring devices. This paper explores the possibility of using much cheaper audio recorders for frequency analysis. Comparative research were performed in laboratory and field conditions, which showed that, with some limitations, these devices can be useful for the needs of environmental noise frequency analysis. This fact gives an opportunity for reduce the cost of noise analysis experimental work.

Cochlear implantation (CI) allows rehabilitation for patients with severe to profound hearing impairment. Although the use of a robotic assistant provides technical assistance to the surgeon, the assessment of the impact of its use on auditory outcomes remains uncertain. We aim to com-pare the hearing results of patients who underwent bilateral cochlear implantation, one side was performed with manual insertion and the other side with robot-assisted insertion. The electrode array intrascalar positioning and the surgery duration were also studied. This retrospective intra-individual study involved 10 patients who underwent bilateral cochlear implantation. The study included 2 infants and 8 adults. The unique composition of this cohort enabled us to utilize each patient as their own control. Regarding speech disyllabic recognition, pure tone average, ECAP, ratio of array translocation, basilar membrane rupture, percentage of translocated electrodes there was no difference between manual and robot-assisted CI groups. This study is the first to compare intra-individual hearing performance after cochlear implantation either manually or robot-assisted. The number of patients and the time delay between manual and robotic implantation may have led to a lack of power, but there was no apparent difference in hearing performance between manual and robotic implantation.

Deep brain stimulation (DBS) of the subthalamic nucleus (STN) has become an effective and widely used tool in the treatment of Parkinson’s disease (PD). STN-DBS has varied effects on speech. Clinical speech ratings suggest worsening following STN-DBS, but quantitative intelligibility, perceptual, and acoustic studies have produced mixed and inconsistent results. Improvements in phonation and declines in articulation have frequently been reported during different speech tasks under different stimulation conditions. Questions remain about preferred STN-DBS stimulation settings. Seven right-handed, native speakers of English with PD treated with bilateral STN-DBS were studied off medication at three stimulation conditions: stimulators off, 60 Hz (low frequency stimulation - LFS), and the typical clinical setting of 185 Hz (High frequency - HFS). Spontaneous speech was recorded in each condition and excerpts were prepared for transcription (intelligibility) and difficulty judgements. Separate excerpts were prepared for listeners to rate abnormalities in voice, articulation, fluency, and rate. Intelligibility for spontaneous speech was reduced at both HFS and LFS when compared to STN-DBS off. Speech produced at HFS was more intelligible than that produced at LFS, but HFS made the intelligibility task (transcription) subjectively more difficult. Both voice quality and articulation were judged to be more abnormal with STN-DBS on. STN-DBS reduced the intelligibility of spontaneous speech at both LFS and HFS but lowering the frequency did not improve intelligibility. Voice quality ratings with STN-DBS were correlated with the ratings made without stimulation. This was not true for articulation ratings. STN-DBS exacerbated an existing voice disorder and may have introduced new articulatory abnormalities.

This paper proposes a novel approach, based on the adaptive rate processing and analysis, for the isolated speech recognition. The idea is to smartly combine the event-driven signal acquisition and windowing along with adaptive rate processing, analysis and classification for realizing an effective isolated speech recognition. The incoming speech signal is digitized with an event-driven A/D converter (EDADC). The output of EDADC is windowed with an activity selection process. These windows are later on resampled uniformly with an adaptive rate interpolator. The resampled windows are de-noised with an adaptive rate filter and their spectrum are computed with an adaptive resolution short time Fourier transform (ARSTFT). Later on, the magnitude, Delta and Delta-Delta spectral coefficients are extracted. The Dynamic Time Warping (DTW) technique is employed to compare these extracted features with the reference templates. The comparison outcomes are used to make the classification decision. The system functionality is tested for a case study and results are presented. An 8.2 times reduction in acquired number of samples is achieved by the devised approach as compared to the classical one. It aptitudes a significant computational gain and power consumption reduction of the proposed system over the counter classical ones. An average subject dependent isolated speech recognition accuracy of 96.8% is achieved. It shows that the proposed approach is a potential candidate for the automatic speech recognition applications like rehabilitation centers, smart call centers, smart homes, etc.

Speech Emotion Recognition (SER) is an interesting and difficult problem to handle. In this paper, we deal with it through the implementation of deep learning networks. We have designed and implemented six different deep learning networks, a Deep Belief Network (DBN), a simple deep neural network (SDNN), a LSTM network (LSTM), a LSTM network with the addition of an attention mechanism (LSTM-ATN), a Convolutional neural network (CNN), and a Convolutional neural network with the addition of an attention mechanism (CNN-ATN), having in mind, apart from solving the SER problem, to test the impact of attention mechanism to the results. Dropout and Batch Normalization techniques are also used to improve the generalization ability (prevention of overfitting) of the models as well as to speed up the training process. The Surrey Audio-Visual Expressed Emotion database (SAVEE), and the Ryerson Audio-Visual Database (RAVDESS) database were used for training and evaluation of our models. The results showed that networks with the addition of the attention mechanism did better than the others. Furthermore, they showed that CNN-ATN was the best among tested networks, achieving an accuracy of 74% for the SAVEE and 77% for the RAVDESS dataset, and exceeded existing state-of-the-art systems for the same datasets.

In recent years, with the popularity of smart mobile devices, the interaction between devices and users, especially in the form of voice interaction, has become increasingly important. If smart devices can understand more users' emotional states through voice data, more customized services can be provided for users. This paper proposes a novel machine learning model for speech emotion recognition, which combines convolutional neural networks (CNN), long short-term memory neural networks (LSTM), and deep neural networks (DNN), called CLDNN. To make the designed system can recognize the audio signal closer to the human auditory system does, this article uses the Mel frequency cepstral coefficients (MFCCs) of audio data as the input of the machine learning model. First, the MFCCs of the voice signal is extracted as the input of the model, and the feature values of the data are calculated using several local feature learning blocks (LFLB) composed of one-dimensional CNN. Because the audio signals are time-series data, the feature values obtained from LFLBs then input into LSTM layer to enhance the learning on time-series level. Finally, fully connected layers are used for classification and prediction. Three databases RAVDESS, EMO-DB and IEMOCAP are used for the experiments in this paper. The experimental results show that the proposed method can improve the accuracy compared to other related researches in speech emotion recognition.

Speech signals are degraded in real-life environments, product of background noise or other factors. The processing of such signals for voice recognition and voice analysis systems presents important challenges. One of the conditions that make adverse quality difficult to handle in those systems is reverberation, produced by sound wave reflections that travel from the source to the microphone in multiple directions.To enhance signals in such adverse conditions, several deep learning-based methods have been proposed and proven to be effective. Recently, recurrent neural networks, especially those with long and short-term memory (LSTM), have presented surprising results in tasks related to time-dependent processing of signals, such as speech. One of the most challenging aspects of LSTM networks is the high computational cost of the training procedure, which has limited extended experimentation in several cases. In this work, we present a proposal to evaluate the hybrid models of neural networks to learn different reverberation conditions without any previous information. The results show that some combination of LSTM and perceptron layers produce good results in comparison to those from pure LSTM networks, given a fixed number of layers. The evaluation has been made based on quality measurements of the signal's spectrum, training time of the networks and statistical validation of results. Results help to affirm the fact that hybrid networks represent an important solution for speech signal enhancement, with advantages in efficiency, but without a significan drop in quality.

Vitamin K occupies a unique and often obscured place among its fellow fat-soluble vitamins. Evidence is mounting however, that vitamin K (VK) may play an important role in the visual system apart from the hepatic carboxylation of hemostatic related proteins. However, to our knowledge, no review covering the topic has appeared in the medical literature. Recent studies have confirmed that matrix Gla protein (MGP), a vitamin K-dependent protein (VKDP), is essential for the regulation of intraocular pressure in the mouse. The PREDIMED (Prevención con Dieta Mediterránea) study, a randomized trial involving 5860 adults at risk for cardiovascular disease, demonstrated a 29% reduction in the risk of cataract surgery in participants with the highest tertile of dietary vitamin K1 (PK) intake compared with the lowest tertile. However, the specific requirements of the eye and visual system (EVS) for VK, and what might constitute an optimized VK status, is currently unknown and largely unexplored. It is the intention therefore of this narrative review to provide an introduction concerning VK and the visual system, review ocular VK biology, and to provide some historical context to recent discoveries. Potential opportunities and gaps in current research efforts will be touched upon, in the hope of raising awareness, and encouraging continued VK related investigations in this important and highly specialized sensory system.

Under optimal conditions, just 3–6 ms of visual stimulation suffices for humans to see motion. Motion perception on this time scale implies that the visual system under these conditions reliably encodes, transmits, and processes neural signals with near-millisecond precision. Motivated by in vitro evidence for high temporal precision of motion signals in the primate retina, we investigated how neuronal and perceptual limits of motion encoding relate. Specifically, we examined the correspondence between the time scale at which cat retinal ganglion cells in vivo represent motion information and temporal thresholds for human motion discrimination. The time scale for motion encoding by ganglion cells ranged from 4.6–91 ms, depended nonlinearly on temporal frequency but not on contrast. Human psychophysics revealed that minimal stimulus durations required for perceiving motion direction were similarly brief, 5.6–65 ms, similarly depended on temporal frequency but, above ~10%, not on contrast. Notably, physiological and psychophysical measurements corresponded closely throughout (r = 0.99), despite more than a 20-fold variation in both human thresholds and optimal time scales for motion encoding in the retina. These results demonstrate that neural circuits for motion vision in cortex can maintain and make use of the high temporal fidelity of the retinal output signals.

The promised dataset was obtained from the daily Turkish words and phrases pronounced by various people in the videos posted on YouTube. The purpose of collecting the dataset is to provide detection of the spoken word by recognizing patterns or classifying lip movements with supervised, unsupervised, semi-supervised learning and machine learning algorithms. Most of the datasets related with lip reading consist of people recorded on camera with fixed backgrounds and the same conditions, but the dataset presented here consists of images compatible with machine learning models developed for real-life challenges. It contains a total of 2335 instances taken from TV series, movies, vlogs, and song clips on YouTube. The images in the dataset vary due to factors such as the way people say words, accent, speaking rate, gender and age. Furthermore, the instances in the dataset consist of videos with different angles, shadows, resolution, and brightness that are not created manually. The most important feature of our lip reading dataset is that we contribute to the non-synthetic Turkish dataset pool, which does not have wide dataset varieties. Machine learning studies can be carried out in many areas, such as the defense industry and social life, with this dataset.

The electrical stimulation of the visual cortices has the potential to restore vision to blind individuals. Until now, the results of visual cortical prosthetics has been limited as no prosthesis has restored a full working vision but the field has shown a renewed interest these last years thanks to wireless and technological advances. However, several scientific and technical challenges are still open in order to achieve the therapeutic benefit expected by these new devices. One of the main challenges is the electrical stimulation of the brain itself. In this review, we analyze the results in electrode-based visual cortical prosthetics from the electrical point of view. We first briefly describe what is known about the electrode-tissue interface and safety of electrical stimulation. Then we focus on the psychophysics of prosthetic vision and the state-of-the-art on the interplay between the electrical stimulation of the visual cortex and phosphene perception. Lastly, we discuss the challenges and perspectives of visual cortex electrical stimulation and electrode array design to develop the new generation implantable cortical visual prostheses.

Over the years, there has been a significant rise in the world's scientific knowledge. However, most of it lacks structure and is often termed as Dark Data. Both humans and expert systems have continually faced difficulty in analyzing and comprehending such overwhelming amounts of information which is crucial in solving several real-world problems. Information and data visualization techniques proffer a promising solution to explore such data by allowing quick comprehension of information, the discovery of emerging trends, identification of relationships and patterns, etc. In this tutorial, we utilize the rich corpus of PubMed comprising of more than 30 million citations from biomedical literature to visually explore and understand the underlying key-insights using various information visualization techniques. With this study, we aim to diminish the limitation of human cognition and perception in handling and examining such large volumes of data by speeding up the process of decision making and pattern recognition and enabling decision-makers to fully understand data insights and make informed decisions.

Visual imagery, i.e., the mental experience of an object occurring in the absence of external visual stimulus, might encompass diverse content according to different observers. Further, subjectively experienced time is encoded in the later entorhinal cortex. Starting from these two observations, and considering Einstein’s account of spacetime, we show how, in terms of special relativity, the content of visual imagery is not stationary and fixed, rather depends on the standpoint of the observer. We elucidate how the subjective definition of time (perceived by our mind as static) might give rise to modifications in the length of the imagined object which are experimentally assessable and quantifiable. In particular, based on recent neuroscientific literature, we show how changes in our mental time windows are able to squeeze the visual content of mental imagery.

The paper presents a study of two novel visual motion onset stimulus-based brain–computer interfaces (vmoBCI). Two settings are compared with afferent and efferent to a computer screen center motion patterns. Online vmoBCI experiments are conducted in an oddball event–related potential (ERP) paradigm allowing for “aha–responses” decoding in EEG brainwaves. A subsequent stepwise linear discriminant analysis classification (swLDA) classification accuracy comparison is discussed based on two inter–stimulus–interval (ISI) settings of 700 and 150 ms in two online vmoBCI applications with six and eight command settings. A research hypothesis of classification accuracy non–significant differences with various ISIs is confirmed based on the two settings of 700 ms and 150 ms, as well as with various numbers of ERP response averaging scenarios.The efferent in respect to display center visual motion patterns allowed for a faster interfacing and thus they are recommended as more suitable for the no–eye–movements requiring visual BCIs.

High-tech augmentative and alternative communication (AAC) methods are on a constant rise; however, the interaction between the user and the assistive technology is still challenged for an optimal user experience centered around the desired activity. This review presents a range of signal sensing and acquisition methods utilized in conjunction with the existing high-tech AAC platforms for speech disabled individuals, including imaging methods, touch-enabled systems, mechanical and electro-mechanical access, breath-activated methods, and brain computer interfaces (BCI). The listed AAC sensing modalities are compared in terms of ease of access, affordability, complexity, portability, and typical conversational speeds. A revelation of the associated AAC signal processing, encoding, and retrieval highlights the roles of machine learning (ML) and deep learning (DL) in the development of intelligent AAC solutions. The demands and the affordability of most systems were found to hinder the scale of usage of high-tech AAC. Further research is indeed needed for the development of intelligent AAC applications reducing the associated costs and enhancing the portability of the solutions for a real user’s environment. The consolidation of natural language processing with current solutions also needs to be further explored for the amelioration of the conversational speeds. The recommendations for prospective advances in coming high-tech AAC are addressed in terms of developments to support mobile health communicative applications.

This study aimed to assess the feasibility of employing audio and vision technologies as an integrated monitoring system to monitor the health and behavior of pigs under different environmental conditions and to associate these factors. A total of 81 growing pigs ((Landrace x Largewhite) x Duroc) with distinct weight categories (10.47 ± 2.57 kg and 28.81 ± 5.03 kg) were distributed into two distinct housing conditions: Control (characterized by low NH3 levels and low stocking density) and Treatment (marked by elevated NH3 levels and varying stocking densities). Each house was installed with a SoundTalks monitor for automated daily evaluation of respiratory health status (ReHS), and a camera with a microphone to facilitate manual tracking of respiratory symptoms and behavioral patterns. Results showed that the Treatment group encountered significantly high room temperature, NH3 concentration, and carbon dioxide levels, resulting in compromised growth performance—a phenomenon further exacerbated in high stocking density conditions. Pigs within the treatment exhibited increases (p &lt; 0.05) in lateral and total lying behaviors, and incidences of ear and tail injuries, increases screaming frequency, conjunctivitis incidences, coughing frequency, elevated respiration rates, and a decrease in ReHS. The environmental conditions affect the behavior and health of the pigs. Moreover, the behavioral patterns of the pigs are associated with their health conditions. The SoundTalks system did not trigger any warnings during the experimental period. Nevertheless, trend analysis indicated a significant reduction in respiratory health in the treatment. In conclusion, this study underscores the efficacy of merging audio and visual technologies to holistically monitor pig health and behavior, enabling enhanced management strategies. Findings emphasize that monitoring the ReHS trend serves as a pivotal marker for identifying respiratory health problems, complementing the system's innate alarm functions.

Background: In the present study, we assessed the feasibility and success outcomes of voice prosthesis (VP) changes when performed by speech-language pathologist (SLP). Methods: Patients treated with total laryngectomy (TL) from January 2020 to December 2020 were prospectively recruited from our medical center. Patients benefited from tracheoesophageal puncture. The VP changes were performed by the senior SLP and the following data were collected for each VP change: date of placement; change or removal; VP type and size; reason for change or removal; and use of a washer for periprosthetic leakage. A patient-reported outcome questionnaire including 6 items was proposed to patients at each VP change (Appendix 1). Items were assessed with a 10-point Likert-scale. Results: Fifty-two VP changes were performed by the senior SLP during the study period. The mean duration of the SLP consultation, including patient history, examination and VP change procedure was 20 min (range: 15-30). The median prosthesis lifetime was 88 days. The main reasons for VP changes were transprosthetic (N=34; 79%) and periprosthetic (N=7; 21%) leakages, respectively. SLP successfully performed all VP changes. He did not change one VP but used a periprosthetic silastic to stop the periprosthetic leakages. In two cases, SLP needed the surgeon examination to discuss about the following indication: implant mucosa inclusion and autologous fat injection. The patient satisfaction was high according to the speed and the quality of care by the SLP. Conclusion: The delegation of VP change from the otolaryngologist-head and neck sur-geon to the speech-language pathologist (SLP) may be done without significant complications. The delegation of VP change procedure to SLP may be interesting in some rural regions with otolaryngologist shortage.

Speech emotion recognition remains a heavy lifting in natural language processing. It has strict requirements to the effectiveness of feature extraction and that of acoustic model. With that in mind, a Heterogeneous Parallel Convolution Bi-LSTM model is proposed to address these challenges. It consists of two heterogeneous branches: the left one contains two dense layers and a Bi-LSTM layer, while the right one contains a dense layer, a convolution layer, and a Bi-LSTM layer. It can exploit the spatiotemporal information more effectively, and achieves 84.65%, 79.67%, and 56.50% unweighted average recall on the benchmark databases EMODB, CASIA, and SAVEE, respectively. Compared with the previous research results, the proposed model achieves better performance stably.

Speech/music classification that facilitates optimized signal processing from classification results has been extensively adapted as an essential part of various electronics applications, such as multi-rate audio codecs, automatic speech recognition, and multimedia document indexing. In this paper, a new technique to improve the robustness of speech/music classifier for 3GPP enhanced voice service (EVS) using long short-term memory (LSTM) is proposed. For effective speech/music classification, feature vectors implemented with the LSTM are chosen from the features of the EVS. Experiments show that LSTM-based speech/music classification produces better results than conventional EVS under a variety of conditions and types of speech/music data.

Successful geophysical exploration works in Gucheng-Yaxi area located in Gaochun District, Jiangsu province, China, have been limited partly due to complex geological condition of the area and high artificial noise in data acquired using electrical and electromagnetic methods. In this work, we deployed the new anti-interference spread spectrum induced polarization method (SSIP) and audio magnetotellurics (AMT) method to detect a copper-polymetallic deposit in the area. Two-dimensional (2D) inversion results at Gucheng-Yaxi Area revealed a high chargeability anomalous zone on SSIP profile that coincided with a zone of moderate resistivity located between two resistor bodies on AMT profile. A follow up 1200 meters drill hole was established at this high chargeability-moderate resistivity zone which encountered polymetallic (copper, lead, zinc, gold and silver) mineralization at the depth ≥ 400 m. Drill hole data analysis showed that mineralization occurs interspaced in marble rock mass at varies depth. Furthermore, several low resistivity-weak chargeability sections were revealed and inferred attributed to Cretaceous sediments and faults. These faults are thought to have played critical role in the polymetallic mineralization genesis. In summary, this work demonstrated the successful of application of SSIP and AMT at detecting a metallic deposit in an area with high artificial noise. Hence the geophysical prospection potential of Gucheng-Yaxi area is great.

This study aimed to evaluate the discrepancies between genetic and visual coat color assignment in the Sarcidano Horse and to elucidate potential reasons. Individual DNA from 90 Sarcidano Horses was used for genetic assignment of coat color to explore the correspondence with individual forms containing phenotypical traits. The MC1R exon 1 and ASIP exon 3 have been genotyped and sequenced to obtain a picture of the coat color distribution in this breed. Surprisingly, once we compared the genetic results with the individual forms reporting the phenotypic data for each subject, a certain degree of non-correspondence between the phenotypic and genetic data in relation to coat color emerged. From the genetic analysis, Chestnuts (n = 58) resulted the most common Sarcidano Horse (n = 58), followed by a quite large number of Blacks (n = 28) and a very small number of Bays (n = 4), whereas phenotypic distribution resulted in 38 Chestnuts, 40 Bays, only 2 Blacks, and 10 Grays (without the possibility of recognizing the true color they carried). Chestnut resulted a very representative coat color, while many horses that visually identified as Bays were genetically Blacks. This discrepancy, that could be due to a variety of individual and external factors, including age, time of year, living situation and dietary condition, suggesting the importance of accurate coat color identification to ensure adequate features registration and reliable prediction of offspring’s coat color.

“A Picture is worth a thousand words”. Given an image, humans are able to deduce various cause-and-effect captions of past, current, and future events beyond the image. The task of visual commonsense generation aims at generating three cause-and-effect captions (1) what needed to happen before, (2) what is the current intent, and (3) what will happen after for a given image. However, such a task is challenging for machines owing to two limitations: existing approaches (1) directly utilize conventional vision-language transformers to learn relationships between input modalities, and (2) ignore relations among target cause-and-effect captions but consider each caption independently. We propose Cause-and-Effect BART (CE-BART) which is based on (1) Structured Graph Reasoner that captures intra- and inter-modality relationships among visual and textual representations, and (2) Cause-and-Effect Generator that generates cause-and-effect captions by considering the causal relations among inferences. We demonstrate the validity of CE-BART on VisualCOMET and AVSD benchmarks. CE-BART achieves SOTA performances on both benchmarks, while extensive ablation study and qualitative analysis demonstrate the performance gain and improved interpretability.

Visual inertial odometry (VIO) has recently received much attention for efficient and accurate ego-motion estimation of unmanned aerial vehicle systems (UAVs). Recent studies have shown that optimization-based algorithms achieve typically high accuracy when given enough amount of information, but occasionally suffer from divergence when solving highly non-linear problems. Further, their performance significantly depends on the accuracy of the initialization of inertial measurement unit (IMU) parameters. In this paper, we propose a novel VIO algorithm of estimating the motional state of UAVs with high accuracy. The main technical contributions are the fusion of visual information and pre-integrated inertial measurements in a joint optimization framework, and the stable initialization of scale and gravity using relative pose constraints. To count for ambiguity and uncertainty of VIO initialization, a local scale parameter is adopted in the online optimization. Quantitative comparisons with the state-of-the-art algorithms on the EuRoC dataset verify the efficacy and accuracy of the proposed method.

A monocular vision pose estimation and identification algorithm used on a small spacecraft for future orbital servicing is studied in this paper. A tracker spacecraft equipped with a short-range vision system is proposed to recover the 3D structural model of a space target in orbit and automatically identify its solar panels and main body using only visual information from an onboard camera. The proposed reconstruction and identification framework is tested using structure-from-motion and point cloud identification methods. The Efficient Perspective-n-Points (EPnP) descriptor is used for pose estimation. Triangulated points are used for component segmentation by means of orientation histogram descriptors. Experimental results based on laboratory images of a spacecraft model show the effectiveness and robustness of our approach.

The automatic classification of bird sounds is an ongoing research topic and several results have been reported for the classification of selected bird species. In this contribution we use an artificial neural network fed with pre-computed sound features to study the robustness of bird sound classification. We investigate in detail if and how classification results are dependent on the number of species and the selection of species in the subsets presented to the classifier. In more detail, a bag-of-birds approach is employed to randomly create balanced subsets of sounds from different species for repeated classification runs. The number of species present in each subset is varied between 10 and 300, randomly drawing sounds of species from a dataset of 659 bird species taken from Xeno-Canto database. We observe that the shallow artificial neural network trained on pre-computed sound features is able to classify the bird sounds relatively well. The classification performance is evaluated using several common measures such as precision, recall, accuracy, mean average precision and area under the receiver operator characteristics curve. All of these measures indicate a decrease in classification success as the number of species present in the subsets is increased. We analyze this dependence in detail and compare the computed results to an analytic explanation assuming dependencies for an idealized perfect classifier. Moreover, we observe that the classification performance depends on the individual composition of the subset and varies across 20 randomly drawn subsets.

This paper presents a novel approach for indoor acoustic source localization using microphone arrays and based on a Convolutional Neural Network (CNN). The proposed solution is, to the best of our knowledge, the first published work in which the CNN is designed to directly estimate the three dimensional position of an acoustic source, using the raw audio signal as the input information avoiding the use of hand crafted audio features. Given the limited amount of available localization data, we propose in this paper a training strategy based on two steps. We first train our network using semi-synthetic data, generated from close talk speech recordings, and where we simulate the time delays and distortion suffered in the signal that propagates from the source to the array of microphones. We then fine tune this network using a small amount of real data. Our experimental results show that this strategy is able to produce networks that significantly improve existing localization methods based on SRP-PHAT strategies. In addition, our experiments show that our CNN method exhibits better resistance against varying gender of the speaker and different window sizes compared with the other methods.

Radiological imaging is an essential component of a swallowing assessment. Artificial intelligence (AI), and especially deep learning (DL) models, have enhanced the efficiency and efficacy through which imaging is interpreted, and subsequently have important implications for swallow diagnostics and intervention planning. However, the application of AI for the interpretation of videofluoroscopic swallowing studies (VFSS) is still emerging. This review showcases recent literature in the use of AI to interpret VFSS and highlights clinical implications for speech pathologists (SPs). With a surge in AI research, there have been advances made in dysphagia assessment. Several studies have demonstrated successful implementation of DL algorithms to analyze VFSS. Notably, convolutional neural networks (CNNs) have been used to detect pertinent aspects of the swallowing process with high levels of precision. DL algorithms have the potential to streamline VFSS interpretation, improve efficiency and accuracy, and enable precise interpretation of instrumental dysphagia evaluation, which is especially advantageous when access to skilled clinicians is not ubiquitous. By enhancing precision, speed, and depth of VFSS interpretation, SPs can obtain a more comprehensive understanding of swallow physiology and deliver targeted and timely intervention that is tailored towards the individual. This has practical applications for both clinical practice and dysphagia research. As this research area grows and AI technologies progress, the application of DL in the field of VFSS interpretation is clinically beneficial and has the potential to transform dysphagia assessment and management. With broader validation and inter-disciplinary collaborations, AI-augmented VFSS interpretation is likely to transform swallow evaluation and ultimately improve outcomes for individuals with dysphagia.

Human and Computer interaction has been a part of our day-to-day life. Speech is one of the essential and comfortable ways of interacting through devices as well as a human being. The device, particularly smartphones have multiple sensors in camera and microphone, etc. speech recognition is the process of converting the acoustic signal to a smartphone as a set of words. The efficient performance of the speech recognition system highly enhances the interaction between humans and machines by making the latter more receptive to user needs. The recognized words can be applied for many applications such as Commands & Control, Data entry, and Document preparation. This research paper highlights speech recognition through ANN (Artificial Neural Network). Also, a hybrid model is proposed for audio-visual speech recognition of the Tamil and Malay language through SOM (Self-organizing map0 and MLP (Multilayer Perceptron). The Effectiveness of the different models of NN (Neural Network) utilized in speech recognition will be examined.

Despite advances in the reliability of sensory devices used by drones, the integrity of information from some devices is still considered an obstacle to ensuring successful flight plans. It is widely known that GNSS can suffer attacks or lose the signal from satellites, which can cause the drone to fail to complete its flight plan. In this context, we propose SiaN-VO, a Siamese network for visual odometry prediction. In our initial studies, this approach proved satisfactory for flights in static conditions (speed and height). Although interesting, these conditions do not reflect real flight conditions. In this sense, we have advanced our studies to propose the SiaN-VO, which fuses data from different sensors to enable displacement predictions to be made in dynamic flight conditions.

Hand gestures are an essential part of human-to-human communication and interaction, and therefore for technical applications, the aim is increasingly to achieve interaction between humans and computers that is as natural as possible, for example by means of natural language or hand gestures. In the context of human-machine interaction research, these methods are consequently being explored more and more. However, the realization of natural communication between humans and computers is a major challenge. In the field of hand gesture recognition, research approaches are being pursued that use additional hardware such as special gloves to classify gestures with high accuracy. Recently, deep learning techniques using artificial neural networks have been increasingly proposed for the problem of gesture recognition without using such tools. In this context, in our approach convolutional neural network (CNN) will be explored in detail for the task of hand gesture recognition. CNN is a deep neural network that can be used in the field of visual object processing and classification. The goal of this work is to recognize ten types of static hand gestures in front of complex backgrounds and different hand sizes based on raw images without the use of extra hardware. We achieved good results with a CNN network architecture consisting of seven layers. Through data augmentation and skin segmentation, a significant increase of the model accuracy was achieved. On public benchmarks the ten gestures have been classified almost perfectly with a testing accuracy of 96.5%

Positioning is an essential aspect of robot navigation, and visual odometry an important technique for continuous updating the internal information about robot position, especially indoors without GPS. Visual odometry is using one or more cameras to find visual clues and estimate robot movements in 3D relatively. Recent progress has been made, especially with fully integrated systems such as the RealSense T265 from Intel, which is the focus of this article. We compare between each other three visual odometry systems and one wheel odometry, on a ground robot. We do so in 8 scenarios, varying the speed, the number of visual features, and with or without humans walking in the field of view. We continuously measure the position error in translation and rotation thanks to a ground truth positioning system. Our result show that all odometry systems are challenged, but in different ways. In average, ORB-SLAM2 has the poorer results, while the RealSense T265 and the Zed Mini have comparable performance. In conclusion, a single odometry system might still not be sufficient, so using multiple instances and sensor fusion approaches are necessary while waiting for additional research and further improved products.

A low-resource emotional speech synthesis system for empathetic speech synthesis based on modelling prosody features is presented here. Secondary emotions, identified to be needed for empathetic speech, are modelled and synthesised in this paper. As secondary emotions are subtle in nature, they are difficult to model compared to primary emotions. They are also less explored, and this is one of the few studies that model secondary emotions in speech. Current speech synthesis research uses large databases and deep learning techniques to develop emotion models. There are many secondary emotions, and hence, developing large databases for each of the secondary emotions is expensive. This research presents a proof-of-concept using hand-crafted feature extraction and modelling of these features using a low resource-intensive machine learning approach, thus creating synthetic speech with secondary emotions. Here, a quantitative model-based transformation is used to shape the emotional speech fundamental frequency contour. Speech rate and mean intensity are modelled via rule-based approaches. Using these models, an emotional text-to-speech synthesis system to synthesise five secondary emotions - anxious, apologetic, confident, enthusiastic and worried is developed. A perception test to evaluate the synthesised emotional speech is also conducted.

Background: In order to achieve the best possible hearing and understanding with a cochlear implant (CI), regular hearing speech therapy treatment is necessary after implantation. This treatment should also be accessible to the growing proportion of hearing-impaired people with a migration background. This requires an alternative to therapy in the therapist's native language. The aim of this study was to evaluate six multilingual conversation applications with regard to their usefulness for therapy. Material and Methods: The six most commonly used applications were reviewed in terms of accuracy of content and grammatical translation, as well as pronunciation for English, Spanish, Arabic, Turkish, and Russian by native speakers. The number of available languages, availability, cost, and additional features were also analyzed. The accuracy of the content and grammatical translation as well as the pronunciation were statistically evaluated, and the differences were highlighted. The results of the different applications were compared with the performance of a native speaker. Results: All applications tested differed significantly from the native speaker level, with Google Translator showing the closest approximation to the native speaker level. All apps offer translations for multiple languages and, with exceptions, are available in both app stores. Furthermore, all apps have additional therapist-facilitating features. Conclusion: Multilingual conversation apps can make speech therapy in a foreign language much easier when used with patients. An adaptation of the software to the specific requirements of a hearing speech therapy is necessary to achieve a linguistic level that corresponds to the native language of the therapist and to enable an easy use in the therapy.

Nowadays, and with the mechanization of life, speech processing has become so crucial for the interaction between humans and machines. Deep neural networks require a database with enough data for training. The more features are extracted from the speech signal, the more samples are needed to train these networks. Adequate training of these networks can be ensured when there is access to sufficient and varied data in each class. If there is not enough data; it is possible to use data augmentation methods to obtain a database with enough samples. One of the obstacles to developing speech emotion recognition systems is the Data sparsity problem in each class for neural network training. The current study has focused on making a cycle generative adversarial network for data augmentation in a system for speech emotion recognition. For each of the five emotions employed, an adversarial generating network is designed to generate data that is very similar to the main data in that class, as well as differentiate the emotions of the other classes. These networks are taught in an adversarial way to produce feature vectors like each class in the space of the main feature, and then they add to the training sets existing in the database to train the classifier network. Instead of using the common cross-entropy error to train generative adversarial networks and to remove the vanishing gradient problem, Wasserstein Divergence has been used to produce high-quality artificial samples. The suggested network has been tested to be applied for speech emotion recognition using EMODB as training, testing, and evaluating sets, and the quality of artificial data evaluated using two Support Vector Machine (SVM) and Deep Neural Network (DNN) classifiers. Moreover, it has been revealed that extracting and reproducing high-level features from acoustic features, speech emotion recognition with separating five primary emotions has been done with acceptable accuracy.

Several researchers have contemplated deep learning-based post-filters to increase the quality of statistical parametric speech synthesis, which perform a mapping of the synthetic speech to the natural speech, considering the different parameters separately and trying to reduce the gap between them. The Long Short-term Memory (LSTM) Neural Networks have been applied successfully in this purpose, but there are still many aspects to improve in the results and in the process itself. In this paper, we introduce a new pre-training approach for the LSTM, with the objective of enhancing the quality of the synthesized speech, particularly in the spectrum, in a more efficient manner. Our approach begins with an auto-associative training of one LSTM network, which is used as an initialization for the post-filters. We show the advantages of this initialization for the enhancing of the Mel-Frequency Cepstral parameters of synthetic speech. Results show that the initialization succeeds in achieving better results in enhancing the statistical parametric speech spectrum in most cases when compared to the common random initialization approach of the networks.

As large language models (LLMs) continue to advance, evaluating their comprehensive capabilities becomes significant for their application in various fields. This research study comprehensively evaluates the language, vision, speech, and multimodal capabilities of GPT-4o. The study employs standardized exam questions, reasoning tasks, and translation assessments to assess the model's language capability. Additionally, GPT-4o's vision and speech capabilities are tested through image classification and object recognition tasks, as well as accent classification. The multimodal evaluation assesses the model's performance in integrating visual and linguistic data. Our findings reveal that GPT-4o demonstrates high accuracy and efficiency across multiple domains in language and reasoning capabilities, excelling in tasks that require few-shot learning. GPT-4o also provides notable improvements in multimodal tasks compared to its predecessors. However, the model shows variability and faces limitations in handling complex and ambiguous inputs, particularly in audio and vision capabilities. This paper highlights the need for more comprehensive benchmarks and robust evaluation frameworks, encompassing qualitative assessments involving human judgment as well as error analysis. Future work should focus on expanding datasets, investigating prompt-based assessment, and enhancing few-shot learning techniques to test the model's practical applicability and performance in real-world scenarios.

We present improvements in n-best rescoring of code-switched speech achieved by n-gram augmentation as well as optimised pretraining of long short-term memory (LSTM) language models with larger corpora of out-of-domain monolingual text. In addition, we consider the application of large pretrained transformer-based architectures. Our experimental evaluation is performed on an under-resourced corpus of code-switched speech comprising four bilingual code-switched sub-corpora, each containing a Bantu language (isiZulu, isiXhosa, Sesotho, or Setswana) and English. We find in our experiments that, by combining n-gram augmentation with the optimised pretraining strategy, speech recognition errors are reduced for each individual bilingual pair by 3.51% absolute on average over the four corpora. Importantly, we find that even speech recognition at language boundaries improves by 1.14% even though the additional data is monolingual. Utilising the augmented n-grams for lattice generation, we then contrast these improvements with those achieved after fine-tuning pretrained transformer-based models such as distilled GPT-2 and M-BERT. We find that, even though these language models have not been trained on any of our target languages, they can improve speech recognition performance even in zero-shot settings. After fine-tuning on in-domain data, these large architectures offer further improvements, achieving a 4.45% absolute decrease in overall speech recognition errors and a 3.52% improvement over language boundaries. Finally, a combination of the optimised LSTM and fine-tuned BERT models achieves a further gain of 0.47% absolute on average for three of the four language pairs compared to M-BERT. We conclude that the careful optimisation of the pretraining strategy used for neural network language models can offer worthwhile improvements in speech recognition accuracy even at language switches, and that much larger state-of-the-art architectures such as GPT-2 and M-BERT promise even further gains.