Structural magnetic resonance imaging (sMRI) is widely used in the clinical diagnosis of diseases due to its advantages: high definition and noninvasive. Therefore, computer-aided diagnosis based on sMRI images is broadly applied in classifying Alzheimer’s disease (AD). Due to the excellent performance of Transformer in computer vision, Vision Transformer (ViT) has been employed for AD classification in recent years. ViT relies on access to large datasets, while the sample size of brain imaging datasets is relatively insufficient. Moreover, the pre-processing procedures of brain sMRI images are complex and labor-intensive. To overcome the limitations mentioned above, we propose Resizer Swin Transformer (RST), a deep learning model that can extract information from brain sMRI images that are only briefly processed to achieve multi-scale and cross-channel features. In addition, we pre-trained our RST on a natural image dataset and obtained better performance. The experimental results of ADNI and AIBL datasets prove that RST can achieve better classification performance in AD prediction compared with CNN-based and Transformer models.

Breast cancer is the most prevalent type of disease among women. It has become one of the foremost causes of death among women globally. Early detection plays a significant role in administering personalized treatment and improving patient outcomes. Mammography procedures are often used to detect early-stage cancer cells. This traditional method of mammography while valuable has limitations of potential for false positives and negatives, patient discomfort, and radiation exposure. Therefore, there is a probe for more accurate techniques required in detecting breast cancer, leading to exploring the potential of machine learning in the classification of diagnostic images due to its efficiency and accuracy. This study conducted a comparative analysis of pre-trained CNNs (ResNet50 and VGG16) and Vision Transformers (ViT-Base and SWIN Transformer) with the inclusion of ViT-Base trained from scratch model architectures to effectively classify breast cancer mammographic images into benign and malignant cases. The Swin transformer exhibits superior performance with 99.8% accuracy and a precision of 99.8%. These findings demonstrate the efficiency of deep learning to accurately classify breast cancer mammographic images for the diagnosis of breast cancer, leading to improvement in patient outcomes.

Agriculture is pivotal in national economies, with pest detection significantly influencing food quality and quantity. Pest classification remains challenging in automated agriculture monitoring systems, exacerbated by the non-uniform pest scales and the scarcity of high-quality datasets. In this study, we constructed a pest dataset by acquiring domain-agnostic images from the Internet and resizing them to a standardized 299x299 pixel format. Additionally, we employed diffusion models to generate supplementary data. While Convolutional Neural Networks (CNNs) are prevalent for prediction and classification, they often lack effective global information integration and discriminative feature representation. To address these limitations, we propose the RS Transformer, an innovative model that combines elements like the Region Proposal Network, Swin Transformer, and ROI Align. Additionally, we introduce the Randomly Generated Stable Diffusion Dataset (RGSDD) to augment the availability of high-quality pest datasets. Extensive experimental evaluations demonstrate the superiority of our approach compared to both two-stage models (SSD and Faster R-CNN) and one-stage models (YOLOv3, YOLOv4, YOLOv5m, YOLOv8, and DETR). We rigorously assess performance using metrics such as mean Average Precision (mAP), F1Score, Recall, and mean Detection Time (mDT). Our research contributes to advancing pest detection methodologies in automated agriculture systems, promising improved food production and quality.

In recent years, the analysis of macro- and micro-expression has drawn the attention of researchers since they provide visual cues of an individual's emotions for a broad range of potential applications such as lie detection and criminal detection. In this paper, we address the challenge of spotting facial macro- and micro-expression from videos and present compelling results by using a deep learning approach to analyze the optical flow features. Different from other deep learning approaches that are mainly based on Convolutional Neural Networks (CNNs), we propose a Transformer-based deep learning approach that predicts a score indicating the probability of a frame being within an expression interval. In contrast to other Transformer-based models that achieve high performance by being pre-trained on large datasets, our deep learning model, called SL-Swin, which incorporates Shifted Patch Tokenization and Locality Self-Attention into the backbone network Swin Transformer, effectively spots macro- and micro-expressions by being trained from scratch on small-size expression datasets. Our evaluation outcomes surpass the MEGC 2022 spotting baseline result, obtaining an overall F1-score of 0.1366. Additionally, our approach also performs well in the MEGC 2021 spotting task with an overall F1-score of 0.1824 and 0.1357 on CAS(ME)^2 and SAMM Long Videos, respectively. The code is publicly available on GitHub (https://github.com/eddiehe99/pytorch-expression-spotting).

Multimodal knowledge graph completion necessitates the integration of information from multiple modalities (such as images and text) into the structural representation of entities to improve link prediction. However, most existing studies have overlooked the interaction between different modalities. To address this issue, this paper proposed a Transformer-based knowledge graph link prediction model (MM-Transformer) that fuses multimodal features. Different modal encoders are employed to extract structural, visual and textual features, and hybrid key-value calculations are performed on features from different modalities based on the Transformer architecture. The similarities of textual tags to structural tags and visual tags are calculated and aggregated respectively, and multimodal entity representations are modeled and optimized to reduce the heterogeneity of the representations. Experimental results demonstrate that, compared to the current multimodal state-of-the-art methods, the proposed method achieves significant performance improvements in knowledge graph link prediction tasks. This proves that the proposed method effectively addresses the problem of multimodal feature fusion in knowledge graph link prediction tasks.

Color image binarization plays a pivotal role in image preprocessing work, significantly impacting subsequent tasks, particularly in text recognition. This paper concentrates on Document Image Binarization (DIB), aiming to separate a image into foreground (text) and background (non-text content). Through a thorough analysis of conventional and deep learning-based approaches, we conclude that prevailing DIB methods leverage deep learning technology. Furthermore, we explore the receptive fields pre- and post-network training to underscore the Transformer model's advantages. Subsequently, we introduce a lightweight model based on the U-Net structure, enhanced with the Mobile ViT module to better capture global information features in document images. Given its adeptness at learning both local and global features, our proposed model exhibits superior performance on two standard datasets (DIBCO2012 and DIBCO2017) compared to state-of-the-art methods. Notably, our proposed DIB method presents a straightforward end-to-end model devoid of additional image preprocessing or post-processing. Moreover, its parameter count is less than a quarter of the HIP'23 model, which achives best results on three datasets(DIBCO2012, DIBCO2017 and DIBCO2018). Finally, two sets of ablation experiments were conducted to verify the effectiveness of the proposed binarization model.

One of the most crucial elements in deeply understanding humans on a psychological level is manifested through facial expressions. The analysis of a human behavior can be informed by their facial expressions, making it essential to employ indicators such as expression (Expr), valence-arousal (VA), and action units (AU). In this paper, we introduce the method proposed in the Challenge of the 6th Workshop and Competition on Affective Behavior Analysis in-the-wild (ABAW) at CVPR 2024. Our proposed method utilizes the multi-modal Aff-wild2 dataset, which is splitted into spatial and audio modalities. For the spatial data, we extract features using a SimMiM model that was pre-trained on a diverse set of facial expression data. For the audio data, we extract features using a WAV2VEC model. To fusion the extracted spatial and audio features, we employed the cascaded cross-attention mechanism of a transformer.

In the era of Artificial Intelligence (AI), comprehending and responding to non-verbal communication is increasingly vital. This research extends AI's reach in bridging communication gaps, notably benefiting American Sign Language (ASL) and Taiwan Sign Language (TSL) communities. It focuses on employing various AI models, especially the Hierarchical Vision Transformer with Shifted Windows (Swin), for recognizing diverse sign language datasets. The study assesses Swin architecture's adaptability to different sign languages, aiming to create a universal platform for Unvoiced communities. Utilizing deep learning and transformer technologies, hybrid application prototypes have been developed for ASL-to-English translations and vice versa, with plans to expand this to multiple sign languages. The Swin models, trained on varied dataset sizes, show considerable accuracy, indicating their flexibility and effectiveness. This research underscores major advancements in sign language recognition and underlines a commitment to inclusive communication in the digital era. Future work will focus on enhancing these models and broadening their scope to include more sign languages, integrating multimodality and Large Language Models (LLMs), thereby fostering global inclusivity.

Against the background of the sky, imaging and perception of aircraft are cru- cial. Among these, the location of aircrafts is not a very difficult problem in general, because that deep learning algorithms and object detection models based on convolutional neural network(CNN) are ever-evolving. In order to have a better identification of various aircrafts, or to accurately locate different types of flying targets in the sky, it becomes significant to distinguish and recognize different types of aircrafts. One question is that, distinguishing and recognizing between sub-categories of aircrafts pose great challenges. Although fine-grained recognition focuses on exploring and studying such problems, aircrafts under different sub-categories and granularities lead us to rethink the application of features. We noticed that features in swin-transformer demonstrates the under- standing and processing of images, fully showcasing the encoding and indexing of information. Through this research and proposed approach, we discovered a better understanding of features encoding and use, which are inspired by the feature encoding and computation in swin-transformer. In our paper, our ap- proach has achieved effects on features encoded graphically, manifested from the architecture design and convolutional neural network computation, and outper- forms other famous fine-graiend classification models on this issue. Not only approach we proposed has demonstrated superior performance in fine-grained aicraft classification, but also the mechanisms of the feature encoding under different sample space partitions is revealed for this issue, which provides a re- search for the representation of aircraft fineness characteristics. The relationship between representation orientation of aircraft feature information under various grained divisions, shows that the recognition of different categories according to man-made, such as aircraft, can be achieved by means of the feature represen- tations studied in this paper, for more specific defined classification tasks and various man-made targets partition criteria, which may influences the princi- ple of design for calculation and feature extraction in fine-grained classification models.

Systematic reviews represent a powerful instrument to summarize the existing evidence in medical literature. However, articles for a systematic review are hard to identify, and mostly require a structured search of the literature through a number of databases, using keyword-based search strategies, followed by the painstaking manual selection of pieces of evidence that are pertinent to the query. A.I. algorithms may offer solutions to reduce the workload on investigators. We applied BERTopic, a newer and much-recognized transformer-based topic-modeling algorithm, to two datasets of 6137 and 5309 articles of newly published systematic reviews in the area of peri-implantitis and bone regeneration in implant dentistry. In the two datasets, BERTopic identified 14 and 22 clusters, respectively, and it automatically created labels describing the nature of the topics for each individual cluster based on semantic interpretation of their titles. Most themes regarded the query theme, but in both conditions, BERTopic also uncovered articles related to off-themes, which composed around 30% of the dataset—with sensitivity up to 0.79 and specificities of at least 0.99. Our results suggest that adding a topic-modeling step to the screening process could potentially save working hours for researchers involved in systematic reviews of the literature.

Ocean exploration is crucial for utilizing its extensive resources. Images captured by underwater robots suffer from issues such as color distortion and reduced contrast. To address the issue, we propose an innovative enhancement algorithm that integrates Transformer and Convolutional Neural Network (CNN) in a parallel fusion manner. Firstly, a novel transformer model is intro-duced to capture local features, employing peak-signal-to-noise ratio (PSNR) attention and linear operations. Subsequently, to extract global features, both temporal and frequency domain features are incorporated to construct convolutional neural network. Finally, the Fourier’s high and low-frequency information of the original image are utilized to fuse different features. To demon-strate the algorithm's effectiveness, underwater images with various levels of color distortion are selected for both qualitative and quantitative analyses. The experimental results demonstrate that our approach surpasses other mainstream methods, achieving superior PSNR and structural sim-ilarity index measure (SSIM) metrics and leading to a detection performance improvement of over ten percent.

The research investigates the factors affecting the financial performance of Micro, Small, and Medium Enterprises (MSMEs) in Indonesia and their impact on the country’s economic growth. The research methods of the studies involve data collection through surveys or interviews, followed by statistical analysis. The study found that entrepreneurial qualities such as motivation, entrepreneurial orientation, and risk behavior positively influence the financial performance of MSMEs. MSMEs that manage working capital optimally tend to increase profitability and growth, despite operational difficulties like worker shortages. MSMEs engaged in foreign trade usually achieve higher profitability. Bank loans can enhance profitability and sales, but the collateral requirement may hinder growth. The owner’s involvement doesn’t affect growth but can enhance profitability. Factors like the Human Capital Index, interest rates, the Indonesian Rupiah exchange rate, and inflation do not significantly impact MSME investment. However, business units, labor, investment, and exports of MSMEs significantly influence Indonesia’s economic growth. MSMEs play a crucial role in job creation, rural development, economic diversification, and GDP contribution in Indonesia. Despite challenges, they have the potential for sustainable growth. The research also explores strategies for overcoming challenges faced by MSMEs and investigates the role of universities in supporting the MSME sector. The University of Indonesia (UI) supports the MSME sector through strategic initiatives, including education, training, mentoring, and internship programs. The research provides valuable insights into the operation of MSMEs in Indonesia and their contribution to the economy, highlighting areas for further exploration and potential strategies for supporting the growth and sustainability of these enterprises.

Non-toroidal shape primary pass through protection current transformers (CTs) are used to measure high currents. Their design provides them with a big airgap that allow passing several cables per phase though them, which is the main advantage versus toroidal types, as the number of CTs required to measure the whole phase current is drastically reduced. The cables pass through the transformer window can be in several positions. As the isolines of the magnetic field generated by the primary currents are centered in the cables, if these cables are not centered in the transformer window, the magnetic field will be non-uniform along the transformer core. Consequently, local saturations can appear if the cables are not properly disposed causing the malfunction of the CT. In this paper, the performance of a non-toroidal shape protection CT is studied. The research is focused on the cable position influence in possible partial saturations of the CT when it is operating near to its accuracy limit. Depending on the cable position, the ratio of the primary and secondary currents can depart from assigned ratio. The validation of this phenomenon has been carried out via Finite Element Analysis (FEA) showing that partial transformer core saturations appears in areas of the magnetic core close to the cable. Applying FEA, also the admissible accuracy region for cable positioning inside the CT is delimited. Finally, the simulation results are ratified with experimental tests performed in non-toroidal protection CTs varying the primary cables’ positions, which are subjected to currents up to 5 kA, achieving satisfactory results. From this analysis, installation recommendations are given.

Few-shot object detection (FSOD) aims to address the challenge of requiring a substantial amount of annotations for training in conventional object detection, which is very labor-intensive. However, existing few-shot methods achieve high precision with the sacrifice of time-consuming for exhaustive fine-tuning, or take poor performance in novel-class adaptation. We presume the major reason is that the valuable correlation feature among different categories is insufficiently exploited, hindering the generalization of knowledge from base to novel categories for object detection. In this paper, we propose Few-Shot object detection via Correlation-RPN and Transformer Encoder-Decoder (CRTED), a novel training network to learn object-relevant features of inter-class correlation and intra-class compactness while suppressing object-agnostic features in the background with limited annotated samples. And we also introduce a 4-way tuple-contrast training strategy to positively activate the training progress of our object detector. Experiments over two few-shot benchmarks (Pascal VOC, MS-COCO) demonstrate that, our proposed CRTED without further fine-tuning can achieve comparable performance with current state-of-the-art fine-tuned works. The codes and pre-trained models will be released.

The objective of person re-identification (ReID) tasks is to match a specific individual across different times, locations, or camera viewpoints. The prevalent issue of occlusion in real-world scenarios affects image information, rendering the affected features unreliable. The difficulty and core challenge lie in how to effectively discern and extract visual features from human images under various complex conditions, including cluttered backgrounds, diverse postures, and the presence of occlusions. Some works have employed pose estimation or human keypoint detection to construct graph-structured information to counteract the effects of occlusions. However, this approach introduces new noise due to issues such as the invisibility of keypoints. Our proposed module, in contrast, does not require the use of additional feature extractors. Our module employs multi-scale graph attention for the reweighting of feature importance. This allows features to concentrate on areas genuinely pertinent to the re-identification task, thereby enhancing the model's robustness against occlusions. To address these problems, a model that employs multi-scale graph attention to re-weight the importance of features is proposed in this study, significantly enhancing the model’s robustness against occlusions. Our experimental results demonstrate that, compared to baseline models, the method proposed herein achieves a notable improvement in mAP on occluded data sets, with increases of 0.5%, 31.5%, and 12.3% in mAP scores.

Despite significant advancements in Artificial Intelligence (AI) and Large Language Models (LLMs), detecting and mitigating bias remains a critical challenge, particularly within social media platforms like X (formerly Twitter) and in addressing cyberbullying present on them. This research investigates the effectiveness of leading LLMs in generating synthetic biased and cyberbullying data and evaluates the proficiency of Transformer AI models in detecting bias and cyberbullying within both authentic and synthetic contexts. The study involves semantic analysis and feature engineering on a dataset of over 48,000 sentences related to cyberbullying collected from Twitter (before it became X). Leveraging state-of-the-art LLMs such as ChatGPT-4o, Pi AI, Claude 3 Opus, and Gemini-1.5, synthetic biased, cyberbullying, and neutral data were generated to deepen the understanding of bias in human-generated data. AI models including DeBERTa, Longformer, BigBird, HateBERT, MobileBERT, DistilBERT, BERT, RoBERTa, ELECTRA, and XLNet were initially trained to classify Twitter cyberbullying data and subsequently fine-tuned, optimized, and quantized for multilabel classification (detecting both biases and cyberbullying). The study's outcomes include a prototype of a hybrid application that combines a Bias Data Detector and a Bias Data Generator, validated through extensive testing.

A Partial discharge (PD) occurrence in high-voltage (HV) and medium-voltage (MV) power assets is a warning sign of insulation system degradation or defective manufacturing/assembly, that may cause catastrophic damage and eventually lead to the breakdown of the power equipment. Ultra-high frequency (UHF) PD measurement technology has extensively been used to monitor the insulation health of the HV or MV equipment due to its high immunity against ambient noise and conducted disturbances. This paper presents experimental validation of a new design of a UHF planar rectangular microstrip antenna that can be used for PD monitoring of HV and/or MV power equipment, such as power transformers. The designed planar rectangular microstrip antenna has a compact size of 80mmx70mmx1.6mm. Laboratory experiments were performed to validate the antenna sensitivity to PD activity caused by insulation defects created inside an oil-filled transformer tank model. PD measurement results confirm that the proposed rectangular microstrip antenna has high sensitivity and high signal-to-noise ratio (SNR), thanks to its capability to suppress external noise disturbances, especially during online PD monitoring. Thus, the proposed UHF antenna seems to be a promising and alternative PD sensor that can be employed in UHF PD monitoring systems of HV or MV power equipment.

This study introduces a sophisticated neural network structure for segmenting breast tumors. It achieves this by combining a pretrained Vision Transformer (ViT) model with a U-Net framework. The U-Net architecture, commonly employed for biomedical image segmentation, is further enhanced with Depthwise Separable Convolutional Blocks to decrease computational complexity and parameter count, resulting in better efficiency and less overfitting. The Vision Transformer, renowned for its robust feature extraction capabilities utilizing self-attention processes, efficiently captures the overall context within images, surpassing the performance of conventional convolutional networks. By using a pretrained ViT as the encoder in our U-Net model, we take advantage of its extensive feature representations acquired from extensive datasets, resulting in a major enhancement in the model’s ability to generalize and train efficiently. The suggested model has exceptional performance in segmenting breast cancers from medical images, highlighting the advantages of integrating transformer-based encoders with efficient U-Net topologies. This hybrid methodology emphasizes the capabilities of transformers in the field of medical image processing and establishes a new standard for accuracy and efficiency in activities related to tumor segmentation.

Neural decoding from spiking activity is an essential tool for understanding the information encoded in population neurons, especially in applications like brain-computer interface (BCI). Various quantitative methods have been proposed and have shown superiorities under different scenarios respectively. From the machine learning perspective, the decoding task is to map the high-dimensional spatial & temporal neuronal activity to the low-dimensional physical quantities (e.g., velocity, position). Because of the complex interactions and the abundant dynamics among neural circuits, good decoding algorithms usually have the capability of capturing flexible spatiotemporal structures embedded in the input feature space. Recently, the Transformer-based models are widely used in processing natural languages and images due to its superior performances in handling long-range and global dependencies. Hence, in this work we examine the potential applications of Transformers in neural decoding and introduce two Transformer-based models. Besides adapting the Transformer to neuronal data, we also propose a data augmentation method for overcoming the data shortage issue. We test our models on three experimental datasets and their performances are comparable to the previous state-of-the-art (SOTA) RNN-based methods. In addition, Transformer-based models show increased decoding performances when the input sequences are longer, while LSTM-based models deteriorate quickly. Our research suggests that Transformer-based models are important additions to the existing neural decoding solutions, especially for large datasets with long temporal dependencies.

As split learning (SL) becomes popular for addressing data privacy and resource limitation issues in edge model training process, it is crucial to pre-validate whether the extensive use of wireless and computing resources in SL can achieve the expected training latency and convergence. While digital twin network (DTN) can potentially estimate network strategy performance in pre-validation environments, they are still in their infancy for SL tasks, facing challenges like unknown non-i.i.d. data distributions, inaccurate channel states, and misreport resource availability across devices. To address these challenges, this paper proposes a TransNeural algorithm for DTN pre-validation environment to estimate SL latency and convergence. First, the TransNeural algorithm integrates transformers to efficiently model data similarities between different devices, considering different data distributions and device participate sequence greatly influence SL training convergence. Second, it leverages neural network to automatically establish the complex relationships between SL latency and convergence with data distributions, wireless and computing resources, dataset sizes, and training iterations. Deviations in user reports are also accounted for in the estimation process. Simulations show that the TransNeural algorithm improves latency estimation accuracy by 13.44% and convergence estimation accuracy by 116.74% compared to traditional equation-based methods.

In this paper, we introduce Syntax-Enhanced Transformer Model (SET), a groundbreaking approach in the realm of Transformer-based language modeling that seeks to redefine the boundaries of linguistic analysis and comprehension. SET innovatively combine (i) the well-established high-level performance, scalability, and adaptability of traditional Transformers with (ii) a sophisticated analysis of syntactic structures. This synergy is enabled by a novel attention mechanism tailored to parse syntactic nuances and a deterministic process adept at transforming linearized parse trees into meaningful linguistic representations. Our comprehensive experiments reveal that SET significantly advance the field by surpassing existing benchmarks in sentence-level language modeling perplexity. They exhibit exceptional proficiency in tasks that require an acute awareness of syntax, setting new standards for language models in understanding complex linguistic structures. Furthermore, SET demonstrate an enhanced capability to grasp nuanced linguistic patterns that have traditionally been challenging for standard Transformer models. However, our studies also uncover a unique aspect of SET: while they excel in sentence-level tasks, their representation of sentences as singular vectors—owing to the syntactic composition constraints intrinsic to their design—introduces certain limitations in document-level language modeling. This observation points to an intriguing area for future exploration; it suggests the potential need for an alternative or complementary memory mechanism within Transformer models, one that functions independently from, yet in harmony with, syntactic structures. Such a mechanism could be pivotal in enhancing the model's ability to comprehend and process long-form texts effectively. In conclusion, SET mark a significant stride in the journey towards more sophisticated, syntax-aware language models. They offer promising insights into the integration of deep linguistic knowledge with cutting-edge machine learning techniques, potentially opening doors to a new era of natural language understanding and processing.

The evolution of sentence representation learning, especially with parse tree encoders, has shown remarkable progress. Traditional approaches predominantly rely on recursive encoding of tree structures, which impedes parallel processing capabilities. Additionally, these methods often overlook the significance of dependency tree arc labels. To overcome these limitations, we introduce the Syntax-Enhanced Transformer (SET), incorporating a novel dual-attention mechanism that integrates relation-focused attention alongside traditional self-attention. This design effectively encodes both dependency and spatial positional relationships within sentence dependency trees. Our approach innovatively incorporates syntactic information into the Transformer framework without compromising its inherent parallelizability. The SET demonstrates superior or comparable performance to contemporary methods across various sentence representation tasks, significantly enhancing computational efficiency.

Recent advancements in neural language models, particularly attention-based architectures like the Transformer, have substantially surpassed traditional methods in various natural language processing tasks. These models adeptly generate nuanced token representations by considering contextual relations within a sequence. However, augmenting these models with explicit syntactic knowledge, such as part of speech tags, has been found to remarkably bolster their effectiveness, especially under constrained data scenarios. This study introduces the Linguistic Enhanced Transformer (LET), which integrates multiple syntactic features, showing a notable increase in translation accuracy, evidenced by an improvement of up to 1.99 BLEU points on subsets of the WMT '14 English-German dataset. Furthermore, this paper demonstrates that enriching BERT models with syntax-aware embeddings enhances their performance on several GLUE benchmark tasks.

Time series forecasting has been a challenging area in the field of Artificial Intel-ligence. Various approaches such as linear Neural Networks, Recurrent Linear Neural Networks, Convolutional Neural Networks and recently Transformers have been attempted for the time series forecasting domain. Although, Trans-former-based architectures have been outstanding in the Natural Language Pro-cessing domain, especially in autoregressive language modeling, the initial at-tempts to use transformers in the time series arena have met with mixed success. One of the recent papers demonstrated that modeling the multi-variate time se-ries problem as independent channels on a transformer architecture produced lower mean square and mean absolute errors on benchmarks, however, another recent paper claimed that a simple linear neural network out-performs previous approaches including transformer-based designs. We investigate this paradox in detail comparing the linear Neural Network and Transformer based designs, provide insights to why a certain approach may be better for a particular type of problem. We also improve upon the recently proposed simple linear Neural Network based architecture by using dual pipelines with batch normalization and reversible instance normalization. Our enhanced architecture outperforms all existing architectures for time series forecasting on majority of the popular benchmarks.

Multispectral image (MSI) and hyperspectral image (HSI) fusion (MHIF) aims to address the challenge of acquiring high-resolution (HR) HSI images. This field combines a low-resolution (LR) HSI with an HR-MSI to reconstruct HR-HSI. Existing methods directly utilize transformers to perform feature extraction and fusion. Despite the demonstrated success, there exist two limitations: 1) Employing the entire transformer model for feature extraction and fusion fails to fully harness the transformer’s potential in integrating the spectral of the HSI and spatial information of the MSI. 2) HSI has a strong spectral correlation and exhibits sparsity in the spatial domain. Existing transformer-based models do not optimize this physical property, which makes their methods prone to spectral distortion. To accomplish these issues, this paper introduces a novel framework for MHIF called Sparse Mix-Attention Transformer (SAMformer). Specifically, to fully harness the advantages of the Transformer architecture, we propose a Spectral Mix Attention Block (SMAB), which concatenates the keys and values extracted from LR-HSI and HR-MSI to create a new multi-head attention module. This design facilitates the extraction of detailed long-range information across spatial and spectral dimensions. Besides, to address the spatial sparsity inherent in HSI, we incorporated a sparse mechanism within the core of SMAB called Sparse Spectral Mix Attention Block (SSMAB). In the SSMAB, we compute attention maps from queries and keys and select the K highly correlated values as the sparse attention map. This approach enables us to achieve a sparse representation of spatial information while eliminating spatially disruptive noise. Extensive experiments conducted on three benchmark datasets, namely Cave, Harvard, and Pavia Center, demonstrate the SMAformer method outperforms state-of-the-art methods.

Heat transfer fluids are used in various industrial systems to maintain them in perfect operating conditions. Extensive research efforts have been dedicated to enhance the thermal properties of these heat transfer fluids to improve their efficiency. Developing nanofluids is a potential candidate for such enhancements. This study investigates the impact of incorporating TiO2 nanoparticles into two types of oils: transformer oil (NYTRO LIBRA) and virgin coconut oil (manufactured by Govi Aruna Pvt. Ltd.) at different temperatures and with varying volume fractions. The nanofluids were prepared using a two-step method by adding CTAB (Cetyltrimethylammonium bromide) surfactant. To minimize nanoparticle agglomeration, this study employed relatively low-volume fractions. Thermal properties by means of thermal conductivity, thermal diffusivity, and volumetric heat capacity were measured in accordance with ASTM (American Society for Testing and Materials) standard methods, using a multifunctional thermal conductivity meter (LAMBDA thermal conductivity meter). The measured thermal conductivity values were compared with theoretical models and previous research findings. It was confirmed that the modification of thermal properties was enhanced by doping TiO2 nanoparticles with different volume fraction.

Identification of plants is a challenging task which aims to identify the family, genus, and species level according to morphological features. Automated deep learning-based computer vision algorithms are widely used for identifying plants and can help users to narrow down the possibilities. However, numerous morphological similarities between and within species make the classification difficult. In this paper, we tested a custom convolution neural network (CNN) and vision transformer (ViT) based models using the PyTorch framework to classify plants. We used a large dataset of 88K and 16K images for classifying plants at genus and species levels respectively. Our results show that for classifying plants at the genus level, ViT models perform better compared to CNN-based models ResNet50 and ResNet-RS-420, and other state-of-the-art CNN-based models suggested in previous studies on a similar dataset. The ViT model achieved top accuracy of 83.3% for classifying plants at the genus level. ViT models also perform better for classifying plants at the species level compared to CNN-based models ResNet50 and ResNet-RS-420, with a top accuracy of 92.5%. We show that the correct set of augmentation techniques plays an important role in classification success.

This paper describes the successful application of the Transformer model used in the natural language processing and vision tasks as a means of processing the time series of signals from gyroscope and accelerometer sensors for the classification of human activities. The Transformer model is based on deep neural networks with many layers which can generalize well on signals. All measured signals come from a smartphone placed in a waist bag. Activity prediction is sequence-to-sequence, each time step of the signal is assigned a designation of the performed activity. Emphasis is placed on attention mechanisms, which express individual dependencies between signal values within a time series. In comparison with another recent result, the recognition precision was improved from 89.67 percent to 99.2 percent. The transformer model should in the future be included among the top options in machine learning methods for human activity recognition.

Synthetic Aperture Radar has been extensively used in numerous fields and can gather a wealth of information about the area of interest. This large-scene data-intensive technology puts a high value on automatic target recognition (ATR) which can free the utilizers and boost the efficiency. Recent advances in artificial intelligence have made it possible to create a deep learning-based SAR ATR that can automatically identify target features from massive input data. In the last 6 years, intensive research has been conducted in this area, however, most papers in the current SAR ATR field used recurrent neural network (RNN) and convolutional neural network (CNN)-varied models to deepen the regime’s understanding of the SAR images. To equip SAR ATR with updated deep learning technology, this paper tries to apply a lightweight vision transformer (LViT)-based model to classify SAR images. The entire structure was verified by an open- accessed SAR data set and recognition results show that the final classification outcomes are robust and more accurate in comparison with referred traditional network structures without even using any convolutional layers.

Blood cell analysis is a crucial diagnostic process in medical practice. In particular, detecting white blood cells (WBCs) is essential for diagnosing of many diseases. The manual screening of blood films is a time-consuming and subjective process, which can lead to inconsistencies and errors. Therefore, automated detection of blood cells can improve the accuracy and efficiency of the screening process. In this study, an explainable Vision Transformer (ViT) model was proposed for the automatic detection of WBCs from blood films. The proposed model utilizes the self-attention mechanism to extract relevant features from the input images and leverages transfer learning by incorporating pre-trained model weights to improve its performance. The proposed model achieved a classification accuracy of 99.40% for five distinct types of WBCs and exhibited potential in reducing the time required for manual screening of blood films by pathologists. Upon examination of the misclassified test samples, it was observed that incorrect predictions were correlated with the presence or absence of granules in the cell samples. To validate this observation, the dataset was divided into two classes, namely Granulocytes and Agranulocytes, and a secondary training process was conducted. The resulting ViT model trained for binary classification achieved an accuracy of 99.70%, recall of 99.54%, precision of 99.32%, and F-1 score of 99.43% during the test phase. To ensure the reliability of the ViT model's multi-class classification of WBCs, the pixel areas that the model focuses on in its predictions are visualized through the Score-CAM algorithm.

Estimating the Remaining Useful Life (RUL) of aircraft engines holds a pivotal role in enhancing safety, optimizing operations, and promoting sustainability, thus being a crucial component of modern aviation management. Precise RUL predictions offer valuable insights into an engine’s condition, enabling informed decisions regarding maintenance and crew scheduling. In this context, we propose a novel RUL prediction approach in this paper, harnessing the power of Bi-directional LSTM and Transformer architectures, known for their success in sequence modeling, such as natural languages. We adopt the encoder part of the full Transformer as the backbone of our framework, integrating it with a self-supervised denoising autoencoder that utilizes Bidirectional LSTM for improved feature extraction. Within our framework, a sequence of multivariate time series sensor measurements serves as the input, initially processed by the Bidirectional LSTM autoencoder to extract essential features. Subsequently, these feature values are fed into our Transformer encoder backbone for RUL prediction. Notably, our approach simultaneously trains the autoencoder and Transformer encoder, different from the naive sequential training method. Through a series of numerical experiments carried out on the C-MAPSS datasets, we demonstrate that the efficacy of our proposed models either surpasses or stands on par with that of other existing methods.

Indoor localization is an active area of research dominated by traditional machine-learning techniques. Deep learning-based systems have shown unprecedented improvements and have accomplished exceptional results over the past decade, especially the Transformer network within natural language processing (NLP) and computer vision domains. We propose the hyper-class Transformer (HyTra), an encoder-only Transformer with multiple classification heads (one per class) and learnable embeddings, to investigate the effectiveness of Transformer-based models for received signal strength (RSS) based WiFi fingerprinting. HyTra leverages learnable embeddings and the self-attention mechanism to determine the relative position of the wireless access points (WAPs) within the high-dimensional embedding space, improving the prediction of user location. From an NLP perspective, we consider a fixed order sequence of all observed WAPs as a sentence and the captured RSS value(s) for every given WAP at a given reference point from a given user as words. We test our proposed network on public and private datasets of different sizes, proving that the quality of the learned embeddings and overall accuracy improves with increments in samples.

In this research paper, a high gain transformer less inverter modeling which is used for the smart grid technology or as the stand alone for home appliances is proposed. The proposed transformer less topology provides a higher voltage gain, low cost, small size and simple control stand alone transformer less inverter for home appliances. The above inverter is modeled by using software which is known as MATLAB/ Simulimk.

Visual object tracking is an important technology in camera based sensor networks, which has a wide range of practicability in auto drive system. A transformer is a deep learning model that adopts the mechanism of self-attention, and it differentially weights the significance of each part of the input data. It has been widely applied in the field of visual tracking. Unfortunately, the security of the transformer model is unclear. It makes such transformer-based applications be exposed to security threats. In this work, the security of the transformer model is investigated with the important component of autonomous driving, visual tracking. Such deep-learning-based visual tracking is vulnerable to adversarial attacks, so adversarial attacks are implemented as the security threats to conduct the investigation. First, adversarial examples are generated on top of video sequences to degrade tracking performance, and the frame-by-frame temporal motion is taken into consideration when generating perturbations over the predicted tracking results. Then, the influence of perturbations on performance is sequentially investigated and analyzed. Finally, numerous experiments on OTB100, VOT2018, and GOT-10k data sets demonstrate that the executed adversarial examples are effective on the performance drops of the transformer-based visual tracking.

Electrification of society, increasing renewable energy sources and mobility charging lead to new loading patterns for power transformers. Dynamic load conditions induce enhanced mechanical stress on the transformers’ windings, potentially causing degradation of the solid insulation over time and compromising the transformer's short-circuit withstand capability. Thermal expansion of the windings, caused by losses in the copper conductors, occurring as the transformer is load-ed, increases the stress. Conversely, magnetic losses in the core and tie plate expansion contribute to a reduction in stress. This paper presents the effect of step changes in core losses and copper losses by on-line measurements of the clamping pressure, to better understand the mechanical stresses acting upon the solid insulation cellulose materials.

Detection of partial discharge (PD) activities in high-voltage equipment can be done according to the several mechanisms of signal detection, which are electromagnetic wave signal detection, acoustic signal detection, chemical reactions, electrical signal detection, or optical emission detection. Recently, multiple methods of detection and localization of partial discharge activities occurred in power transformers and gas-insulated switchgear (GIS) have been proposed to monitor the health condition of high-voltage equipment, especially when the awareness regarding preventive maintenance has been emphasized at the industrial level and electrical providers. In aligning the needs of industrial and the improvement of PD detection methods, this manuscript focused on reviewing the current practice methods for the detection and localization of PD signals in high-voltage equipment and comparing their efficacy, in summarizing the future direction of research work-related methods of PD detection. The comparative reviews are discussed in terms of the mechanism of PD signal detection, indication parameters, calibration technique, advantages, and limitations of each method of PD measurement in detail.

Diabetic retinopathy, a common complication of diabetes, is further exacerbated by factors such as hypertension and obesity. This study introduces the Diabetic Retinopathy Convolutional Transformer (DRCT) model, which combines convolutional and transformer techniques to enhance the classification of retinal images. The DRCT model achieved an impressive average F1 score of 0.97, reflecting its high accuracy in detecting true positives while minimizing false positives. Throughout 100 training epochs, the model exhibited strong generalization capabilities, achieving superior validation accuracy with minimal overfitting. On a newly evaluated dataset, the model attained precision and recall scores of 96.93% and 98.89%, respectively, indicating a well-balanced handling of false positives and false negatives. The model's ability to classify retinal images into five distinct diabetic retinopathy categories demonstrates its potential to significantly improve automated diagnosis and aid in clinical decision-making.

Recommendation systems are crucial in navigating the vast digital market. However, user data’s dynamic and non-stationary nature often hinders their efficacy. Traditional models struggle to adapt to the evolving preferences and behaviours inherent in user interaction data, posing a significant challenge for accurate prediction and personalisation. Addressing this, we propose a novel theoretical framework, the Non-stationary Transformer, designed to capture and leverage the temporal dynamics within data effectively. This approach enhances the traditional transformer architecture by introducing mechanisms accounting for non-stationary elements, offering a robust and adaptable solution for recommendation systems. Our experimental analysis, encompassing deep learning and reinforcement learning paradigms, demonstrates the framework’s superiority over benchmark models. The empirical results confirm the efficacy of our proposed framework, which not only provides significant performance enhancements, approximately 8% in Logloss reduction and up to 2% increase in F1 score but also underscores its potential applicability across accumulative reward scenarios. These findings advocate adopting Non-stationary Transformer models to tackle the complexities of today’s recommendation tasks.

We propose Context-aware Feature Transformer Network (CaFTNet), a novel network for human pose estimation. To address the issue of limited modeling of global dependencies in convolutional neural networks, we design Transformerneck to strengthen the expressive power of features. Transformerneck directly substitutes the 3×3 convolution in bottleneck of HRNet with Contextual Transformer (CoT) block, while reducing the complexity of the network. Specifically, CoT first produces keys with static contextual information through 3×3 convolution. Then, relying on the query and contextualization keys, the dynamic contexts are generated through two concatenated 1×1 convolutions. Static and dynamic contexts are eventually fused as an output. Additionally, for the multi-scale networks, in order to further refine the features of the fusion output, we propose an Attention Feature Aggregation Module(AFAM). Technically, given an intermediate input, AFAM successively deduces attention maps along channel and spatial dimensions. Then, Adaptive refinement module(ARM) is exploited to activate the obtained attention maps. Finally, the input undergoes adaptive feature refinement through multiplication with the activated attention maps. Through the above studies, our lightweight network provides a powerful clue for detection of keypoints. Experiments are implemented on the COCO and MPII datasets. The model achieves 76.2 AP on the COCO val2017. Compared to other methods with the CNN as the backbone, CaFTNet reduces the number of parameters by 72.9 %. On the MPII, our method uses only 60.7% of the number of parameters, acquiring semblable results to other methods with the CNN as the backbone.

Due to the influence of background conditions, lighting conditions, occlusion issues and the image resolution, how to extract robust person features is one of the difficulties in ReID research. Vision in Transformers (ViT) has achieved significant results in the field of computer vision. However, the existing problems still limit its application in ReID due to slow extraction of person features and difficulty in utilizing local features of people. To solve the mentioned problems, we utilize Pyramid Vision Transformer (PVT) as the backbone of feature extraction and propose a PVT-based ReID method in conjunction with other studies. Firstly, some improvements suitable for ReID are used on the PVT backbone, and we establish a basic model by using powerful methods verified on CNN-based ReID. Secondly, in an effort to further promote the robustness of the person features extracted by the PVT backbone, two new modules are designed. (1) The local feature clustering (LFC) is recommend to enhance the robustness of person features by calculating the distance between local features and global feature to select the most discrete local features and clustering them. (2) The side information embeddings (SIE) are used to encode non-visual information and send it into the network for training to reduce its impact on person features. Finally, the experiments show that PVTReID has achieved excellent results in ReID datasets and are 20% faster on average than CNN-based ReID methods.

Long unpunctuated texts containing complex linguistic sentences are a stumbling block to processing any low-resource languages. Thus, approaches that attempt to segment lengthy texts with no proper punctuation into simple candidate sentences are a vitally important preprocessing task in many hard-to-solve NLP applications. In this paper, we propose (PDTS) a punctuation detection approach for segmenting Arabic text, built on top of a multilingual BERT-based model and some generic linguistic rules. Furthermore, we showcase how PDTS can be effectively employed as a text tokenizer for unpunctuated documents (i.e., mimicking the transcribed audio-to-text documents). Experimental findings across two evaluation protocols (involving an ablation study and a human-based judgment) demonstrate that PDTS is practically effective in both performance quality and computational cost.

With the development of human-computer interaction, the importance of emotion recognition is increasing. Emotion recognition technology provides practical benefits in various industries such as improving user experience, education, and organizational productivity. In education, emotion recognition can enable real-time understanding of students' emotional states and provide tailored feedback, and in the workplace, by monitoring employees' emotional states, it can improve work performance and job satisfaction. Therefore, multimodal-based research that combines text, speech, and video data for emotion recognition is being conducted in various industries. In this study, we propose an emotion recognition method that combines text and speech data, reflecting the characteristics of the Korean language. For text, KoELECTRA is used to perform embedding, and for speech, MFCC and pitch analysis are used to extract features. Finally, a multimodal transformer model is proposed that combines these two data types to perform emotion recognition. The multimodal transformer model processes text and speech data separately, and then learns the interaction between the two modalities through a Cross-Modal Attention mechanism. Through this, the complementary information from text and speech is effectively combined, improving emotion recognition performance. Experimental results show that the proposed model outperforms single-modality models, achieving a high accuracy of 73.13% and an F1-Score of 0.7344 in emotion classification. This study contributes to the advancement of emotion recognition technology by combining various language and modality data, and higher performance can be expected through the integration of additional modalities in the future.

In a densely populated country like Bangladesh, universities grapple with the challenge of efficiently addressing myriad queries from a large student body, leading to a heightened workload for university stakeholders. To tackle these challenges, we introduce PixieGPT, a tailor-made Generative Pre-Trained Transformer for Bangladeshi universities. PixieGPT significantly mitigates workload by adeptly handling common university-related queries, thereby enhancing user experience. The hierarchical structure plays a crucial role in managing diverse queries from thousands of students about the university system. The solution introduces a modular hierarchical knowledge base (KB) with simpler complexities, addressing the intricacies of efficiently managing large volumes of queries. PixieGPT is designed in a modular way so that the solution is also adaptable for the implementation of other universities worldwide based on the requirements of a particular administrative system. The modular nature facilitates easy adaptation with minor changes based on specific university requirements, ensuring a seamless integration process. This paper delves into the intricacies of PixieGPT's design, emphasizing its pivotal role in mitigating workload challenges for university stakeholders in Bangladesh. The incorporation of BERT for Natural Language Understanding(NLU) and GPT models for Natural Language Generation(NLG) enhances PixieGPT's capabilities, contributing to the scalability and efficiency of the system. The presented use case underscores the practical benefits of PixieGPT, positioning it as a promising solution for universities globally with similar operational frameworks.

The purpose of this study is to ​​explore the use of a custom Transformer model in brain-computer interfaces (BCIs) that translate the neural activity present when an individual with limited verbal and fine-motor skills attempts to handwrite. As found in previous studies, Transformers have performed better than recurrent neural networks (RNNs) in translation tasks which are similar to its usage here in decoding neural signals into intended handwritten text. Due to the known benefits of a Transformer, the hypothesis was that the Transformer would show promise in the context of a BCI through the recorded metrics. The neural signals of a tetraplegic individual, when they attempted to handwrite, were provided by existing research. Four trials were conducted using data with or without augmentation which the model used when training to separately minimize training loss and validation loss. When comparing the results of a BCI with the implementation of a Transformer model with the original RNN BCI (the original data source), the Transformer model performed less favorably across all four trials. Although the results do not indicate that the Transformer model currently outperforms an RNN BCI, it is important to note that further testing of the model's capabilities (such as training it with a larger and more preferable dataset and/or for longer, comparing training times between the RNN and Transformer, and/or seeing how the Transformer is improved with an offline autocorrect feature) is necessary before determining whether Transformers can enhance communication in this manner.

Deep Learning (DL) has made significant advances in computer vision with the advent of Vision Transformers (ViT). Unlike Convolutional Neural Networks (CNNs), ViTs use self-attention to extract both local and global features from image data, and then use residual connections to feed these features directly into a fully networked multilayer perceptron head. In hospitals, hematologists prepare peripheral blood smears (PBSs) and read them under a medical microscope to detect abnormalities in blood counts such as leukemia. However, this task is time-consuming and prone to human error. This study investigates the transfer learning process of Google ViT and ImageNet CNNs to automate the reading of PBSs. The study used two online PBS datasets, PBC and BCCD, and transferred them into balanced datasets to investigate the influence of data amount and noise immunity on both neural networks. The PBC results show that Google ViT is an excellent DL neural solution for data scarcity. The BCCD results show that Google ViT is superior to ImageNet CNNs in dealing with unclean, noisy image data because it is able to extract both global and local features and use residual connections, despite the additional time and computational overhead.

Safe, high-quality and economical electric energy transportation is the basic requirements of modern power system operation. Transformer, as one of the core equipment of power system and national grid system, has the characteristics of diverse types, variable models and wide deployment. It is the basic equipment for power system to realize voltage change and electric energy distribution. Because the power system transformer needs to operate with load for a long time, the probability of failure is usually higher than that of other power equipment in general. This paper starts from the transformer fault and fault diagnosis. Firstly, the transformer fault types, traditional transformer fault diagnosis techniques and the advantages of machine learning in transformer fault diagnosis are reviewed. Secondly, the application of information system theory and information entropy in transformer fault diagnosis is introduced. Then it introduces machine learning technology, feature selection and extraction technology in transformer fault diagnosis, and machine learning algorithms such as support vector machine and extreme learning machine for transformer fault diagnosis. Finally, the potential development trend of information system theory, information entropy and machine learning in transformer fault diagnosis is forecasted. The combination of transformer fault prediction and machine learning algorithm is helpful for power system maintenance personnel to accurately predict the running state of power equipment, and also provides a new method and technology for the safe and reliable operation and regular maintenance of power system.

A recently published article posited the use of polymer matrix composites (PMCs) as a replacement material for low-carbon steel (LCS) in large power transformer (LPT) tanks [1]. PMCs offer potential advantages such as weight reduction, corrosion prevention, and self-healing of the tank material which may not be achievable with traditional LCS. LCS can arrest ballistic projectiles at relatively low velocities, but at higher velocities, penetration can occur leading to a potential major transformer failure. Numerical simulations of steel projectile impacts upon LCS plates and glass and/or carbon fiber polymer composite laminates showed that with sufficient thickness both could prevent projectile penetration. This prevention was especially efficient when the plates were coated with a high strain rate sensitive material such as polyurea (PU). Our simulations show that the addition of a 3 mm coating of PU to a standard 10 mm thick LCS plate used in LPT tanks could prevent penetration of a 400 m/s steel projectile. For PMC plates, the simulations with the same projectile velocity showed three times faster deceleration with a 3 mm PU coating than without the coating. Thus, two realistic approaches to improving the ballistic protection of LPTs are presented. In addition to their other advantages, this research clearly shows that PMCs with the coating could provide significant ballistic protection for the next generation of LPT tanks.

In the realm of the five-category classification endeavor, there has been limited exploration into applied techniques for classifying Arabic text. These methods have primarily leaned on singletask learning, incorporating manually crafted features that lack robust sentence representations. Recently, the Transformer paradigm has emerged as a highly promising alternative. However, when these models are trained using single-task learning, they often face challenges in achieving outstanding performance and generating robust latent feature representations, especially when dealing with small datasets. This issue is particularly pronounced in the context of the Arabic dialect, which has a scarcity of available resources. Given these constraints, this study introduces an innovative approach to dissecting sentiment in Arabic text. This approach combines Inductive Transfer (INT) with the Transformer paradigm to augment the adaptability of the model and refine the representation of sentences. By employing self-attention SE-A and feed-forward sub-layers as a shared Transformer encoder for both the five-category and three-category Arabic text classification tasks, this proposed model adeptly discerns sentiment in Arabic dialect sentences. The empirical findings underscore the commendable performance of the proposed model, as demonstrated in assessments of the Hotel Arabic-Reviews Dataset, the Book Reviews Arabic Dataset, and the LARB dataset.

Although wireless capsule endoscopy (WCE) detects small bowel diseases effectively, it has some limitations. For example, the reading process can be time-consuming due to the numerous images generated per case, and lesion detection accuracy may rely on the operators' skills and experiences. Hence, many researchers have recently developed deep learning-based methods to address these limitations. However, they tend to select only a portion of the images from a given WCE video and analyze each image individually. In this study, we note that more information can be extracted from the unused frames and temporal relations of sequential frames. Specifically, to increase the accuracy of lesion detection without depending on experts' frame selection skills, we suggest using whole video frames as the input to the deep-learning system. Thus, we propose a new Transformer-based neural encoder that takes the entire video as the input, exploiting the power of the Transformer to extract long-term global correlation within and between the input frames. Subsequently, we can capture the temporal context of the input frames and the attentional features within a frame. Tests on benchmark datasets of four WCE videos showed 95.1% sensitivity and 83.4% sensitivity. These results may significantly advance automated lesion detection techniques for WCE images. Our code is available at https://github.com/syupoh/VWCE-Net.git.

We introduce a novel hybrid Transformer-CNN architecture for robotic grasp detection, designed to enhance the accuracy of grasping unknown objects. Our proposed architecture has two key designs. Firstly, we develop a hierarchical transformer as the encoder, incorporating the external attention to effectively capture the correlation features across the data. Secondly, the decoder is constructed with cross-layer connections to efficiently fuse multi-scale features. Channel attention is introduced in the decoder to model the correlation between channels and to adaptively recalibrate the channel correlation feature response, thereby increasing the weight of the effective channels. Our method is evaluated on the Cornell and Jacquard public datasets, achieving an image-wise detection accuracy of 98.3% and 95.8% on each dataset, respectively. Additionally, we achieve object-wise detection accuracy of 96.9% and 92.4% on the same datasets. A physical experiment is also performed using the Elite 6Dof robot, with a grasping accuracy rate of 93.3%, demonstrating the proposed method's ability to grasp unknown objects in real-world scenarios. The results of this study show that our proposed method outperforms other state-of-the-art methods.

Incorporating source-side linguistic knowledge into the neural machine translation (NMT) model has recently achieved impressive performance on machine translation tasks. One popular method is to generalize the word embedding layer of the encoder to encode each word and its linguistic features. The other method is to change the architecture of the encoder to encode syntactic information. However, the former cannot explicitly balance the contribution from the word and its linguistic features. The latter cannot flexibly utilize various types of linguistic information. Focusing on the above issues, this paper proposes a novel NMT approach that models the words in parallel to the linguistic knowledge by using two separate encoders. Compared with the single encoder based NMT model, the proposed approach additionally employs the knowledge-based encoder to specially encode linguistic features. Moreover, it shares parameters across encoders to enhance the model representation ability of the source-side language. Extensive experiments show that the approach achieves significant improvements of up to 2.4 and 1.1 BLEU points on Turkish→English and English→Turkish machine translation tasks, respectively, which indicates that it is capable of better utilizing the external linguistic knowledge and effective improving the machine translation quality.

To enhance the overall reliability of the power system, engineers have redirected their focus toward health monitoring and early detection of faults in transformers. Among these faults, transformer core defects demand particular attention. While fault simulation using software has traditionally been the preferred approach, these methods suffer from data inaccuracies in real-world conditions. Consequently, conducting actual experimental setups with induced faults is imperative to investigate core issues. This study uses Hilbert Transform (HT) as a signal processing technique to extract crucial data characteristics, thereby enhancing the performance of the classifier model. The research involves analyzing electric current signals from a single-phase 1kVA transformer. A comparative assessment of our proposed model was conducted using raw data and Fast Fourier Transform (FFT), evaluating accuracy, precision, recall, F1-score, and computational time. The results demonstrate a significant improvement in all metrics for the classifier models, particularly the k-nearest neighbor (KNN) algorithm, which exhibited values of 83.89%, 84.39%, 83.89%, 83.79%, and 0.0156 seconds, respectively. Future work aims to extend this analysis under different loading conditions.

The question of molecular similarity is core in cheminformatics, and is usually assessed via a pairwise comparison based on vectors of properties or molecular fingerprints. We recently exploited variational autoencoders to embed 6M molecules in a chemical space, such that their (Euclidean) distance within the latent space so formed could be assessed within the framework of the entire molecular set. However, the standard objective function used did not seek to manipulate the latent space so as to cluster the molecules based on any perceived similarity. Using a set of some 160,000 molecules of biological relevance, we here bring together three modern elements of deep learning to create a novel and disentangled latent space, viz transformers, contrastive learning, and an embedded autoencoder. The effective dimensionality of the latent space was varied such that clear separation of individual types of molecules could be observed within individual dimensions of the latent space. The capacity of the network was such that many dimensions were not populated at all. As before, we assessed the utility of the representation by comparing clozapine with its near neighbours, and did the same for various antibiotics related to flucloxacillin. Transformers, especially when as here coupled with contrastive learning, effectively provide one-shot learning, and lead to a successful and disentangled representation of molecular latent spaces that at once uses the entire training set in their construction while allowing ‘similar’ molecules to cluster together in an effective and interpretable way.

In Brain-Computer Interface Motor Imagery (BCI-MI) systems, Convolutional Neural Networks (CNNs) have traditionally dominated as the deep learning method of choice, demonstrating significant advancements in state-of-the-art studies. Recently, Transformer models with attention mechanisms have emerged as a sophisticated technique, enhancing the capture of long-term dependencies and intricate feature relationships in BCI-MI. This research investigates the performance of EEG-TCNet and EEG-Conformer models, which are trained and validated using various hyperparameters and bandpass filters during preprocessing to assess improvements in model accuracy. Additionally, this study introduces the EEG-TCNTransformer, a novel model that integrates the convolutional architecture of EEG-TCNet with a series of self-attention blocks employing a multi-head structure. The EEG-TCNTransformer achieves an accuracy of 82.97% without the application of bandpass filtering. The source code of EEG-TCNTransformer is available on GitHub.

Accurate estimation of Remaining Useful Life (RUL) for aircraft engines is essential for ensuring safety and uninterrupted operations in the aviation industry. Numerous investigations have leveraged the success of attention-based Transformer architecture in sequence modeling tasks, particularly in its application to RUL prediction. These studies primarily focus on utilizing onboard sensor readings as input predictors. While various Transformer-based approaches have demonstrated improvement in RUL predictions, their exclusive focus on temporal attention within multivariate time series sensor readings, without considering sensor-wise attention, raises concerns about potential inaccuracies in RUL predictions. To address this concern, our paper proposes a novel solution in the form of a two-stage attention based hierarchical transformer (STAR) frame-work. This approach incorporates a two-stage attention mechanism, systematically addressing both temporal and sensor-wise attentions. Furthermore, we enhance the STAR RUL prediction framework by integrate hierarchical encoder-decoder structures to capture valuable information across different time scales. By conducting extensive numerical experiments with the CMAPSS datasets, we demonstrate that our proposed STAR framework significantly outperforms current state-of-the-art models for RUL prediction.

Social media platforms are increasingly enabling the propagation of content from groups related to drug use, thus posing risks for the wider population and, in particular, individuals who are amenable to drug use and drug addiction. The detection of drug use content on social media platforms is a priority for governments, technology companies, and drug law enforcement organizations. To counter this issue, various techniques have been developed to identify and promptly remove drug use content, while also blocking its creators from network access. In this paper, we introduce a manually annotated Twitter dataset, comprising 156,521 tweets published between 2008 and 2022, specifically compiled for the purpose of drug use detection. The dataset underwent annotation by several group of expert annotators who classified the tweets as either drug use or non-drug use. Exploratory data analysis was conducted to comprehend the dataset's characteristics. Various classification algorithms, including SVM, XGBoost, RF, NB, LSTM, and BERT were employed using the dataset. Among the traditional machine learning models, SVM utilizing term frequency-inverse document frequency features achieved the highest F1-Score (0.9017). However, BERT with textual features concatenated with numerical and categorical features in ensemble method surpassed the performance of traditional models, attaining F1-Score of 0.9112. To facilitate future research and enhance English online drug use classification accuracy, the dataset will be made publicly available.

The operation of transformers suffers from natural or man-made disasters, subjecting the equipment to prolonged exposure to temperature, lightning strikes, transient over-voltages, and over-current, gradually producing insulation aging phenomenon, which results in partial discharges within the transformer. Without dealing with it beforehand properly, it may lead to insulation breakdown. Nowadays, with the mature development of microchip control technology and wireless communication technology, which is helpful for realizing the research for online real-time and long-term monitoring capabilities for power equipment. A transformer operating condition monitoring and diagnostic system that can rapidly and effectively prevent the breakdown of transformer insulation, which can impact the overall power supply system is developed in this paper. The system is divided into two parts based on its function. The first part is the operating condition monitoring and real-time alert unit for the transformer, primarily composed of an embedded computing module, a sensor module, and a transmission module. Its work content is to capture real-time online partial discharge ultrasonic signals and high frequency current comparator pulse signals, and analyze and determine the condition of the internal insulation and operational status whether it is fault or not. The second part is the central control unit, consisting mainly of an industrial computer and a transmission module. Its work content is to store the ultrasonic and high frequency current comparator pulse signals from the transformer operation monitoring and real-time alert unit. It uses the software to eliminate signal noise and interference, and then utilizes artificial neural networks to identify the type of fault. In this paper, the research for the phenomenon of insulation degradation in transformer equipment aims to establish aging trend profiles through the dual malfunctioning alert and aging process analysis, as well as the long-term measurement of ultrasonic signals from partial discharges and high-frequency current comparator pulse signals. This approach provides crucial reference data and a foundation for disposable on-site measurement, assessing the phenomenon of insulation aging stages, defects and damage of the transformers effectively. It allows for timely replacement or maintenance in advance, achieving the prevention of malfunctioning and predictive diagnosis of aging.

Machinery degradation assessment can offer meaningful prognosis and health management in-formation. Although numerous machine prediction models based on artificial intelligence have emerged in recent years, they still face a series of challenges: (1) Many models continue to rely on manual feature extraction. (2) Deep learning models still struggle with long sequence prediction tasks. (3) Health indicators are inefficient for remaining useful life (RUL) prediction with cross-operational environments when dealing with high-dimensional datasets as inputs. This research proposes a health indicator construction methodology based on a transformer self-attention transfer network (TSTN). This methodology can directly deal with the high-dimensional raw dataset and keep all the information without missing when the signals are taken as the input of the diagnosis and prognosis model. First, we design an encoder with a long-term and short-term self-attention mechanism to capture crucial time-varying information from a high-dimensional dataset. Second, we propose an estimator that can map the embedding from the encoder output to the estimated degradation trends. Then, we present a domain dis-criminator to extract invariant features from different machine operating conditions. The case studies with the FEMTO-ST bearing dataset and the Monte Carlo method for RUL prediction during the degradation process are conducted. The experiment results fully exhibited the signif-icant advantages of the proposed method compared to other state-of-the-art techniques.

In recent times, BERT based transformer models have become an inseparable part of the 'tech stack' of text processing models. Similar progress is being observed in the speech domain with a multitude of models observing state-of-the-art results by using audio transformer models to encode speech. This begs the question of what are these audio transformer models learning. Moreover, although the standard methodology is to choose the last layer embedding for any downstream task, but is it the optimal choice? We try to answer these questions for the two recent audio transformer models, Mockingjay and wave2vec2.0. We compare them on a comprehensive set of language delivery and structure features including audio, fluency and pronunciation features. Additionally, we probe the audio models' understanding of textual surface, syntax, and semantic features and compare them to BERT. We do this over exhaustive settings for native, non-native, synthetic, read and spontaneous speech datasets

This study leverages transformer-based models and focuses on data representation strategies in the named entity recognition task, including "single" (one sentence per vector), "merged" (multiple sentences per vector), and "context" (sentences joined with attention to context). Performance analysis reveals that models trained with a single strategy may not perform well on different data representations. A combined training procedure is proposed to address this limitation, using all three strategies to enhance the stability and adaptability of the model. The results of this approach are presented and discussed for various datasets for four languages (English, Polish, Czech, and German), demonstrating the effectiveness of the combined strategy.

Buildings are responsible for approximately 40% of the world’s energy consumption and 36% of the total carbon dioxide emissions. Building occupancy is essential, enabling Occupant-Centric Control for zero emissions and decarbonization. Although existing machine learning and deep learning methods for building occupancy prediction have achieved remarked progress, their analyses remain limited when applied to complex real-world scenarios. Besides, there is a high expectation for Transformer algorithms to predict building occupancy accurately. Therefore, this paper presents an Occupancy Prediction Transformer network (OPTnet). We fuse and feed multi-sensor data (building occupancy, indoor environmental conditions, HVAC operations) into a Transformer model to forecast the future occupancy presence in multiple zones. We perform experimental analysis and compare it to different occupancy prediction methods (e.g., Decision Tree, Long Short-Term Memory networks, Multi-layer perceptron) and diverse time horizons (1,2,3,5,10,20,30mins). The performance metrics (e.g., Accuracy and Mean Squared Error) are employed to evaluate the effectiveness of prediction algorithms. Our OPTnet method achieves superior performance on our experimental two-week data compared to existing methods. The improved performance signifies its potential to enhance HVAC control systems and energy optimization strategies. We will make the code publicly available at https://github.com/kailaisun/occupancy-prediction-binary to promote transparency and reproducibility.

We demonstrate an RF photonic fractional Hilbert transformer based on an integrated Kerr micro-comb source featuring a record low free spectral range of 49 GHz. By programming and shaping the comb lines according to calculated tap weights for up to 39 wavelengths across the C-band, we achieve tunable bandwidths ranging from 1.2 to 15.3 GHz as well as variable center frequencies from baseband to 9.5 GHz, for both standard integral and arbitrary fractional orders. We experimentally characterize the RF amplitude and phase response of the tunable bandpass and lowpass Hilbert transformers with 90 and 45-degree phase shifts. The experimental results show good agreement with theory, confirming the effectiveness of our approach as a powerful way to implement standard and fractional order Hilbert transformers with broad and variable bandwidths and center frequencies, with high reconfigurability and greatly reduced size and complexity. Tan, and D. J. Moss are with the Optical Sciences Centre, Swinburne University of Technology, Hawthorn, VIC 3122, Australia. (Corresponding e-mail: [email protected]). Xu is with the Electro-Photonics Laboratory, Department of Electrical and Computer System Engineering, Monash University, Clayton, 3800 VIC, Australia

Oil immersed paper insulation condition is a crucial aspect of power transformer’s life condition diagnostic. The measurement testing database collected over the years made it possible for researchers to implement classification analysis to in-service power transformer. This article presents classification analysis of transformer oil-immersed paper insulation condition. The measurements data (dielectric characteristics, dissolved gas analysis, and furanic compounds) of 149 transformers with primary voltage of 150 kV had been gathered and analyzed. The algorithm used for developing classification model is Support Vector Machine (SVM). The model has been trained and tested using different datasets. Different models have been created and the best chosen, resulting in 90.63% accuracy in predicting the oil-immersed paper insulation condition. Further implementation was executed to classify oil-paper condition of 19 Transformers which Furan data is not available. The classification results combined, reviewed, and compared to conventional assessment methods and standards, confirming that the model developed has the ability to do classification of current oil-paper condition based on Dissolved Gasses and Dielectric Characteristics.

Utilizing deep learning for semantic segmentation of cropland from remote sensing imagery has become a crucial technique in land surveys. Cropland illustrates diverse morphologies and degrees of fragmentation on the Earth’s surface, underscoring the importance of accurately perceiving the complex boundaries of cropland which are crucial for effective segmentation. This paper introduces a UNet-like boundary-aware compensation model BAFormer. Cropland boundaries typically exhibit rapid transformations in pixel values and texture features, often appearing as high-frequency features in remote-sensing images. To enhance the recognition of these high-frequency features as represented by cropland boundaries, the proposed BAFormer integrates a Feature Adaptive Mixer (FAM) and develops a Deep Wide Large Kernel Multi-Layer Perceptron (DWLK-MLP) to enrich the global and local cropland boundaries features separately. Specifically, FAM adaptively mixes high-frequency and low-frequency features through the advantages of convolution and self-attention; DWLK-MLP expands the convolutional receptive field by deeply decomposing large kernel convolutions. The efficacy of BAFormer has been evaluated on the Vaihingen, Potsdam, and LoveDA public datasets, as well as the Mapcup dataset. It has demonstrated advanced performance, achieving mIoU scores of 84.5%, 87.3%, 53.5%, and 83.1% on these datasets respectively. Notably, BAFormer-T, the lightweight iteration of the model, surpasses other lightweight models on the Vaihingen dataset with scores of 91.3% F1 and 84.1% mIoU. The source code is available at https://github.com/WangYouM1999/BAFormer.

In recent times, models like the Transformer have showcased remarkable prowess in tasks related to natural language processing. However, these models tend to be excessively intricate and demand extensive training. Additionally, while the multi-head self-attention mechanism in the Transformer model aims to capture semantic connections between words in a sequence, it encounters limitations when handling short sequences, thereby limiting its effectiveness in 5-polarity Arabic sentiment analysis tasks. The switch-transformer model has recently emerged as a high-performing alternative. Nevertheless, when these models are trained using single-task learning, they often fall short of achieving exceptional performance and struggle to generate robust latent feature representations, especially when working with compact datasets. This challenge is particularly pronounced in the case of the Arabic dialect, which is considered a low-resource language. Given these constraints, this research introduces a novel approach to sentiment analysis in Arabic text. This method leverages multitask learning in tandem with the switch-transformer shared encoder to enhance model adaptability and refine sentence representation. By introducing a mixture of expert (MoE) mechanism that break down the problem into smaller, more manageable sub-problems, the model becomes adept at handling lengthy sequences and intricate input-output relationships, benefiting both five-point and three-polarity Arabic sentiment analysis tasks. This proposed model effectively discerns sentiment in Arabic dialect sentences. The empirical results highlight the outstanding performance of the suggested model, as evidenced in evaluations on the Hotel Arabic-Reviews Dataset, the Book Reviews Arabic Dataset, and the LARB dataset.

Smartphones are prone to SMS phishing due to the rapid growth in the availability of smart mobile technologies driven by Internet connections. Also, detecting phishing SMS is a challenging task due to the unstructured nature of SMS text data with non-linear complex correlations. In this concern, considering the recent advancements in the domain of cybersecurity, we have proposed a hybrid deep learning framework that extracts robust features from SMS texts followed by an automatic detection of Phishing SMS. Due to combining the potential capability of individual models into one hybrid framework, it has outperformed various other individual machine learning and deep learning models. The proposed Phishing Detection framework is an effective hybrid combination of pretrained transformer model, MPNet (Masked and Permuted Language Modeling), with supervised ConvNets (CNN) and Bi-directional Gated Recurrent Units (GRU). It is intended to successfully detect unstructured short phishing text messages that contain complex patterns.

This paper presents a symmetrical topology for the design of solid-state transformer, made up of power switching converters, to replace conventional bulky transformers. The proposed circuitry not only reduces the overall size but also provides power flow control with the ability to be interfaced with renewable energy resources (RESs) to fulfill the future grid requirements at consumer end. Solid state transformer provides bidirectional power flow with variable voltage and frequency operation and has the ability to maintain unity power factor, and current total harmonic distortion (THD) for any type of load within defined limits of IEEE standard. Solid State Transformer offers much smaller size as compared to that of the conventional iron core transformer. MATLAB/Simulink platform is adopted to test the validity of the proposed circuit for different scenarios by providing the simulation results evaluated at 25 kHz switching frequency.

A high step-up DC-to-DC converter that integrates an isolated transformer and a switched-clamp capacitor is presented in this study. The voltage stress of the main power switch should be clamped to 1/4 V by using the turn ratio and switched-clamp capacitor of an isolated transformer to achieve a high voltage gain. In addition, a passive clamp circuit is employed reduce voltage stress on the main power switch. The energy of the leakage inductor can be recycled by the clamp capacitor because of the passive clamp circuit, thereby improving the power converter efficiency. The converter consists of one isolated transformer, one main switch, three capacitors, and four diodes. Operating principle and steady-state analyses are also discussed. Finally, a 24-V-input voltage to 200-V-output voltage and a 150 W output power prototype converter are fabricated in the laboratory. The maximum efficiency of the converter is 95.1 at 60 W.

In the leading/trailing edge’s adaptive machining of the near-net-shaped blade, a small portion of the theoretical part is retained for securing aerodynamic performance by manual work. However, this procedure is time-consuming and depends on the human experience. In this paper, we defined retained theoretical leading/trailing edge as the reconstruction area. To accelerate the reconstruction process, an anchor-free neural network model based on Transformer was proposed, named LETR (Leading/trailing Edge Transformer). LETR extracts image features from an aspect of mixed frequency and channel domain. We also integrated LETR with the newest meta-Acon activation function. We tested our model on the self-made dataset LDEG2021 on a single GPU and got an mAP of 91.9\%, which surpassed our baseline model, Deformable DETR by 1.1\%. Furthermore, we modified LETR’s convolution layer and named the new model after GLETR (Ghost Leading/trailing Edge Transformer) as a lightweight model for real-time detection. It is proved that GLETR has fewer weight parameters and converges faster than LETR with an acceptable decrease in mAP (0.1\%) by test results.

The ‘inverse problem’ of mass spectrometric molecular identification (‘given a mass spectrum, calculate/predict the 2D structure of the molecule whence it came’) is largely unsolved, and is especially acute in metabolomics where many small molecules remain unidentified. This is largely because the number of experimentally available electrospray mass spectra of small molecules is quite limited. However, the forward problem (‘calculate a small molecule’s likely fragmentation and hence at least some of its mass spectrum from its structure alone’) is much more tractable, because the strengths of different chemical bonds are roughly known. This kind of molecular identification problem may be cast as a language translation problem in which the source language is a list of high-resolution mass spectral peaks and the ‘translation’ a representation (for instance in SMILES) of the molecule. It is thus suitable for attack using the deep neural networks known as transformers. We here present MassGenie, a method that uses a transformer-based deep neural network, trained on ~6 million chemical structures with augmented SMILES encoding and their paired molecular fragments as generated in silico, explicitly including the protonated molecular ion. This architecture (containing some 400 million elements) is used to predict the structure of a molecule from the various fragments that may be expected to be observed when some of its bonds are broken. Despite being given essentially no detailed nor explicit rules about molecular fragmentation methods, isotope patterns, rearrangements, neutral losses, and the like, MassGenie learns the effective properties of the mass spectral fragment and valency space, and can generate candidate molecular structures that are very close or identical to those of the ‘true’ molecules. We also use VAE-Sim, a previously published variational autoencoder, to generate candidate molecules that are ‘similar’ to the top hit. In addition to using the ‘top hits’ directly, we can produce a rank order of these by ‘round-tripping’ candidate molecules and comparing them with the true molecules, where known. As a proof of principle, we confine ourselves to positive electrospray mass spectra from molecules with a molecular mass of 500Da or lower, including those in the last CASMI challenge (for which the results are known), getting 49/93 (53%) precisely correct. The transformer method, applied here for the first time to mass spectral interpretation, works extremely effectively both for mass spectra generated in silico and on experimentally obtained mass spectra from pure compounds. It seems to act as a Las Vegas algorithm, in that it either gives the correct answer or simply states that it cannot find one. The ability to create and to ‘learn’ millions of fragmentation patterns in silico, and therefrom generate candidate structures (that do not have to be in existing libraries) directly, thus opens up entirely the field of de novo small molecule structure prediction from experimental mass spectra.

Smart Internet of Vehicles (IoVs) combined with Artificial Intelligence (AI) will contribute to vehicle decision-making in the Intelligent Transportation System (ITS). Multi-Vehicle Pursuit games (MVP), a multi-vehicle cooperative ability to capture mobile targets, is becoming a hot research topic gradually. Although there are some achievements in the field of MVP in the open space environment, the urban area brings complicated road structures and restricted moving spaces as challenges to the resolution of MVP games. We define an Observation-constrained MVP (OMVP) problem in this paper and propose a Transformer-based Time and Team Reinforcement Learning scheme (T3OMVP) to address the problem. First, a new multi-vehicle pursuit model is constructed based on decentralized partially observed Markov decision processes (Dec-POMDP) to instantiate this problem. Second, by introducing and modifying the transformer-based observation sequence, QMIX is redefined to adapt to the complicated road structure, restricted moving spaces and constrained observations, so as to control vehicles to pursue the target combining the vehicle’s observations. Third, a multi-intersection urban environment is built to verify the proposed scheme. Extensive experimental results demonstrate that the proposed T3OMVP scheme achieves significant improvements relative to state-of-the-art QMIX approaches by 9.66%~106.25%. Code is available at https://github.com/pipihaiziguai/T3OMVP.

Fine-grained image classification remains an ongoing challenge in the computer vision field, which is particularly intended to identify objects within sub-categories. It is a difficult task since there is a minimal and substantial intra-class variance. The current methods address the issue by first locating selective regions with Region Proposal Networks (RPN), object localization, or part localization, followed by implementing a CNN Network or SVM classifier to those selective regions. This approach, however, makes the process simple by implementing a single-stage end-to-end feature encoding with a localization method, which leads to improved feature representations of individual tokens/regions by integrating the transformer encoder blocks into the Yolov5 backbone structure. These Transformer Encoder Blocks, with their self-attention mechanism, effectively captured the global dependencies and enabled the model to learn relationships between distant regions. This improved the model ability to understand context and captured long-range spatial relationships in the image. We also replaced the Yolov5 detection heads with three transformer heads at the output for object recognition using the discriminative and informative features maps from transformer encoder blocks. We established the potential of the single stage detector for the fine-grained image recognition task, by achieving state of the art 93.4% accuracy, as well as outperforming the existing Yolov5 model. The effectiveness of our approach is assessed using the Stanford car dataset, which includes 16,185 images of 196 different classes of vehicles with significantly identical visual appearances.

In the realm of power delivery to end-users, transformers are indispensable, with their malfunctions leading to substantial economic, safety, and environmental repercussions. The need for persistent surveillance is accentuated for in-situ oil-filled transformers, given the potential degradation of oil and the emergence of related ageing by-products. As the focus tilts towards online detection methodologies for transformer oil ageing, bypassing challenges associated with traditional offline methods such as sample contamination and misinterpretation, fiber optic sensors are gaining trac-tion due to their compact nature, cost-effectiveness, and resilience to electromagnetic disturbances typical in high-voltage environments. This work delves into the sensitivity analysis of intensi-ty-modulated plastic optical fiber sensors. The investigation encompasses key determinants such as the influence of optical source wavelengths, noise response dynamics, ramifications of varying sensing lengths, and repeatability assessments. Findings underscore that elongating the sensing length detrimentally affects both the linearity response and repeatability, largely attributed to the diminished resistance to external noise. Additionally, the choice of the optical source wavelength proved to be a critical variable in assessing sensor sensitivity.

Plasma generation by means of electrical discharge requires specialized power supply systems. The applicability of plasma for various plasma processes depends on its parameters, and these in turn depend on the parameters of the power supply systems. Arc plasma can be unstable, generating a lot of electromagnetic interference and overvoltage and overcurrent. The power system of a plasma reactor must guarantee good plasma control characteristics, be immune to disturbances and ensure good cooperation with the power grid. The article analyzes the cooperation of a three-phase plasma reactor, with gliding arc discharge, with a power supply system of a new type. This system integrates an AC/DC/AC converter with a five-column transformer of special design in a single device. Using the properties of magnetic circuits, it was possible to integrate the functions of ignition and sustaining the burning of the discharge in the reactor in a single transformer. Proper design of the transformer is crucial to achieve good cooperation of the AC/DC/AC converter with both the plasma reactor and the power supply network. The presented power supply design shows a number of positive features predisposing it to powering arc plasma reactors.

To realize high response and selectivity gas sensor in detecting dissolved gases in transformer oil, in this study, the adsorption of four kinds of gases (H2, CO, C2H2 and CH4) on Pd-graphyne as well as the gas sensing properties evaluation were investigated. The energetically favorable structure of Pd doped γ-Graphyne was first studied, including the comparison of different adsorption sites and discussion of electronic properties. Then, the adsorption of these four molecules on Pd-graphyne was explored. The adsorption structure, adsorption energy, electron transfer and electron distribution, the band structure and density of states were calculated and analyzed. The results show that the Pd atom prefers to be adsorbed on the middle of three C≡C bonds and the band gap is smaller. The CO adsorption exhibits the largest adsorption energy and electron transfer and brings obvious change to the structure and electron properties to Pd-graphyne. Because of the conductance decrease after adsorption CO and acceptable recovery time at high temperature, the Pd-graphyne can be promising gas sensing materials to detect CO with high selectivity. This work offers theoretical support for the design of nanomaterials based gas sensor using novel structure for industrial application.

U-Net, based on a deep convolutional neural network (CNN), has been clinically used to au-to-segment normal organs and potentially target volumes. However, CNNs with local geometric dependencies may limit the accuracy of segmentation. Additionally, the performance of CNNs can vary depending on the selection of network hyper-parameters, which was mitigated by the proposition of nnU-Net. We chose a vision transformer architecture called VT U-Net, which features a self-attention excluding the convolution layer, to overcome the limitations of CNNs by utilizing global geometric information of images. The VT U-Net v.2 became more powerful thanks to the adaptive hyper-parameter optimizer embedded in nnU-Net. However, despite leveraging the benefits of nnU-Net, VT U-Net v.2 still had additional network hyper-parameters that needed to be optimally chosen. Accordingly, among various hyper-parameters, this study attempted to find the optimal combination of the patch size and the embedded dimension regarding the transformer. From the 4-fold cross-validation, the modified VT U-Net v.2 showed the highest average performance for planning target volume (PTV) segmentation among the investigated networks. Though nnU-Net was based on convolution layers, the adaptive hyper-parameter optimizers turned out to enhance the performance. It was also confirmed that network hyper-parameters affected the segmentation accuracy of vision transformers.

Because imbalanced power will cause the loss of the propulsion motor of the railway vehicle, and the increase in temperature will shorten the service life of the electric vehicle. Not only this, but also increase the cost of electricity and maintenance. In the past, the industry only focused on methods to improve power quality such as load capacity, relay setting, and harmonic resolution. Now, the consider of three-phase unbalance rate (TPUR) must be applied. I propose special transformers wiring (STW) to improve the three unbalance rates and provide different transformer wiring methods. According to the IEEE Committee, in the future, power companies will need to install balanced relay stations to improve three-phase unbalance rate. the internal regulations of Taipower must be less than 4.5% (voltage unbalance rate (NPSUR)of 2.5% and motor temperature rise of 12.5%). the derivation of the transformer "three-phase unbalance rate" model is the focus of the railway system. This research is based on the model derivation of different wiring methods to improve the hot problem caused by the three-phase imbalance and improve the service life of the train. And pointed out that Scott, Le-Blanc, Modified-Woodbridge three wiring methods can be applied to future railway system routes to improve the three-phase unbalance rate, in line with the IEEE standard of less than 2%.

This paper develops a novel dynamic correction method for the reliability assessment of large oil-immersed power transformers. First, with the transformer oil-paper insulation system (TOPIS) as the target of evaluation and the winding hot spot temperature (HST) as the core point, an HST-based static ageing failure model is built according to the Weibull distribution and Arrhenius reaction law, in order to describe the transformer ageing process and calculate the winding HST for obtaining the failure rate and life expectancy of TOPIS. A grey target theory based dynamic correction model is then developed, combined with the data of Dissolved Gas Analysis (DGA) in power transformer oil, in order to dynamically modify the life expectancy calculated by the built static model, such that the corresponding relationship between the state grade and life expectancy correction coefficient of TOPIS can be built. Furthermore, the life expectancy loss recovery factor is introduced to correct the life expectancy of TOPIS again. Lastly, a practical case study of an operating transformer has been undertaken, in which the failure rate curve after introducing dynamic corrections can be obtained for the reliability assessment of this transformer. The curve shows a better ability of tracking the actual reliability level of transformer, thus verifying the validity of the proposed method and providing a new way for transformer reliability assessment. This contribution presents a novel model for the reliability assessment of TOPIS, in which the DGA data, as a source of information for the dynamic correction, is processed based on the grey target theory, thus the internal faults of power transformer can be diagnosed accurately as well as its life expectancy updated in time, ensuring that the dynamic assessment values can commendably track and reflect the actual operation state of the power transformers.

The fusion of global contextual information with local cropped block details is crucial for segmenting ultra-high resolution images. In this study, we introduce a novel fusion mechanism termed Global-Local Deep Fusion (GL-Deep Fusion) based on an enhanced transformer architecture, which efficiently integrates global contextual information and local details. Specifically, we propose the Global-Local Synthesis Networks (GLSNet), a dual-branch network where one branch processes the entire original image, while the other branch handles cropped local patches as input. The feature fusion of different branches in GLSNet is achieved through GL-Deep Fusion, significantly enhancing the accuracy of ultra-high resolution image segmentation. Particularly effective is GLSNet in identifying tiny overlapping items. To optimize GPU memory utilization, we meticulously design a dual-branch architecture that proficiently leverages the features it extracts, seamlessly integrating them into the enhanced transformer framework of GL-Deep Fusion. Extensive experiments conducted on challenging benchmarks, including DeepGlobe and Vaihingen datasets, demonstrate that GLSNet achieves a new state-of-the-art performance in terms of GPU memory utilization and segmentation accuracy tradeoff.

Speech emotion recognition (SER) is a technology that can be applied in distance education to analyze speech patterns and evaluate speakers’ emotional states in real-time. It provides valuable insights and can be used to enhance the learning experience by enabling the assessment of instructors’ emotional stability, a factor that significantly impacts information delivery effectiveness. Students demonstrate different engagement levels during learning activities, and assessing this engagement is an important aspect of controlling the learning process and improving e-learning systems. An important aspect that may influence student engagement is the emotional states of their instructors. Accordingly, this research uses deep learning techniques to create an automated system for recognizing instructors’ emotions in their speech when delivering distance learning. This methodology entails integrating Transformer, convolutional neural network, and long short-term memory architectures into an ensemble to enhance SER. Feature extraction from audio data used Mel-frequency cepstral coefficients, chroma, Mel spectrogram, zero crossing rate, spectral contrast, centroid, bandwidth, roll-off, and root-mean square, with subsequent optimization processes adding noise to, conducting time stretching, and shifting the audio data. Notably, several Transformer blocks were incorporated, and a multi-head self-attention mechanism was employed to identify the relationships between the input sequence segments. The pre-processing and data augmentation methodologies significantly enhanced the precision of the results in that the model achieved accuracy rates of 96.3%, 99.86%, 96.5%, and 85.3% on the Ryerson Audio-Visual Database of Emotional Speech and Song, Berlin Database of Emotional Speech, Surrey Audio-Visual Expressed Emotion, and Interactive Emotional Dyadic Motion Capture datasets. Furthermore, it achieved 83% accuracy on another dataset created for this research—the Saudi Higher Education Instructor Emotions dataset. The results demonstrate this model’s considerable accuracy in detecting emotions in speech data across different languages and datasets.

Many engineering systems are described by using differential models whose solutions, generally obtained after discretization, can exhibit a noticeable deviation with respect to the response of the physical systems that those models are expected to represent. In those circumstances one possibility consists of enriching the model in order to reproduce the physical system behavior. The present paper considers a dynamical system, and proposes enriching the model solution by learning the dynamical model of the gap between the system response and the model-based prediction, while ensuring that the time integration of the learnt model remains stable. The proposed methodology is applied on the simulation of the top-oil temperature evolution of a power transformer, whose data is provided by RTE, the French Electricity Transmission System Operator.

Smart Transformer (ST)-based Meshed Hybrid AC/DC Microgrids (MHM) are a feasible alternative to increase the performance of conventional microgrids (MG) and to increase the penetration of Distributed Energy Resource (DER) at the same time, active and reactive power dispatch on the system. Despite this, MHMs present challenges in managing resources under uncertainties and controlling electronic converters associated with the ST and DER, making it complex to achieve optimal system performance. In this paper, a Data-Driven Distributionally Robust Optimization (DDDRO) for Day-Ahead Operation Planning of an ST-based MHM considering the reactive power dispatch of the DER that minimizes network losses, voltage deviations, and operation cost simultaneously, considering uncertainties of Photovoltaic Generators (PVG) and demand, is proposed. The Column and Constraint Generation (C&CG) algorithm and Duality-Free Decomposition (DFD) method are adopted. Besides, the original mixed-integer non-linear planning problem is converted into a Mixed-Integer (MI) Second-Order Cone Programming (SOCP) problem through second-order cone relaxation and positive octagonal constraint method. Simulation experiments and results with a connected MHM system demonstrate the effectiveness and performance of the proposed model. Finally, we observe the effect of the meshed structure of MG and the positive impact of integrating the ST to form MHM, taking advantage of the degrees of freedom of this multi-stage converter for optimal energy management under uncertain conditions.

Background: Early diagnosis of postoperative complications is an urgent task, allowing timely prescribing of appropriate therapy and reducing the cost of patient treatment. Prognostic models based on various data could help in the early diagnosis of postoperative complications after cardiac surgery. Objectives: The purpose of the study was to determine whether an integrated approach based on clinical data along with metabolites and biomarkers had greater predictive value than the models built on less data. Methods: The study included patients (n=62) admitted for planned cardiac surgery (coronary artery bypass grafting with cardiopulmonary bypass), who were retrospectively divided into two groups depending on the presence (n=26) or absence (n=36) of postoperative complications of all types. Clinical and laboratory data on the first day after surgery were analyzed. Additionally, patients' blood samples were collected before and on the first day after surgery to determine biomarkers (interleukin-6, procalcitonin, NT-proBNP, high-sensitivity troponin T) and metabolites (8 aromatic and 2 dicarboxylic acids were detected using gas chromatography-mass spectrometry). Results: Patients before surgery, compared with healthy donors, differed in the concentration of metabolites (p < 0.05), and had concentrations of a number of biomarkers exceeding the corresponding reference values. Multivariate PLS-DA models, predicting the presence or absence of postoperative complications, were built using clinical data (ROC-AUC=0.71), concentrations of metabolites and biomarkers (ROC-AUC=0.60), and the entire data set (ROC-AUC=0.75). For comparison we built univariate models using the EuroScore2 (ROC-AUC=0.59) and SOFA (ROC-AUC=0.76) scales; concentrations of lactate (ROC-AUC=0.71), the dynamic changes of 4-hydroxyphenyllactic acid (ROC-AUC=0.69), and the sum of three sepsis-associated metabolites (ROC-AUC=0.70). The proposed complex model using entire data set was comparable to the SOFA scale, which confirms the expediency of searching for new predictive models based on a variety of factors.

Milk serves as a crucial source of natural bioactive compounds essential for human nutrition and health. The gastrointestinal digestion of milk proteins releases various biologically active peptides that positively impact human health and enhance physiological performance. With the growing production of cheese, Greek yogurt, and other dairy products, the volume of unutilized acid whey is significantly increasing. Acid whey, containing almost all whey proteins and approximately one-quarter of casein, presents a valuable raw material for generating peptides with potential health benefits. These peptides exhibit properties such as antioxidant, antimicrobial, anti-inflammatory, anticarcinogenic, antihypertensive, antithrombotic, opioid, mineral-binding, and growth-stimulating activities, contributing to improved human immunity and the treatment of chronic diseases. Enzymatic hydrolysis of milk proteins has been developed as a method to produce these peptides, facilitating their use in food fortification. To incorporate whey into functional beverages, it undergoes fermentation by lactic acid bacteria with proteolytic activity. To specifically enrich these beverages with peptides exhibiting targeted bioactivities, it is essential to select lactic acid bacteria strains possessing the requisite protease profiles. Additionally, the sensory properties of the final product can be enhanced using starter cultures with diverse metabolic capabilities, even in the absence of added flavoring agents. Given the imperative to develop waste-free technologies and enhance the profitability of the dairy industry, there is a growing need to efficiently utilize and transform acid whey into a broad spectrum of valuable products and food additives. This approach is justified not only by the nutritional benefits of the bioactive components in whey but also by the commercial potential of safe and health-enhancing products.

The paper focuses on the application of a multilevel, integrative and territorial approach to the protection of cultural heritage through planning instruments in Serbia. It points out the problems of the sectoral approach and discusses international recommendations that can be integrated into the planning methodology. The case study is the area of Djerdap National Park, which has an already established international and national importance. The research suggests that the cultural heritage in the area, with the exception of the Roman limes, is still recognized at the level of individual localities. The study emphasizes the importance of aligning all levels of governance to safeguard the whole spatial entity rather than focusing solely on its particular areas. The interna-tional and national importance of the planning area contributes to the strengthening of awareness of heritage and its institutional protection. However, the vitality of the area, which is key to its sustainable development, comes from the local level. Planning documents contribute to the inte-gration of local values and local communities into development programs and projects.

The impact of interaction SS with MJO on rainfall along Java Island to East Nusa Tenggara has been studied using daily rainfall observation data from 2140 weather observation stations, which are also equipped with GSMaP, OLR, and ERA5 data. There were 61 SS events found, 17 during active MJO and 44 in inactive MJO. Rainfall increases more in the western study area when the active MJO and more in the eastern study area when the inactive MJO is due to SS. SS reduces rainfall in the Jakarta region, regardless of the activity or inactivity of the MJO. SS causes extreme rainfall to only occur in a small part of certain areas, so it tends to significantly reduce the possibility of extreme rainfall. In the southern part of the IMC, SS predominates over MJO in supporting increased water vapor transport. Large-scale synoptic circulations like the monsoon, SS, and MJO can interact with local land-sea wind circulation to produce spatiotemporal variability in rainfall on Java Island. Rainfall mostly increases in the afternoon and decreases in the morning when SS occurs, whether there is MJO or not. Convective instability analysis indicates that SS increases precipitation, most likely by lowering CIN and raising VIMFC.

In order to shorten the detection time and improve the average precision on embedded devices, A lightweight and high accuracy model is proposed for passion fruit in complex environments (backlight, occlusion, overlap, sunny, cloudy, rainy). Firstly, replacing the backbone network of YOLOv5 with a lightweight GhostNet model reduces the number of parameters and computation while improving detection speed. Secondly, a new feature branch is added to the GhostNet network, and the feature fusion layer in the neck network is reconstructed to effectively combine the lower-level and higher-level features, which not only improves the accuracy of the model but also maintains its lightweight. Finally, the knowledge distillation methods are used to transfer the knowledge from the more capable teacher model to the less capable student model, which significantly improving the detection accuracy. The improved model is denoted as G-YOLO-NK. The average accuracy of the G-YOLO-NK network is 96.00%, which is 1.00% higher than the original YOLOv5s model. Furthermore, the improved model size is 7.14MB, reduced to half of the original model, and the real-time detection frame rate is 11.25 FPS on the Jetson Nano. Compared to the state-of-the-art model, the proposed model outperforms them in terms of average precision and detection performance. The present work provides an effective model for real-time detection of passion fruits in complex orchard scenes, which can provide valuable technical support for the development of orchard picking robots and greatly improve the intelligence level of orchards.

The Simultaneous Localization And Mapping (SLAM) scanner is an easy and portable Light Detection And Ranging (LiDAR) data acquisition device. Its main output is a 3D point cloud covering the scanned scene. Regarding the importance of accuracy in the survey domain, this paper aims to assess the accuracy of two SLAM scanners: the NavVis VLX and the BLK2GO Scanner. This assessment is conducted for both outdoor and indoor environments. In this context, two types of reference data are used: the Total Station (TS) and the static scanner Z+F Imager 5016. To carry out the assessment, four comparisons are tested: cloud to cloud, cloud to mesh, mesh to mesh, and edge detection board assessment. However, the results of the assessments confirm that the accuracy of indoor SLAM scanner measurements (5 mm) is greater than that of outdoor ones (between 10 mm and 60 mm). Moreover, the comparison of cloud to cloud provides the best accuracy regarding direct accuracy measurement without manipulations. Finally, based on the high accuracy, scanning speed, flexibility, and the accuracy differences between tested cases, it is confirmed that SLAM scanners are effective tools for data acquisition.

In recent years, electroencephalogram (EEG) emotion recognition has emerged as a research hotspot in the field of artificial intelligence (AI). Despite advancements in EEG-based emotion recognition, room for enhancing performance persists due to EEG signal redundancy and limitations in feature extraction, leading to inefficient models and emotional information loss. To fully utilize EEG's emotional information and improve recognition accuracy while reducing computational costs, this paper proposes a Convolutional-Recurrent Hybrid Network with a dual-stream adaptive approach and an attention mechanism (CSA-SA-CRTNN). Firstly, the model utilizes a CSAM module to assign corresponding weights to EEG channels. Then, an adaptive dual-stream convolutional-recurrent network (SA-CRNN and MHSA-CRNN) is applied to extract local spatial-temporal features. After that, the extracted local features are concatenated and fed into a temporal convolutional network with a multi-head self-attention mechanism (MHSA-TCN) to capture global information. Finally, the extracted EEG information is used for emotion classification. We conducted binary and ternary classification experiments on the DEAP dataset, achieving 99.26% and 99.15% accuracy for arousal and valence in binary classification, and 97.69% and 98.05% in ternary classification, surpassing relevant algorithms. Additionally, the model's efficiency is significantly higher than other models, achieving better accuracy with lower resource consumption.

Glycosidic linkage analysis was conducted on the unfractionated polysaccharides in the alcohol-insoluble residues (AIRs) prepared from six red seaweeds (Gracilariopsis sp., Prionitis sp., Mastocarpus papillatus, Callophyllis sp., Mazzaella splendens, and Palmaria palmata) using GC-MS/FID analysis of partially methylated alditol acetates (PMAAs). The cell walls of P. palmata primarily contained mixed-linkage xylans and small amounts of sulfated galactans and cellulose. In contrast, the unfractionated polysaccharides of the other five species were rich in galactans displaying diverse 3,6-anhydro-galactose and galactose linkages with varied sulfation patterns. Different levels of cellulose were also observed. This glycosidic linkage method offers advantages for cellulose analysis over traditional monosaccharide analysis that is known for underrepresenting glucose in crystalline cellulose. Relative linkage compositions calculated from GC-MS and GC-FID measurements showed that anhydro sugar linkages generated more response in the latter detection method. This improved linkage workflow presents a useful tool for studying polysaccharide structural variations across red seaweed species. Furthermore, for the first time, relative linkage compositions from GC-MS and GC-FID measurements, along with normalized FID and total ion current (TIC) chromatograms without peak assignments, were analyzed using principal component analysis (PCA) as a proof-of-concept demonstration of the technique's potential to differentiate various red seaweed species.

Background: Educating college students about the importance of good sleep habits could lead to the adoption of healthier sleep routines, including sufficient sleep duration and sleep quality. The current pre-post study examines the impact of completing a sleep education program called “Sleep 101” on (1) sleep hygiene practices, (2) sleep quality, and (3) weekday and weekend sleep behaviors among college students. “Sleep 101” is an interactive online course developed to educate college students about the importance of sleep by addressing basic sleep physiology, the impact of sleep on mood, academic and physical performance, the interactions between sleep and various substances, and common sleep disorders. Methods: 338 college students were recruited from a midsized public university in the US and invited to participate in “Sleep 101”, of which 25 participants completed the baseline and post-intervention questionnaire. The Sleep Hygiene Index (SHI) was utilized to evaluate sleep hygiene practices, while standard questions were employed to assess weekday and weekend sleep behaviors and overall sleep quality. Results: Those who completed the study demonstrated a significant increase in weekday sleep duration and were less likely to engage in negative prior bedtime activities. In addition, 40% of those who completed baseline and post-intervention assessments noticed improvements in their sleep quality. Conclusion: These findings suggest that a brief online program is beneficial in improving sleep behaviors among college and could be a scalable approach to addressing current sleep issues and preventing future sleep problems in college students.

The profile of secondary metabolites occurring in apple cuticular layer is not only characteristic for particular apple cultivar, but also dynamically reflects various external factors in growing environment. In this study, the possibility to authenticate apple samples based on analysis of their cuticular layer extracts was investigated. Ultra-high-performance liquid chromatography coupled to high resolution tandem mass spectrometry (UHPLC-HRMS/MS) was employed for obtaining metabolomic fingerprints. In total, 274 authentic apple samples, four cultivars harvested within the seasons 2020 to 2022 in the Czech Republic and Poland were analysed. The generated complex data processed by univariate and multivariate statistical methods enabled to build classification models for discrimination of apple cultivars as well as their geographical origin. The models showed very good performance in differentiation of Czech and Polish samples for three of four cultivars: ‘Gala’, ‘Golden Delicious’ and ‘Idared’, moreover, the validity of the models was verified over harvest seasons. The diagnostic markers, besides metabolites of triterpene biosynthetic pathway, mainly represented wax esters. ‘Jonagold’, which is known to be susceptible to mutations, was the only cultivar for which the unambiguous classification of geographical origin was not possible.

Superior fiber quality is one of the most important objectives in cotton breeding. To detect the genetic basis underlying fiber quality, an F2 population containing 413 plants was constructed by crossing Jifeng 914 and Jifeng 173, both of which have superior fiber quality, with Jifeng 173 being better. Five fiber quality traits were investigated in the F2, F2:3, F2:4, and F2:5 populations. Quantitative trait loci (QTL) mapping was conducted based on a high-density genetic map containing 11,488 single nucleotide polymorphisms (SNPs) and spanning 4,202.12 cM in length. Transgressive segregation patterns and complex correlations in the five tested traits were observed. A total of 108 QTLs were found, including 27 for fiber length (FL), 16 for fiber strength (FS), 24 for micronaire (MC), 17 for fiber uniformity (FU), and 24 for fiber elongation rate (FE). Chromosome A7 contained 12 QTL, ranking the first. No QTL was found on chromosome D1 and D11. Each QTL contributed 1.98%–21.45% to the phenotypic variation (PV), including 13 major effect QTLs that contributed more than 10% toward PV. Two QTLs could be repeatedly detected in three populations, including qFL-D3-2 in F2, F2:4, and F2:5 with 9.18%–21.45% of PV, qFS-A11-1 in F2:3, F2:4 and F2:5 with 6.05%–10.41% of PV. Another seven stable QTLs could be detected in two populations, including four major effect QTLs: qFL-A12-3, qFS-D10-2, qMC-D6-2, and qMC-D8-1. Fourteen QTL-overlapping regions were found, which might explain the complex correlations among the five phenotypic traits. Four regions on chromosome A11, D3, D6, and D10 covered by both stable and major effect QTLs are promising for further fine mapping. The genomic regions of the two QTLs detected in three populations and the four major effect QTLs contain 810 genes. Gene functional analysis revealed that the annotated genes are mainly involved in protein binding and metabolic pathways. Fifteen candidate genes in the qFL-D3-2 region are highly expressed in fiber or ovules during fiber initiation, elongation, secondary cell wall thickening, or maturation stages. qRT-PCR revealed that Ghir_D03G005440.1 and Ghir_D03G011310.1 may play a role in promoting fiber initiation, Ghir_D03G006470.1 may be beneficial for promoting fiber elongation. This study provides more information for revealing the molecular genetic basis underlying cotton fiber quality.

Currently there are fewer depth models applied to pepper picking detection, while the existing generalized neural networks have problems such as large model parameters, long training time, and low model accuracy.In order to solve the above problems, this paper proposes a Yolo-chili target detection algorithm for chili pepper detection. First, the classical target detection algorithm yolov5 is used as a benchmark model, and an adaptive spatial feature pyramid structure combining the attention mechanism and the idea of multi-scale prediction is introduced to improve the model's detection effect on occluded peppers and small target peppers. Secondly, a three-channel attention mechanism module is introduced to improve the algorithm's long-distance recognition ability and reduce the interference of redundant testers. Finally, the quantized pruning method is used to reduce the model parameters and realize the lightweight processing of the model. Applying the method to the homemade chili pepper dataset, the AP value of chili pepper reaches 93.11%; the accuracy rate is 93.51% and the recall rate is 92.55%.The experimental results show that yolo-chili is able to achieve accurate and real-time pepper detection under complex orchards.

The temporary immersion system bioreactor named GreenTray®[2], [3]presents advantages over the existing ones. Firstly, there is no need to use forceps to move the shoots or plantlets in or out of the recipient, neither the use of scalpel to divide them. Secondly, the basis of the shoot can remain in the GreenTray® and can sprout again in several cycles of growth. These two characteristics reduce the costs in the process of micropropagation. Besides, shoots or plantlets development is enhanced by the aeration of the recipient, thus enriching with CO2 the recipient atmosphere. The chapter describes and provides images and values of interest in (1) the micropropagation process of some commercial Prunus or Pyrus rootstocks; (2) the plantlet growth under autotrophic conditions; and (2) the inoculation of plants with pathogens that reproduce in vitro the symptoms observed in vivo, using the GreenTray® bioreactor.

The aim of this retrospective study was to compare the clinical outcomes of treating isolated surgical neck fractures of the humerus using either the intramedullary nail or the MIROS (Minimally Invasive Reduction and Osteosynthesis System®) system. The study included 42 patients who underwent surgical treatment at the Polyclinic from January 2014 to June 2020. The results indicated that both treatments were effective; however, the MIROS system exhibited a higher complication rate. Intramedullary nailing was more reliable, thus recommended as the primary treatment method. This study underscores the importance of selecting the appropriate fixation technique to minimize complications and enhance patient outcomes.

Aiming at the problems of inaccurate segmentation of long object and information loss of small object in real-time semantic segmentation algorithm, this paper proposes a lightweight multi-branch real-time semantic segmentation network based on BiseNetV2. The new auxiliary branch makes full use of spatial details and context information to cover the long object in the field of view. Meanwhile, in order to ensure the inference speed of the model, the asymmetric convolution is used in each stage of the auxiliary branch to design a structure with low computational complexity. In the multi-branch fusion stage, the alignment and fusion module is designed to provide guidance information for deep and shallow feature mapping, so as to make up for the problem of feature misalignment in the fusion of information at different scales, and thus reduce the loss of small target information. In order to further improve the model’s awareness of key information, a global context module is designed to capture the most important features in the input data. The proposed network uses NVIDIA GeForce RTX 3080 Laptop GPU experiment, on the road street view data set Cityscapes and CamVid average occurring simultaneously ratio reached 77.1% and 78.4% respectively, with running speed of 127 frames/s respectively and 112 frames/s. The experimental results show that the proposed algorithm can achieve real-time segmentation and improve the accuracy significantly, showing good semantic segmentation performance

Excess nitrogen in agricultural drainage poses a serious threat to the water quality safety of the Yellow River basin. Utilizing aquatic plants to modify the rhizosphere microbial community structure and facilitate nitrogen transformation is a crucial strategy for mitigating regional water eutrophication. Our results revealed that the denitrification rate (DR), dissimilatory nitrate reduction to ammonium rate (DNRAR) and nitrogen fixation rate (NFR) of rhizosphere sediments exhibited a significant increase in artificially cultivated ditch. Compared with those in natural ditches, the richness of the bacterial community and the relative abundances of Bacteroidota, Verrucomicrobiota, Firmicutes, Anaeromyxobacter and Geobacter, which are involved mainly in the nitrogen cycle, in the rhizosphere sediments of artificially cultivated ditch were significantly greater due to improvements in environmental conditions. In artificially cultivated ditch, the dominant functional genes affecting DRNARs in the rhizosphere sediments of P. australis were nrfC and nrfA, while DRs were driven mainly by norB and napA, which were influenced by the nitrogen and carbon levels; and the dominant functional genes affecting NFRs in the rhizosphere sediments of T. orientalis were nifD, nifK and nifH. Our results provide a scientific basis for the use of aquatic plants for mitigating excess nitrogen levels in agricultural drainage.

Strawberry root rot caused by Neopestalotiopsis clavispora is one of the main diseases of strawberry and significantly impacts the yield and quality of strawberry fruit. There are now no effective control techniques available except for fungicide sprays, which may directly impact consumers. Biological control is becoming an alternative method for the control of plant diseases to replace or decrease the application of traditional synthetic chemical fungicides. Trichoderma is widely used as a biological agent for controlling strawberry root rot. In order to provide resources for screening the highly effective biocontrol fungus for controlling strawberry root rot caused by Neopestalotiopsis clavispora, the biocontrol mechanism,the control effects of Trichoderma asperellum CMT10 against strawberry root rot and growth-promoting effects on strawberry seedlings were investigated by plate culture, microscopy observation and root drenching methods.The results showed that CMT10 had obvious competitive, antibiotic and hyperparasitism effects on Neopestalotiopsis clavispora CMGF3. The CMT10 could quickly occupy nutritional space, and the inhibition rates of CMT10 against CMGF3 in confrontation culture on 7 d was 65.49%. The inhibition rates of volatile metabolite and fermentation metabolite produced by CMT10 against CMGF3 were 79.67% and 69.84%, respectively. The mycelium of strain CMT10 can act as hyperparasites by making contacting, winding and penetrating hyphae of CMGF3. Pot experiment showed that the biocontrol efficacy of CMT10 on strawberry root rot caused by Neopestalotiopsis clavispora was 63.00%. CMT10 had obvious promotion effect on strawberry growth, the plant height, root length, total fresh weight, root fresh weight, stem fresh weight and root dry weight, with the promotion rates were 20.09%, 22.39%, 87.11%, 101.58%, 79.82% and 72.33%, respectively. Overall, this study showed the ability of Trichoderma asperellum CMT10 to control strawberry root rot and their potential to develop as novel biocontrol agents to replace chemical fungicides for eco-friendly, sustainable agriculture.

In this paper we study a class of impulsive delayed neural networks with parameter uncertainties and reaction-diffusion terms. Using the properties of M-matrices and some inequality techniques, some sufficient conditions are derived to guarantee the global exponential stability of the given system. The results obtained are presented by simple algebraic inequalities, which are certainly more concise than previous methods. Finally, the validity and usefulness of our results are illustrated by arithmetic examples.