Multimodal remote sensing data classification can enhance the model’s ability to distinguish land features through multimodal data fusion. In this context, how to help models understand the relationship between multimodal data and target tasks is the focus of researchers. Inspired by human feedback learning mechanisms, causal reasoning mechanisms, and knowledge induction mechanisms, this paper integrates causal learning, reinforcement learning, and meta learning into a unified remote sensing data classification framework and proposed the causal meta-reinforcement learning (CMRL). First, based on feedback learning mechanisms, we have overcome the limitations of traditional implicit optimization of fusion features and customized a reinforcement learning environment for multimodal remote sensing data classification tasks. Through feedback interactive learning between agents and the environment, we help them understand the complex relationships between multimodal data and labels, thereby achieving full mining of multimodal complementary information. Second, based on the causal inference mechanism we designed causal distribution prediction action, classification reward, and causal intervention reward, capturing pure causal factors in multimodal data and cutting off false statistical associations between non-causal factors and class labels. Finally, based on the knowledge induction mechanism, we designed a bi-layer optimization mechanism based on meta-learning. By constructing a meta training task and meta validation task simulation model in the generalization scenario of unseen data, we helped the model induce cross-task shared knowledge, thereby improving its generalization ability for unseen multimodal data. The experimental results on multiple sets of multimodal datasets show that the proposed method achieved state-of-the-art performance in multimodal remote sensing data classification tasks.

The objectives of the study were to determine the spatial rate of CO₂ flux (Net Ecosystem Exchange) and soil moisture in a wetland ecosystem applying Sentinel-1 IW (Interferometric Wide) data of VH (Vertical Transmit/Horizontal Receive—cross polarization) and VV (Vertical Transmit/Vertical Receive—like polarization) polarization. In-situ measurements of carbon flux, soil moisture, and LAI (Leaf Area Index) were carried out over the Biebrza Wetland in north-eastern Poland. The impact of soil moisture and LAI on backscattering coefficient (σ°) calculated from Sentinel-1 data showed that LAI dominates the influence on σ° when soil moisture is low. The models for soil moisture have been derived for wetland vegetation habitat types applying VH polarization (R² = 0.70 to 0.76). The vegetation habitats: reeds, sedge-moss, sedges, grass-herbs, and grass were classified using combined one Landsat 8 OLI (Operational Land Imager) and three TerraSAR-X (TSX) ScanSAR VV data. The model for the assessment of Net Ecosystem Exchange (NEE) has been developed based on the assumption that soil moisture and biomass represented by LAI have an influence on it. The σ° VH and σ° VV describe soil moisture and LAI, and have been the input to the NEE model. The model, created for classified habitats, is as follows: NEE = f (σ° Sentinel-1 VH, σ° Sentinel-1 VV). Reasonably good predictions of NEE have been achieved for classified habitats (R² = 0.51 to 0.58). The developed model has been used for mapping spatial and temporal distribution of NEE over Biebrza wetland habitat types. Eventually, emissions of CO₂ to the atmosphere (NEE positive) has been noted when soil moisture (SM) and biomass were low. This study demonstrates the importance of the capability of Sentinel-1 microwave data to calculate soil moisture and estimate NEE with all-weather acquisition conditions, offering an important advantage for frequent wetlands monitoring.

Plant leaf area index (LAI) is a key characteristic affecting field canopy microclimate. In addition to traditional professional measuring instruments, smartphone camera sensors have been used to measure plant LAI. However, when smartphone methods were used to measure conifer forest LAI, very different performances were obtained depending on whether the smartphone was held at the zenith angle or at a 57.5° angle. To validate further the potential of smartphone sensors for measuring conifer LAI and to find the limits of this method, this paper reports the results of a comparison of two smartphone methods with an LAI-2000 instrument. It is shown that both methods can be used to reveal the conifer leaf-growing trajectory. However, the method with the phone oriented vertically upwards always produced better consistency in magnitude with LAI-2000. The bias of the LAI between the smartphone method and the LAI-2000 instrument was explained with regard to four aspects that can affect LAI: gap fraction, leaf projection ratio, sensor field of view (FOV), and viewing zenith angle (VZA). It was concluded that large FOV and large VZA cause the 57.5° method to overestimate the gap fraction and hence underestimate conifer LAI, especially when tree height is greater than 2.0 m. For the vertically upward method, the bias caused by the overestimated gap fraction is compensated for by an underestimated leaf projection ratio.

Dynamic optical structural color is always desired in various display applications and usually involves active materials. Full-color generation, especially bi-directional full-color generation in both reflective and transmissive modes, without any active materials included has been rarely investigated. Herein, we demonstrate a scheme of bi-directional full-color generation based on a plasmonic metasurface modulated by rotation of the polarization angle of the incident light without varying the geometry and the optical properties of the materials and the environment where the metasurface resides. The metasurface unit cell consists of plasmonic modules aligning in three directions and is patterned in a square array. The metasurface structural color device generates full colors in both reflection and transmission. Based on the proposed polarization-dependent structural color, information encoding of three multiplexed animal images and quick-responsive (QR) codes is demonstrated to verify the efficient information encoding and decoding of the proposed scheme. The animals can be seen under different polarization incidence and the QR codes can be successfully decoded by the polarization rotation in transmission. The proposed bi-directional full-color generation metasurface has great potential in applications such as kaleidoscope generation, anti-counterfeiting, dynamic color display, and optical information encoding.

Image stitching is a crucial aspect of image processing. However, factors like perspective and environment often lead to irregular shapes in stitched images. Cropping or completion methods typically result in substantial loss of information. This paper proposes a method for rectifying irregularly images into rectangles using deformable meshes and residual networks. The method utilizes a convolutional neural network to quantify rigid structures of images. Choosing the most suitable mesh structure based on the extraction results, offering options such as triangular, rectangular, and hexagonal. Subsequently, the irregularly image, predefined mesh structure, and predicted mesh structure are input into a wide residual neural network for regression. The loss function comprises local and global, aimed at minimizing the loss of image information within the mesh and global target. This method not only significantly reduces information loss during rectification but also adapting to different images with various rigid structures. Validation on the DIR-D dataset shows this method outperforms state-of-the-art methods in image rectification.

Surface water distribution extracted from remote sensing data has been used in water resource assessment, coastal management, and environmental change studies. Traditional manual methods for extracting water bodies cannot satisfy the requirements for mass processing of remote sensing data; therefore, accurate automated extraction of such water bodies has remained a challenge. The histogram bimodal method (HBM) is a frequently used objective tool for threshold selection in image segmentation. The threshold is determined by seeking twin peaks, and the valley values between them; however, automatically calculating the threshold is difficult because complex surfaces and image noise which lead to not perfect twin peaks (single or multiple peaks). We developed an operational automated water extraction method, the modified histogram bimodal method (MHBM). The MHBM defines the threshold range of water extraction through mass static data; therefore, it does not require the identification of twin histogram peaks. It then seeks the minimum values in the threshold range to achieve automated threshold. We calibrated the MHBM for many lakes in China using Landsat 8 Operational Land Imager (OLI) images, for which the relative error (RE) and squared correlation coefficient (R2) for threshold accuracy were found to be 2.1% and 0.96, respectively. The RE and root-mean-square error (RMSE) for the area accuracy of MHBM were 0.59% and 7.4 km2. The results show that the MHBM could easily be applied to mass time-series remote sensing data to calculate water thresholds within water index images and successfully extract the spatial distribution of large water bodies automatically.

For multi-sensor integrated systems, such as a mobile mapping system (MMS), data fusion at sensor-level, i.e., the 2D-3D registration between optical camera and LiDAR, is a prerequisite for higher level fusion and further applications. This paper proposes a line-based registration method for panoramic images and LiDAR point cloud collected by a MMS. We first introduce the system configuration and specification, including the coordinate systems of the MMS, the 3D LiDAR scanners, and the two panoramic camera models. We then establish the line-based transformation model for panoramic camera. Finally, the proposed registration method is evaluated for two types of camera models by visual inspection and quantitative comparison. The results demonstrate that the line-based registration method can significantly improve the alignment of the panoramic image and LiDAR datasets under either the ideal spherical or the rigorous panoramic camera model, though the latter is more reliable.

The experiences gathered during the past 30 years support the operational use of irrigation scheduling based on frequent multi-spectral image data. Currently, the operational use of dense time series of multispectral imagery at high spatial resolution makes monitoring of crop biophysical parameters feasible, capturing crop water use across the growing season, with suitable temporal and spatial resolutions. These achievements, and the availability of accurate forecasting of meteorological data, allow for precise predictions of crop water requirements with unprecedented spatial resolution. This information is greatly appreciated by the end users, i.e. professional farmers or decision-makers, and can be provided in an easy-to-use manner and in near-real-time by using the improvements achieved in web-GIS methodologies. This paper reviews the most operational and explored methods based on optical remote sensing for the assessment of crop water requirements, identifying strengths and weaknesses and proposing alternatives to advance towards full operational application of this methodology. In addition, we provide a general overview of the tools which facilitates co-creation and collaboration with stakeholders, paying special attention to these approaches based on web-GIS tools.

Automatic ice charting can not be achieved using only SAR modalities. It is fundamental to combine information from other remote sensors with different characteristics for more reliable sea ice characterization. In this paper, we employ principal feature analysis (PFA) to select significant information from multimodal remote sensing data. PFA is a simple yet very effective approach that can be applied to several types of data without loss of physical interpretability. Considering that different homogeneous regions require different types of information, we perform the selection patch-wise. Accordingly, by exploiting the spatial information, we increase the robustness and accuracy of PFA.

Ambulatory heart rate (HR) monitors that acquire electrocardiogram (ECG) or/and photoplethysmographm (PPG) signals from the torso, wrists, or ears are notably less accurate in tasks associated with high levels of movement compared to clinical measurements. However, reliable estimation of HR can be obtained through data fusion from different sensors. These methods are especially suitable for multimodal hearable devices, where heart rate can be tracked from different modalities, including electrical ECG, optical PPG, and sounds (heart tones). Combined information from different modalities can compensate for single source limitations. In this paper, we evaluate the possible application of data fusion methods in hearables. We assess data fusion for heart rate estimation from simultaneous in-ear ECG and in-ear PPG, recorded on eight subjects while performing 5-minute sitting and walking tasks. Our findings show that data fusion methods provide a similar level of mean absolute error as the best single-source heart rate estimation but with much lower intra-subject variability, especially during walking activities. We conclude that data fusion methods provide more robust HR estimation than a single cardiovascular signal. These methods can enhance the performance of wearable devices, especially multimodal hearables, in heart rate tracking during physical activity.

The atmospheric lightning is one of the natural disasters which need to be considered in the sustainable development. The National Remote Sensing Centre (NRSC) has installed 46 lightning detection sensors (LDS) across the country. These sensors work in radio frequencies having their detection range up to 1000 km with more than 98% detection efficiency within 300 km range. The network is put in a way that sensors have about 50% sensor-to-sensor overlap in their detection ranges to enable high geolocation accuracy and maintain redundancy. The NRSC-LDS network (LDSN) estimates the current associated with each cloud to ground (CG) lightning flash which can be utilized for the risk assessment. The current estimates from one network to the other need to be inter-calibrated and harmonized. Present study compares the LDSN current estimates with Lightning Location Network (LLN) estimates and results are discussed with their impact on the estimation of vulnerable zones. A case study of lightning-induced fire event is presented to showcase the importance of the currents associated with lightning for future planning.

This paper is aimed at studying the temporal evolution of the surface displacements occurred over the past few years in the ocean-reclaimed platforms of the Shanghai megacity (China), which are mainly ascribable to consolidation processes of large dredger fills and alluvial deposits. With respect to previous analyses carried out over the same area, this work provides a joint multi-platform differential interferometry synthetic aperture radar (DInSAR) analysis, based on the application of the advanced Small BAseline Subset (SBAS) algorithm. This led us to retrieve long-term deformation time-series that are helpful for a better understanding of the on-going deformation phenomena. To this aim, we have exploited two sequences of SAR data collected by the ASAR/ENVISAT and by the COSMO-SkyMed (CSK) sensors, respectively, spanning the whole time period from 2007 to 2016. Unfortunately, the large time gap (of about three years) existing between the available ASAR/ENVISAT and CSK datasets gave rise to additional difficulties for their combination. Nevertheless, this problem has been faced by benefiting from the knowledge of a time-dependent model describing the temporal evolution of the expected deformations affecting the Shanghai ocean-reclaimed platforms.

Background: The use of wearable devices has burgeoned over the past decade, including wearables for infants and young children. Typically, such devices assess a single modality, and few have undergone scientific validation. To address this gap, we developed an infant wearable platform, LittleBeats™, that integrates electrocardiogram (ECG), motion, and audio sensors on a single printed circuit board to permit daylong remote assessments of infants and caregivers in home environments. Objective: Complementing our prior reports focused on LittleBeats™ data collected among infants and children in the home context, our main objective here is to present a technical validation of each sensor modality against established laboratory protocols and gold-standard equipment. Materials and Methods: We conducted five studies, including assessments of (a) interbeat interval (IBI) data obtained from the LittleBeats™ ECG sensor versus the gold-standard BIOPAC among adults (Study 1, N=16) and infants (Study 2, N=5), (b) performance of automated activity recognition (upright vs. walk vs. glide vs. squat ) among adults using accelerometer data obtained from LittleBeats™ versus Google Pixel 1 smartphone (Study 3, N=12), and (c) performance of speech emotion recognition (SER; Study 4, N=8) and automatic speech recognition (ASR; Study 5, N =12) algorithms among adult samples using audio data from LittleBeats™ versus smartphone. Results: Results for IBI data obtained from the LittleBeats™ ECG sensor indicate acceptable mean absolute percent error (MAPE) rates for both the adults (MAPE = 5.29% to 5.97%; Study 1) and infants (MAPE =0.96% to 1.66%; Study 2) across low- and high-challenge sessions, as well as expected patterns of change in respiratory sinus arrythmia (RSA) across sessions. For activity recognition (Study 3), LittleBeats™ data showed good to excellent performance (Accuracy = 89%, F1-score=88%; Cohen’s kappa =.79), although the smartphone outperformed LittleBeats™ by less than 4%. Finally, performance on the SER applied to LittleBeats™ versus smartphone audio data (Study 4) indicated comparable performance, with a matched-pairs test indicating no significant difference in error rates (p=.26). On the ASR task (Study 5), the best performing algorithm yielded relatively low word error rates, although LittleBeats™ (4.16%) versus smartphone (2.73%) error rates are somewhat higher. Discussion: Taken together, results from these controlled laboratory studies conducted predominantly with adults indicate that the LittleBeats™ sensors yield data quality that is largely comparable to those obtained from gold-standard devices and protocols that have been used extensively in prior research. The advantages of the LittleBeats™ platform is the (a) integration of multiple sensors into one platform that was (b) designed specifically for use with infants and young children. Leveraging data from the multiple modalities and fine-tuning postprocessing steps to further increase data quality, combined with assessing data quality among infants and young children in their natural environments, will be key next steps in this research program.

Daily maximum surface air temperature (Tamax) is a crucial factor for understanding complex land surface processes under rapid climate change. Remote detection of Tamax has widely relied on the empirical relationship between air temperature and land surface temperature (LST), a product derived from remote sensing. However, little is known about how such a relationship is affected by the high heterogeneity in landscapes and dynamics in seasonality. This study aims to advance our understanding of the roles of land cover and seasonal variation in the estimation of Tamax using the MODIS (Moderate Resolution Imaging Spectroradiometer) LST product. We developed statistical models to link Tamax and LST in the middle and lower reaches of the Yangtze River in China for five major land-cover types (i.e., forest, shrub, water, impervious surface, cropland, and grassland) and two seasons (i.e., growing season and non-growing season). Results show that the performance of modeling the Tamax-LST relationship was highly dependent on land cover and seasonal variation. Estimating Tamax over grasslands and water bodies achieved superior performance; while uncertainties were high over forested lands that contained extensive heterogeneity in species types, plant structure, and topography. We further found that all the land-cover specific models developed for the plant non-growing season outperformed the corresponding models developed for the growing season. Discrepancies in model performance mainly occurred in the vegetated areas (forest, cropland, and shrub), suggesting an important role of plant phenology in defining the statistical relationship between Tamax and LST. For impervious surfaces, the challenge of capturing the high spatial heterogeneity in urban settings using the low-resolution MODIS data made Tamax estimation a difficult task, which was especially true in the growing season.

The mineralogical indices, e.g., Quartz Index (QI), Carbonate Index (CI), Mafic Index (MI) for ASTER multispectral thermal infrared (TIR) data have been applied to various geological materials. The indices are proved to be robust in extracting geological information at the local scale. Considering the relatively narrow spatial coverage of each ASTER scene compared to LANDSAT, many scenes are needed when mosaicking the images to be mapped at regional scales. We develop a system to search ASTER data for a target area given the vast and expanding ASTER data archive. The data is then conditioned, prioritized, and the indices are calculated before finally mosaicking the imagery. The maps of the indices covering the very wide region of the central Tibetan Plateau are produced with this system. The characteristic features of the indices relating to their geology in the study area are analyzed and discussed. Many interesting lithological and structural information that are not currently well understood in the central Tibetan Plateau, the highest and most extensive plateau in the world, with an average elevation of over 4,500 meters above sea level, for example, distributions of the mafic-ultramafic rocks along the suture zones, the quarzitic and marine sedimentology accreted to the Eurasian continent and sulfate layers related to the Tethys and neo-Tethys geological setting can be retrieved from the processed ASTER images.

Improper sitting positions are known as the primary reason for back pain and the emergence of musculoskeletal disorders (MSD) among individuals who spend prolonged time working with computer screens, keyboards, and mice. At the same time, it is well understood that automated technological tools can play an important role in the process of unhealthy habit alteration, so plenty of research efforts are focused on research and technology development (RTD) activities that aim to provide support to the prevention of back pain or the development of MSD. Here, we report on creating a new resource in support of RTD activities aiming at the automated detection of improper sitting positions. It consists of multimodal multisensory recordings of 100 persons made with a video recorder, camera, and wrist-attached sensors that capture physiological signals (PPG, EDA, Skin temperature), as well as motion sensors (three-axis accelerometer). Our multimodal multisensory dataset (MMD-MSD) opens new opportunities for modeling the body stance (sitting posture and movements), physiological state (stress level, attention, emotional arousal, valence), and performance (success rate on the Stroop test) of people working with a computer. Finally, we demonstrate two use cases: improper neck posture detection from pictures and cognitive load detection from physiological signals.

Urban rail transit hub platform is the most important area for passenger flow distribution. In order to calculate passenger flow volume in platform and evaluate platform service level during rush hours, this paper presents a method for modeling and simulation of passenger flow distribution in platform. Passenger flow distribution model (PFDM) is proposed based on the basic analysis and the superposition principle of passenger flow. Simulation design for PFDM is proposed by Anylogic, which contains simulation process and simulation model. Experiment results show that PFDM and simulation design are effective and accordant with the reality scenario, and the simulation precision is comparatively ideal. This research could provide a beneficial reference for train scheduling and operation management under the viewpoint of traffic safety and service level.

The necessity for precise and current data concerning the dynamics of land cover change in Indonesia is crucial for efforts to reduce natural vegetation cover due to agricultural expansion. The functionality of monitoring systems that incorporate Terra-MODIS is currently compromised by the limited availability of data for the immediate future. This study seeks to assess the potential of VIIRS satellite imagery in developing an early warning system for monitoring vegetation cover change in Indonesia. The Normalized Differential Open Area Index (NDOAI) computed from the 8-day’ VIIRS data was employed to detect changes in vegetation cover based on pixel-by-pixel subtraction in the NDOAI data time series. Evaluating the pixel-level accuracy of change detection is complicated due to the fact that we evaluate a change map at a coarser resolution than the Landsat-based reference map. The results revealed that increasing the threshold percentage is associated with improved accuracy. In change detection, there is often a trade-off between accuracy and sensitivity. A threshold that is too low may result in false positives, while a threshold that is too high may lead to missed changes. This study demonstrates that when a threshold value of less than 20% is applied, Landsat can identify vegetation cover changes at an earlier stage. Conversely, when a threshold value greater than 20% is employed, VIIRS will detect the change 4.5 days earlier than Landsat. Additionally, VIIRS is capable of detecting changes 25.4 days and 54.8 days faster than Landsat, respectively, when using thresholds of 40% and 75%.

This article describes the history of research and development of surface-penetrating radar, starting from the 60s in the USSR, and ending today in Russia and neighboring states formed after the collapse of the USSR. The outstanding role played in the initial period of this technology development by Prof. M.I. Finkelstein and his colleagues is highlighted. Their research had a significant impact on subsequent developments in modern Russia and the neighboring countries.

Aerial image classification has become popular and has attracted extensive research efforts in recent decades. The main challenge lies in its very high spatial resolution but relatively insufficient spectral information. To this end, spatial-spectral feature extraction is a popular strategy for classification. However, parameter determination for that feature extraction is usually time-consuming and depends excessively on experience. In this paper, an automatic spatial feature extraction approach based on image raster and segmental vector data cross-analysis is proposed for the classification of very high spatial resolution (VHSR) aerial imagery. First, multi-resolution segmentation is used to generate strongly homogeneous image objects and extract corresponding vectors. Then, to automatically explore the region of a ground target, two rules, which are derived from Tobler’s First Law of Geography (TFL) and a topological relationship of vector data, are integrated to constrain the extension of a region around a central object. Third, the shape and size of the extended region are described. A final classification map is achieved through a supervised classifier using shape, size, and spectral features. Experiments on three real aerial images of VHSR (0.1 to 0.32 m) are done to evaluate effectiveness and robustness of the proposed approach. Comparisons to state-of-the-art methods demonstrate the superiority of the proposed method in VHSR image classification.

In this study, we present comprehensive climatologies of effective ultraviolet (UV) quantities and photosynthetically active radiation (PAR) over Cyprus for the period 2004 to 2023, leveraging the synergy of earth observation (EO) data and radiative transfer model simulations. The EO dataset, encompassing satellite and re-analysis data for aerosols, total ozone column, and water vapor, alongside cloud modification factors, to capture the nuanced dynamics of Cyprus's atmospheric conditions. With a temporal resolution of 15 minutes and a spatial of 0.05°x0.05°, these climatologies undergo rigorous validation against established satellite datasets and are further evaluated through comparisons with ground-based global horizontal irradiance measurements provided by the Meteorological Office of Cyprus. This dual-method validation approach not only underscores the models' accuracy but also highlights their proficiency in capturing intra-daily cloud coverage variations. Our analysis extends to investigating the long-term trends of these solar radiation quantities, examining their interplay with changes in cloud attenuation, aerosol optical depth (AOD), and total ozone column (TOC). Significant decreasing trends in the noon ultraviolet index (UVI), of -2 to -4% per decade have been found in Autumn, especially marked on the island’s Northeastern part, mainly originating from the (significant) positive trends in TOC. The significant decreasing trends in TOC, of -2 to -3% per decade that were found in Spring do not result in correspondingly significant positive trends in the noon UVI since variations in cloudiness and aerosols also have a strong impact on the UVI in this season. The seasonal trends in the Daylight Integral (DLI) were generally not significant. These insights provide a valuable foundation for further studies aimed at developing public health strategies and enhancing agricultural productivity, highlighting the critical importance of accurate and high-resolution climatological data.

The Microwave Humidity and Temperature sounder (MWHTS) on board the Fengyun (FY)-3C satellite measure the outgoing radiance form the Earth surface and atmospheric constituents. MWHTS makes measurements in the isolated oxygen absorption line near 118 GHz and the vicinity of strong water vapor line around 183 GHz, can provide fine vertical distribution structure of both atmospheric humidity and temperature. However, in order to obtain the accurate soundings of humidity and temperature by the physical retrieval method, bias between the observed radiance and those simulated by radiative transfer model from the background or first guess profiles must be correct. In this study, two bias correction methods are developed through the correlation analysis between MWHTS measurements and air mass identified by the first guess profiles of the physical inversion, one is the linear regression correction (LRC) and the other is neural networks correction (NNC), representing the linear and nonlinear nature between MWHTS measurements and air mass, respectively. Both correction methods have been applied to MWHTS observed brightness temperatures over the geographic area (180° W-180° E, 60° S-60° N). The corrected results are evaluated by the probability density function of the difference between corrected observations and simulated values and the root mean square error (RMSE) with respect to simulated observations. The numerical results show that the NNC method perform better, especially in MWHTS channels 1 and 7-9 whose peak weight function heights are close to the surface. In order to assess the effects of bias correction methods proposed in this study on the retrieval accuracy, a one-dimensional variational system was built and applied to the MWHTS uncorrected and corrected brightness temperatures to estimated atmospheric temperature and humidity profiles, The retrieval results show that the NNC has better performance which is to be expected. An indication of the stability and robustness of NNC method is given which suggests that the NNC method has promising application perspectives in the physical retrieval.

Floods are among the most serious natural disasters worldwide; they cause enormous crop losses every year and threaten world food security. There are many studies focused on flood impact assessments for administrative districts but fewer on postdisaster impact assessments for specific crops. Therefore, this study used remote sensing data, including the normalized difference vegetation index (NDVI), elevation data, slope data, and rainfall data, combined with crop growing season data to construct a crop flood damage assessment index (CFAI). First, an analytic hierarchy process (AHP) was used to assign weights to the impact factors; then, the weighted composite score method was used to calculate the CFAI; and finally, the impact was classified as slight, moderate, or severe based on the magnitude of the CFAI . This method was used for the Missouri River floods of 2019 in the U.S. Due to the lack of measured data, the disaster vegetation damage index (DVDI) was used to validate the results. The combined approach achieved a high degree of consistency in spatial distribution. The CFAI can respond well to the degree of crop impact after flooding, providing new ideas and reference standards for agriculture-related departments.

The human mind is multimodal. Yet most behavioral studies rely on century-old measures of behavior—task accuracy and latency (response time). Multimodal and multisensory analysis of human behavior creates a better understanding of how the mind works. The problem is that designing and implementing these experiments is technically complex and costly. This paper introduces versatile and economical means of developing multimodal-multisensory human experiments. We provide an experimental design framework that automatically integrates and synchronizes measures including electroencephalogram (EEG), galvanic skin response (GSR), eye-tracking, virtual reality (VR), body movement, mouse/cursor motion and response time. Unlike proprietary systems (e.g., iMotions), our system is free and open-source; it integrates PsychoPy, Unity and Lab Streaming Layer (LSL). The system embeds LSL inside PsychoPy/Unity for the synchronization of multiple sensory signals—gaze motion, electroencephalogram (EEG), galvanic skin response (GSR), mouse/cursor movement, and body motion—with low-cost consumer-grade devices in a simple behavioral task designed by PsychoPy and a virtual reality environment designed by Unity. This tutorial shows a step-by-step process by which a complex multimodal-multisensory experiment can be designed and implemented in a few hours. When conducting the experiment, all of the data synchronization and recoding of the data to disk will be done automatically.

Volcanic activity essentially consists of the transfer of heat from the Earth’ interior to the surface. The precise signature of this heat transfer relates directly to the processes underway at and within a particular volcano and this can be observed, at a safe distance, remotely, using infrared sensors that are present on Earth-orbiting satellites. For over 50 years, scientists have perfected this art using sensors intended for other purposes, and they are now in a position to determine the particular sort of activity that characterizes different volcanoes. This review will describe the theoretical basis of the discipline and then discuss the sensors available for the task and the history of their use. Challenges and opportunities for future development in the discipline are then discussed.

Rapid and accurate detection of road cracks is of great significance for road health monitoring, but currently this work is mainly completed through manual site surveys. Low-altitude UAV remote sensing can obtain images with centimeter or even subcentimeter ground resolution, which provides a new efficient and economical approach for rapid crack detection. Nevertheless, crack detection networks face challenges such as edge blurring and misidentification due to the heterogeneity of road cracks and the complexity of the background. To address these issues, we propose a real-time edge reconstruction crack detection network (ERNet) which adopts multi-level information aggregation to reconstruct crack edges and improve the accuracy of segmentation between the target and background. To capture global dependencies across spatial and channel levels, we propose an efficient bilateral decomposed convolutional attention module (BDAM) that combines depth separable convolution and dilated convolution to capture global dependencies across spatial and channel levels. To enhance the accuracy of crack detection, we use a coordinate-based fusion module that integrates spatial, semantic, and edge reconstruction information. In addition, we propose an automatic measurement of crack information for extracting the crack trunk and its corresponding length and width. Experimental results demonstrate that our network achieves the best balance between accuracy and inference speed.

Molecular analysis of growing teratoma syndrome has not been extensively studied. Here, we report a 14-year-old boy with a growing mass during treatment for a mixed germ cell tumor of the pineal region. Tumor markers were negative; thus, growing teratoma syndrome was suspected. A radical resection via the occipital transtentorial approach was performed, and histopathological examination revealed a teratoma with malignant features. Methylation classifier analysis confirmed the diagnosis of teratoma, and DMRT1 loss and 12p gain were identified by copy number variation analysis, potentially elucidating the cause of growth and malignant transformation of the teratoma. The patient remains in remission after intense chemoradiation treatment as a high-risk germ cell tumor.

Statistical denoising techniques have been well studied, however many challenges still remain. This 9 article focuses on determining the least necessary number of significant samples in a statistical pop-10 ulation sampling, while minimizing the impact on the statistical detection of the fiber Bragg grating 11 (FBG) power spectra. This is achieved by using Bayesian decision making, small population statis-12 tics while excluding false detection. First, we investigate the impact of two-sided sliding window 13 technique applied near the cell under the test. The window size varies from 8 to 60 samples. The 14 considered maximum size of population sampling is connected with the number of wavelength 15 samples within the bandwidth of the symmetrical FBG power spectra peaks. Regarding the sym-16 metry of FBG power spectra peaks and the possible reduction of computational demands, we inves-17 tigate the impact of a one-sided sliding window. This enables to achieve less processing latency for 18 real time applications thus brings benefits to FBG-based fiber optical sensing. Both, two- and one-19 sided small population sampling techniques are then experimentally verified. Detection results are 20 analyzed in depth via calculating detection thresholds by using the small size window sliding along-21 side the waveband. From results achieved, it was found that the normality three-sigma rule does 22 not need to be followed while using the small population sampling. Experimental demonstrations 23 and analyses showed that the proposed concept of denoising and statistical threshold detection 24 based on the sliding window does not depend on the prior knowledge of probability distribution 25 functions of the FBG power spectra peaks level and background noise. We have also found that the 26 thresholds’ adaptability mainly depends on the statistical numerical characteristics of the above ad-27 ditive mixture of the FBG power spectra peaks signals and background noise.

Detecting and monitoring changes in soil salinity through remote sensing provides an opportunity for field assessment in regions where on-site measurements are limited. This investigation, carried out in Siwa Oasis, Egypt, aimed to assess the efficacy of several soil salinity indices derived from Landsat 5 and 7 satellite images. To achieve this, 56 on-site ground measurements were utilized for evaluation purposes. The study aimed to improve the correlation between EC and index values and explore the relationship between salinity and changes in land cover. Eleven spectral indices were calculated for nine scenes captured in different months from August to December. The first approach involved stacking the data to identify the index with the strongest correlation with ground EC values, without the need for complex analysis or EC value ranging. In the second approach, the ground EC measurements were classified into seven different salinity categories, and the correlation coefficient was calculated between each index and each salinity category. The third approach analyzed the data temporally, considering different salinity levels. Lastly, a spatial correlation analysis was conducted between EC and index values in the fourth scenario. The initial approach revealed a weak correlation due to substantial variation in EC values. However, the SI index demonstrated the highest correlation coefficient of 0.38. In the second scenario, the S2 index exhibited the highest correlation of 0.96 for moderate salinity samples. The third scenario showed that the S1 index achieved the highest correlation value of 0.99 for moderately saline areas. In the fourth scenario, the SI index exhibited the strongest correlation among all four ponds with correlation coefficients of 0.23, 0.23, 0.18, and 0.61. Notably, the correlations observed in the second and third scenarios demonstrated higher correlation coefficients compared to both the first and fourth scenarios. The study highlighted significant variations in EC levels related to land cover types and pond elevation. Additionally, remote sensing methods detected a 48% increase in total vegetated area over 17 years, showing the potential of remote sensing techniques in salinity monitoring for expanding agriculture and improving land management.

Ecosystem physical structure, defined by the quantity and spatial distribution of biomass, influences a range of ecosystem functions. Remote sensing tools permit the non-destructive characterization of canopy and root features, potentially providing opportunities to link above- and belowground structure at fine spatial resolution in functionally meaningful ways. To test this possibility, we employed ground-based portable canopy lidar (PCL) and ground penetrating radar (GPR) along co-located transects in forested sites spanning multiple stages of ecosystem development and, consequently, of structural complexity. We examined canopy and root structural data for coherence at multiple spatial scales ≤ 10 m within each site using wavelet analysis. Forest sites varied substantially in vertical canopy and root structure, with leaf area index and root mass more evenly distributed by height and depth, respectively, as forests aged. In all sites, above- and belowground structure, characterized as mean maximum canopy height and root mass, exhibited significant coherence at a scale of 3.5-4 meters, and results suggest that the scale of coherence may increase with stand age. Our findings demonstrate that canopy and root structure are linked at characteristic spatial scales, which provides the basis to optimize scales of observation. Our study highlights the potential, and limitations, for fusing lidar and radar technologies to quantitatively couple above- and belowground ecosystem structure.

Although, promotion of urbanization culture in recent decades has made inevitable development of cities in the world, however, the development can be guided in a direction that leave, to the extent possible, minimum socioeconomic and environmental impacts. For this, it is required to first forecast auto-spreading orientation of cities and suburbs in rural areas over time and then avoid shapeless growth of cities. This paper is an attempt to develop a dynamic hybrid model based on logistic regression (LR), Markov chain (MC), and cellular automata (CA) for prediction of future urban sprawl in fast-growing cities. The model was developed using 12 widely-used urban development criteria, whose significant coefficient was determined by logistic regression, and validated by relative operating characteristic (ROC) analysis. The validated model was run in Guilan, a tourist province in northern Iran with a very high rate of urban development. For this, changes in the area of urban land use were detected over the period of 1989 to 2013 and then, future sprawl of the province was forecasted by the years 2025 and 2037. The analysis results revealed that the area of urban land use was increased by more than 1.7 % from 36012.5 ha in 1989 to 59754.8 ha in 2013, and the area of Caspian Hyrcanian forestland was reduced by 31628 ha. The results also predicted an alarming increase in the rate of urban development in the province by the years 2025 and 2037, during which urban land use is predicted to develop 0.9 % and 1.38 %, respectively. The development pattern is expected to be uneven and scattered, without following any particular direction. The development will occur close to the existing or newly-formed urban basements as well as around major roads and commercial areas. This development, if not controlled, will lead to the loss of 13863 ha of Hyrcanian forests and if the trend continues, 21013 ha of Hyrcanian forests and 20208 ha of Barren/open lands are expected to be destroyed by the year 2037. In general, the proposed model is an efficient tool for the support of urban planning decisions and facilitates the process of sustainable development of cities by providing decision-makers with an overview on future development of cities where the growth rate is very fast.

We report the relative humidity (R.H.) sensing response of a resistive sensor, employing sensing layers based on a ternary nanohybrid comprising holey carbon nanohorns (CNHox), potassium chloride (KCl), and polyvinylpyrrolidone (PVP) at mass ratios 7/1/2, 6.5/1.5/2, and 6/2/2 (w/w/w/w). The sensing structure comprises a silicon substrate, a SiO2 layer, and interdigitated transducer (IDT) electrodes. The sensing film is deposited on the sensing structure via the drop-casting method. The sensing layers' morphology and composition are investigated through Scanning Electron Microscopy (S.E.M.) and RAMAN spectroscopy. The ternary hybrid-based thin film resistance increases when the sensors are exposed to R.H., ranging from 0% to 100%. The manufactured devices show a room temperature response comparable to a commercial capacitive R.H. sensor and are characterized by excellent linearity, rapid response time, and good sensitivity. The presented sensors exhibit superior performance in terms of sensitivity compared to other similarly manufactured and tested sensors that employ a CNHox-based sensing layer. We explain the sensing role of each constituent of the ternary hybrid nanocomposites based on their chemical and physical properties, electronic properties, and affinity for water molecules. Different alternative sensing mechanisms are considered and discussed, such as the decreasing number of holes in the holey CNHox at the interaction with water molecules, proton conduction, and PVP swelling.

Microbial foodborne pathogens can cause major risks to public health and the food industry. Rapid and accurate detection of these pathogens is crucial to prevent outbreaks and ensure food safety. Traditional single-mode biosensing approaches have their limitations in terms of specificity, sensitivity, and speed. Over the years, there has been an increasing research interest in developing multimodal biosensing strategies that integrate multiple sensing techniques to achieve enhanced accuracy, efficacy, and precision in foodborne pathogen detection. This article review paper explores the current state of multimodal biosensing technologies and their promising applications in the food industry. We discuss various integrated biosensing platforms, including optical, electrochemical, and nanomaterial-based systems, among others. The advantages, challenges, and future prospects of multimodal biosensing for foodborne pathogens are thoroughly examined, highlighting the potential impact of these advanced technologies on food safety and public health.

Federated learning (FL) has become a burgeoning and attractive research area, which provides a collaborative training scheme for distributed data sources with privacy concerns. Most existing FL studies focus on taking unimodal data, such as images and text, as the model input and resolving the heterogeneity challenge, i.e., the non-identically distributed (non-IID) challenge caused by data distribution imbalance related to data labels and data amount. In real-world applications, data are usually described by multiple modalities. However, to the best of our knowledge, only a handful of studies have been proposed to improve the system performance by utilizing multimodal data. In this survey paper, we identify the significance of this emerging research topic – multimodal federated learning (MFL) and perform a literature review on the state-of-art MFL methods. Furthermore, we categorize multi-modal federated learning into congruent and incongruent multimodal federated learning based on whether all clients possess the same modal combinations. We investigate the feasible application tasks and related benchmarks for MFL. Lastly, we summarize the promising directions and fundamental challenges in this field for future research.

Iron is an essential nutrient and necessary for biological functions from DNA replication and repair to transcriptional regulation, mitochondrial respiration, electron transfer, oxygen transport, photosynthesis, enzymatic catalysis, and nitrogen fixation. However, due to iron’s propensity to generate toxic radicals which can cause damage to DNA, proteins, and lipids, multiple processes regulate uptake and distribution of iron in living systems. Understanding how intracellular iron metabolism is optimized and how iron is utilized to regulate other intracellular processes is important to our overall understanding of a multitude of biological processes. One of the tools that the cell utilizes to regulate a multitude of functions is ligation of the iron-sulfur (Fe-S) cluster cofactor. Fe-S clusters comprised of iron and inorganic sulfur are ancient components of living matter on earth that are integral for physiological function in all domains of life. Here we will explore the ways in which the cell utilizes Fe-S clusters to sense the intracellular environment and respond to maintain equilibrium.

The radar sensor described realizes healthcare monitoring capable of detecting subject chest-wall movement caused by cardiopulmonary activities, and wirelessly estimating the respiration and heartbeat rates of the subject without attaching any devices to the body. Conventional single-tone Doppler radar can only capture Doppler signatures because of a lack of bandwidth information with noncontact sensors. In contrast, we take full advantage of impulse radio ultra-wideband (IR-UWB) radar to achieve low power consumption and convenient portability, with a flexible detection range and desirable accuracy. A noise reduction method based on improved ensemble empirical mode decomposition (EEMD) and a vital sign separation method based on the continuous-wavelet transform (CWT) are proposed jointly to improve the signal-to-noise ratio (SNR) in order to acquire accurate respiration and heartbeat rates. Experimental results illustrate that respiration and heartbeat signals can be extracted accurately under different conditions. This noncontact healthcare sensor system proves the commercial feasibility and considerable accessibility of using compact IR-UWB radar for emerging biomedical applications.

Due to the rapid development of unmanned aerial vehicles (UAVs) communication technology, UAVs are gradually competing with primary users (PUs) for spectrum resources. Cognitive radio (CR) is a promising solution to meet the needs. Cooperative spectrum sensing (CSS) is considered as an effective method to detect the PU signal and identify available spectrum resources for UAVs in a cognitive UAV network (CUAVN). However, the cooperative mode among multiple UAVs may incur a high overhead, resulting in performance degradation. Therefore, we introduce a differential sequential 1 (DS1), which incorporates a differential mechanism and leverages the sequential idea based on the classical voting rule to enhance cooperative performance and efficiency. In view of this, we formulate three scenarios to characterize the PU activity and introduce a multi-slot cooperative mode within a single UAV to realize cooperative gain. Further, only the information change about the PU status is sequentially calculated in DS1, and combined with a sequential idea, the efficiency of the voting rule is greatly improved. Moreover, the application of support vector machines (SVM) in dynamic selection enables the selection of the most suitable voting rule based on diverse sensing parameters. This dynamic selection process ensures optimal performance and efficiency by adapting the voting rule to the specific characteristics of the given scenario. Finally, simulation results demonstrate that the superiority of our proposal with respect to the detection performance, sample size and the energy efficiency is evident, which proves the high performance of the proposed policy.

Timely and accurate information on tree species is crucial for the sustainable management of natural resources, forest inventory, biodiversity detection, and carbon stock calculation. The advancement of remote sensing technology and artificial intelligence has facilitated the acquisition and analysis of remote sensing data, resulting in more precise and effective classification of tree species. Multimodal remote sensing data and deep learning seem to be the current tree species classification research mainstream, whether or not. The current review on the remote sensing data and deep learning tree species classification methods perspectives to analyze the unimodal and multimodal remote sensing data and classification methods in this realm is missing. To bridge the gap, we search for major trends in the remote sensing data and tree species classification methods, provide a detailed overview of classic deep learning-based methods for tree species classification, and discuss the limitations.

In light of the critical and emerging global challenge posed by the increasing frequency and severity of heatwaves in recent decades, this study demonstrates a practical and robust workflow that bridges the gap in understanding how the effective integration of remote sensing data with ground-based observations can facilitate a comprehensive assessment of spatiotemporal heatwave patterns and trends in the central region of Thailand. This research transcends traditional methods, utilizing air temperature (Tair) and satellite-derived land surface temperature (LST) data from 1981 to 2019. Results exhibit that an analysis of Tair indicates increasing daytime heatwaves in peri-urban areas, with significant trends in heatwave number (HWN), heatwave frequency (HWF), and heatwave duration (HWD), heatwave amplitude (HWA) rises in urban settings while nighttime heatwaves significantly intensify in rural locales. Notably, LST trends reveal varied patterns, with peri-urban areas showing marked daytime increases in heatwave magnitude (HWM), HWA, and HWF. Correlation analysis (p=0.05) highlights strong daytime associations between Tair and LST in rural (HWN, HWF, HWD, r>0.90) and peri-urban (HWM, HWA, r>0.65) regions. Overall, this study advances approaches to the adaptable measurement and spatial-temporal pattern of heatwave-related risk areas, providing insights for decision-makers to develop sustainable practices and strategies for climate change adaptation.

In today’s Web 2.0 era, online social media has become an integral part of our lives. In the course of the information revolution, the form of information has undergone a radical change, from simple text information to today’s integrated video, image, text and audio, and there has also been a great change in the way of dissemination and access, as people nowadays do not just rely on traditional media to passively receive information, but more actively and selectively obtain information from social media. Therefore, it has become a great challenge for us to effectively utilize these massive and integrated multi-modal media information to form an effective system of retrieval, browsing, analysis and usage. Unlike movies and traditional long-form video content, micro-videos are usually short in length, between a few seconds and tens of seconds, which allows users to quickly browse different contents and make full use of the fragmented time in their lives, while users can also share their micro-videos to their friends or the public, forming a unique social way. Video contains rich multimodal information, and fusing information from multiple modalities in a video recommendation task can improve the accuracy of the video recommendation task.According to the micro-video recommendation task, a new combinatorial network model is proposed to combine the discrete features of each modality into the overall features of various modalities through the network, and then fuse the various modal features to obtain the overall video features, which will be used for recommendation. In order to verify the effectiveness of the algorithm proposed in this paper, experiments are conducted in the public dataset, and it is shown the effectiveness of our model.

Densely urbanized areas, with a low percentage of green vegetation, are highly exposed to Heat Waves (HW) which nowadays are increasing in terms of frequency and intensity also in the middle-latitude regions, due to ongoing Climate Change (CC). Their negative effects may combine with those of the UHI (Urban Heat Island), a local phenomenon where air temperatures in the compact built up cores of towns increase more than those in the surrounding rural areas, with significant impact on the quality of urban environment, on citizens health and energy consumption and transport, as it has occurred in the summer of 2003 on France and Italian central-northern areas. In this context this work aims at designing and developing a methodology based on aero-spatial remote sensing (EO) at medium-high resolution and most recent GIS techniques, for the extensive characterization of the urban fabric response to these climatic impacts related to the temperature within the general framework of supporting local and national strategies and policies of adaptation to CC. Due to its extension and variety of built-up typologies, the municipality of Rome was selected as test area for the methodology development and validation. First of all, we started by operating through photointerpretation of cartography at detailed scale (CTR 1: 5000) on a reference area consisting of a transect of about 5x20 km, extending from the downtown to the suburbs and including all the built-up classes of interest. The reference built-up vulnerability classes found inside the transect were then exploited as training areas to classify the entire territory of Rome municipality. To this end, the satellite EO HR (High Resolution) multispectral data, provided by the Landsat sensors were used within a on purpose developed "supervised" classification procedure, based on data mining and “object-classification” techniques. The classification results were then exploited for implementing a calibration method, based on a typical UHI temperature distribution, derived from MODIS satellite sensor LST (Land Surface Temperature) data of the summer 2003, to obtain an analytical expression of the vulnerability model, previously introduced on a semi-empirical basis.

Accurately estimating shrub biomass in arid and semi-arid regions is critical for understanding ecosystem productivity and carbon stocks at both local and global scales. However, capturing the shrub biomass accurately by satellite observations is challenging due to the short and sparse features of shrubs. In this study, we presented a framework to estimate shrub biomass at a 10m spatial resolution by integrating ground in-situ data, unmanned aerial vehicle (UAV), Sentinel-2, and Sentinel-1 observations. The pilot study was conducted in the Helan mountains of Ningxia province, China. First, the spatial distribution map of shrubland and other land cover types was generated in 2023. Then, a prediction model of shrub biomass was developed in a Random Forest Regression (RFR) approach driven by different predicted variable datasets based on in-situ measurements, UAV, and satellite images. Finally, the developed model was used to produce the biomass map of shrubland in 2023. The uncertainty was characterized by creating a standard deviation (SD) map using the Leave-One-Out Cross-Validation (LOOCV) method in the shrub biomass estimates. The resultant shrubland distribution map in 2023 has an F1 score of 0.92. The shrub biomass model driven by spectral bands and vegetation indices (R2= 0.62) demonstrated superior performance than that only driven by spectral bands (R2=0.33) or vegetation indices (R2=0.55). The uncertainty of the estimated biomass was lower than 4%, with the lowest values (<2%) happening in the high shrub coverage (>30%) and biomass production (>30kg/m2) regions. Furthermore, the findings show that the magnitude of shrub biomass was affected by the aridity and precipitation significantly in the study region. This study provides a workflow to accurately monitor the biomass of shrublands in a complicated environment based on remote sensing images from multi-platform observations.

Remote sensing datasets are increasingly being used to provide spatially explicit large scale evapotranspiration (ET) estimates. The focus of this study was to estimate and thematically map on a pixel-by-pixel basis, the actual evapotranspiration (ET_a) of the Wonji Shoa Sugarcane Estate using the Surface Energy Balance Algorithm for Land (SEBAL), Simplified Surface Energy Balance (SSEB) and Operational Simplified Surface Energy Balance (SSEBop) algorithms. The results obtained revealed that the ranges of the daily ETa estimated on January 25, February 26, September 06 and October 08, 2002 using SEBAL were 0.0 - 6.85, 0.0 – 9.36, 0.0 – 3.61, 0.0 – 6.83 mm/day; using SSEB 0.0 - 6.78, 0.0 – 7.81, 0.0 – 3.65, 0.0 – 6.46 mm/day, and SSEBop were 0.05 - 8.25, 0.0 – 8.82, 0.2 – 4.0, 0.0 – 7.40 mm/day, respectively. The Root Mean Square Error (RMSE) values between SSEB and SEBAL, SSEBop and SEBAL, and SSEB and SSEBop were 0.548, 0.548, and 0.99 for January 25, 2002; 0.739, 0.753, and 0.994 for February 26, 2002;0.847, 0.846, and 0.999 for September 06, 2002; 0.573, 0.573, and 1.00 for October 08, 2002, respectively. The standard deviation of ET_a over the sugarcane estate showed high spatio-temporal variability perhaps due to soil moisture variability and surface cover. The three algorithm results showed that well watered sugarcane fields in the mid-season growing stage of the crop had higher ET_a values compared with the other dry agricultural fields confirming that they consumptively use more water. Generally during the dry season, ET_a is limited to water surplus areas only and in wet season, ET_a was high throughout the entire sugarcane estate. The evaporation fraction (ETrF) results also followed the same pattern as the daily ET_a over the sugarcane estate. The total crop and irrigation water requirement and effective rainfall estimated using the Cropwat model were 2468.8, 2061.6 and 423.8 mm/yr for January 2001 planted and 2281.9, 1851.0 and 437.8 mm/yr for March 2001 planted sugarcanes, respectively. The mean annual ET_a estimated for the whole estate were 107 Mm³, 140 Mm³, and 178 Mm³ using SEBAL, SSEB, and SSEBop, respectively. Even though the algorithms should be validated through field observation, they have potential to be used for effective estimation of ET in the sugarcane estate.

Oncolytic virotherapy is a promising immunotherapy approach for cancer treatment, utilizing viruses to preferentially infect and eliminate cancer cells while stimulating immune responses. In this review we synthesize current literature on the molecular circuits of immune sensing and response to oncolytic virotherapy; focusing on viral DNA or RNA sensing by infected cells, cytokine and danger-associated signal sensing by neighboring cells, and subsequent downstream activation of immune pathways. The sequential sense-and-response mechanisms involve the triggering of molecular sensors by viruses or infected cells to activate transcription factors and related genes for a breadth of immune responses. We describe how molecular signals induced in the tumor upon virotherapy can trigger diverse immune signaling pathways, activating both antigen-presenting cell-based innate and T cell-based adaptive immune responses. Insights into these complex mechanisms provide valuable knowledge for enhancing oncolytic virotherapy strategies.

Sign language recognition is essential in hearing-impaired people’s communication. Sign language recognition is an important concern in computer vision and has been developed with rapid progress in image recognition technology. However, sign language recognition using a general monocular camera has problems with occlusion and recognition accuracy in sign language recognition. In this research, we aim to improve accuracy by using a 2-axis bending sensor as an aid in addition to image recognition. We aim to achieve higher recognition accuracy by acquiring hand keypoint information of sign language actions captured by a monocular RGB camera and adding sensor assist. To improve sign language recognition, we need to propose new AI models. In addition, the amount of dataset is small because it uses the original data set of our laboratory. To learn using sensor data and image data, we used MediaPipe, CNN, and BiLSTM to perform sign language recognition. MediaPipe is a method for estimating the skeleton of the hand and face provided by Google. In addition, CNN is a method that can learn spatial information, and BiLSTM can learn time series data. Combining the CNN and BiLSTM methods yields higher recognition accuracy. We will use these techniques to learn hand skeletal information and sensor data. Additionally, the 2-axis Bending sensor glove data support training AI model. Using these methods, we aim to improve the recognition accuracy of sign language recognition by combining sensor data and hand skeleton data. Our method performed better than using skeletal information, achieving 96.5% accuracy in Top-1.

According to the World Health Organization (WHO), dementia is the seventh leading reason of death among all illnesses and one of the leading causes of disability among the world’s elderly people. Day by day the number of Alzheimer’s patients is raising. Considering the increasing rate and the dangers, Alzheimer’s disease should be diagnosed carefully. Machine learning is a potential technique for Alzheimer’s diagnosis but general users do not trust machine learning models due to the black-box nature. Even, some of those models do not provide the best performance because of using only neuroimaging data. To solve these issues, this paper proposes a novel explainable Alzheimer’s disease prediction model using a multimodal dataset. This approach performs a data-level fusion using clinical data, Freesurfer MRI segmentation data, and psychological data. For Alzheimer’s disease vs cognitively normal prediction, the random forest classifier provides 100% accuracy. Furthermore, Alzheimer’s disease and non-Alzheimer’s dementia should be classified properly because their symptoms are similar. To the best of our knowledge, we are the first to present a three-class classification on Alzheimer’s disease vs cognitively normal vs non-Alzheimer’s dementia and achieved 99.86% accuracy using an ensemble model. Besides, a novel Alzheimer’s patient management architecture is also proposed in this work..

Assessment of extreme events and climate change on reservoir inflow is important for water and power stressed countries. Projected climate is subject to uncertainties related to climate change scenarios and Global Circulation Models (GCMs’). Extreme climatic events will increase with the rise in temperature as mentioned in the AR5 of the IPCC. This paper discusses the consequences of climate change that include extreme events on discharge. Historical climatic and gauging data were collected from different stations within a watershed. The observed flow data was used for calibration and validation of SWAT model. Downscaling was performed on future GCMs’ temperature and precipitation data, and plausible extreme events were generated. Corrected climatic data was applied to project the influence of climate change. Results showed a large uncertainty in discharge using different GCMs’ and different emissions scenarios. The annual tendency of the GCMs’ is bi-vocal: six GCMs’ projected a rise in annual flow, while one GCM projected a decrease in flow. The change in average seasonal flow is more as compared to annual variations. Changes in winter and spring discharge are mostly positive, even with the decrease in precipitation. The changes in flows are generally negative for summer and autumn due to early snowmelt from an increase in temperature. The change in average seasonal flows under RCPs’ 4.5 and 8.5 are projected to vary from -29.1 to 130.7% and -49.4 to 171%, respectively. In the medium range (RCP 4.5) impact scenario, the uncertainty range of average runoff is relatively low. While in the high range (RCP 8.5) impact scenario, this range is significantly larger. RCP 8.5 covered a wide range of uncertainties, while RCP 4.5 covered a short range of possibilities. These outcomes suggest that it is important to consider the influence of climate change on water resources to frame appropriate guidelines for planning and management.

In an era where dietary habits significantly impact health, technological interventions can offer personalized and accessible food choices. This paper introduces Chef Dalle, a recipe recommen-dation system that leverages multimodal human-computer interaction (HCI) techniques to pro-vide personalized cooking guidance. The application integrates voice-to-text conversion via Whisper, ingredient image recognition through GPT-Vision, and employs TF-IDF vectorization alongside cosine similarity for personalized recipe recommendations. These methods enable users to interact with the system using voice, text, or images, accommodating various dietary re-strictions and preferences. Furthermore, the utilization of DALL-E 3 for generating recipe images enhances user engagement. User feedback mechanisms allow for the refinement of future rec-ommendations, demonstrating the system's adaptability. Chef Dalle showcases potential appli-cations ranging from home kitchens to grocery stores and restaurant menu customization, ad-dresses accessibility, promoting healthier eating habits. This paper underscores the significance of multimodal HCI in enhancing culinary experiences, setting a precedent for future developments in the field.

This study compares the performance of existing studies on multimodal emotion recognition, and proposes a model that fuses two modalities with the speaker's text and voice signals as input values and detects depression. Based on the DAIC-WOZ dataset, voice features were extracted using CNN, text features were extracted using Transformers, and two modalities were fused through a tensor fusion network. We also build a model to detect whether the speaker is depressed or not using LSTM in the final layer. This study suggests the possibility of increasing access to mental illness diagnosis by enabling patients to detect depression on their own in daily conversations. If the model proposed in this study is developed and the voice conversation system is connected, it will be easier for patients who cannot visit the hospital periodically or who are reluctant to visit the hospital to check their condition and seek recovery. Furthermore, it can be expanded to multi-label classification for various mental diseases and used as a simple self-mental disease diagnosis tool.

Soil organic matter (SOM) is important for the global carbon cycle, and hyperspectral remote sensing has proven a promising method for fast SOM content estimation. However, soil physical properties significantly affect the sensitivity of satellite hyperspectral imaging to SOM, leading to poor generalization ability of the estimation model. This study aims to improve the spatiotemporal transferability of the SOM prediction model by alleviating the coupling effect of soil physical properties on the spectra. Based on satellite hyperspectral images and soil physical variables, including soil moisture (SM), soil surface roughness (root mean squared height, RMSH), and soil bulk weight (SBW), a soil spectral correction strategy was established based on the information unmixing method. Two important grain-producing areas in Northeast China were selected as study areas to verify the performance and transferability of the spectral correction model and SOM content prediction model. The results showed that soil spectral corrections based on fourth-order polynomials and the XG-Boost algorithm had excellent accuracy and generalization ability, with residual predictive deviations (RPD) exceeding 1.4 in almost all bands. In addition, when the soil spectral correction strategy was adopted, the accuracy of the SOM prediction model and the generalization ability after model migration were significantly improved. The SOM prediction accuracy based on the XG-Boost corrected spectrum was the highest, with a coefficient of determination (R2) of 0.76, root mean square error (RMSE) of 5.74 g/kg, and RPD of 1.68. The prediction accuracy, R2, RMSE, and RPD of the model after migration were 0.72, 6.71 g/kg, and 1.53, respectively. Compared with the direct migration prediction of the model, adopting the soil spectral correction strategy based on fourth-order polynomials and XG-Boost reduced the RMSE of the SOM prediction results by 57.90% and 60.27%, respectively. The performance comparison highlighted the advantages of considering soil physical properties in regional-scale SOM prediction.

Purpose: National Institute for Health and Care Excellence (NICE) has recently published quality standards for assessment of fall risk and preventing further falls. According to the standards, multifactorial fall risk assessments should include: identification of falls history; analysis of gait, balance, mobility and muscle strength, among other factors. Despite being based on subjective analysis or simple timing and not being multifactorial, physiotherapists and physicians quite often use these tests as reference scales to differentiate between lower and higher risk of falling. Instrumented TUG has been recently reported to provide important additional information to the overall score. Objective: To explore a case-based approach of fall risk assessment to identify the most relevant and informative risk factors that in combination could better define a person risk profile. Materials and Methods: A multifactorial assessment of fall risk through questionnaires, standard functional tests, tests instrumented with inertial sensors, and force platforms has been studied within a group aged 55-80 years old. Different fall risk factors and fall risk assessment methods were analyzed in a case-based descriptive study. Results & Discussion: Subjects at higher risk of falling were identified based on their detailed profiles. A set of features were obtained from the instrumented standard tests differing significantly between subjects presenting higher or lower fall risk. Therefore, instrumenting conventional tests with wearables containing inertial sensors and force platforms gives more detailed and quantitative insights. This information can be used to better define and tailor fall prevention exercises and to improve the follow-up of the evolution of the subject.

Neurodegenerative diseases, such as Alzheimer’s disease and Parkinson’s disease, are caused by a combination of multiple events that damage neuronal function. A well-characterized biomarker of neurodegeneration is proteinaceous aggregates accumulation in the brain. However, the gradually worsening symptoms of neurodegenerative diseases are unlikely to be solely due to the result of a mutation in a single gene, but rather a multi-step process involving epigenetic changes. Multiple epigenetic studies have revealed that modification of histones accompanied by both the local and global remodeling of the chromatin structure and alternations in transcriptional patterns are closely associated with the pathogenesis of neurodegenerative diseases. Unlike DNA mutations, epigenetic alterations are reversible, and therefore raise the possibilities for therapeutic intervention including dietary modifications. Additionally, reactive oxygen species may contribute to the pathogenesis of Alzheimer’s disease and Parkinson’s disease. Given that the antioxidant properties of plant-derived phytochemicals are likely to exhibit pleiotropic effects against ROS-mediated epigenetic alternation, dietary intervention may be promising for the management of neurodegeneration in these diseases. In this review, the state-of-the-art applications using single-cell multi-modal omics approaches including genetics, and epigenetics, and dietary approaches for the identification of novel biomarkers and therapeutic approaches for the treatment of neurodegenerative diseases are discussed.

Multimodal user interfaces promise a natural and intuitive human machine interactions. But is the extra effort for the development of a complex multi-sensor system justified, or can users also be satisfied with one input modality already? This study investigates interactions in an industrial weld inspection workstation. Three unimodal interfaces, including spatial interaction with buttons augmented on a workpiece or a worktable, and speech commands, were tested individually and in a multimodal combination. Within the unimodal conditions, users preferred the augmented worktable, but overall, the interindividual usage of all input technologies in the multimodal condition was ranked best. Our findings indicate that the implementation and use of multiple input modalities is valuable, and that it is difficult to predict the usability of individual input modalities for complex systems.

Considering the spectrum shortage problem of IoT devices, we introduce a collaborative spectrum sensing (CSS) framework in this paper to identify available spectrum resources, so that IoT devices can access it and meanwhile avoid causing harmful interference to the normal communication of the primary user (PU). However, in the process of the PU’s signal detection in IoT devices, the issue about the stopping time and decision cost arises. To this end, we propose a distributed cognitive IoT model, which includes two IoT devices independently using sequential decision rules to detect the PU. On this basis, we define the stopping time and cost function for IoT devices, and formulate an average cost optimization problem in CSS. To solve this problem, we further regard the optimal stopping time problem as a finite horizon problem, and solve the threshold of the optimal decision rule by dynamic programming. At last, numerical simulation results demonstrate the correctness of our proposal in terms of the global false alarm and miss detection probability, and it always achieves minimal average cost under various cost of each observation taken and thresholds.

The advent of the era of big data will bring more convenience to people and greater development to society. But at the same time, it will also bring people the problem of 'information overload', i.e., when people are faced with huge information data, there are many redundant and worthless data. The redundant and worthless data information seriously interferes with the accurate selection of information data. Even though people can use Internet search engines to access information data, they cannot meet the individual needs of specific users in specific contexts. The personalized needs of a particular user in a particular context. Therefore, how to find useful and valuable information quickly has become one of the key issues in the development of big data. With the advent of the era of big data, recommendation systems, as an important technology to alleviate information overload, have been widely used in the field of e-commerce. Recommender systems suffer from a key problem: data sparsity. The sparsity of user history rating data causes insufficient training of collaborative filtering recommendation models, which leads to a significant decrease in the accuracy of recommendations. In fact, traditional recommendation systems tend to focus on scoring information and ignore the contextual context in which users interact. There are various contextual modal information in people's real life, which also plays an important role in the recommendation process. In this paper we achieve data degradation and feature extraction, solving the problem of sparse data in the recommendation process. An interaction context-aware sub-model is constructed based on a tensor decomposition model with interaction context information to model the specific influence of interaction context in the recommendation process. Then an attribute context-aware sub-model is constructed based on the matrix decomposition model and using attribute context information to model the influence of user attribute contexts and item attribute contexts on recommendations. In the process of building the model, the method not only utilizes the explicit feedback rating information of users in the original dataset, but also utilizes the interaction context and attribute context information of the implicit feedback and the unlabeled rating data. We evaluate our model by extensive experiments. The results illustrate the effectiveness of our recommender model.

Hateful and abusive speech presents a major challenge for all online social media platforms. Recent advances in Natural Language Processing and Natural Language Understanding allow more accurate detection of hate speech in textual streams. This study presents a multimodal approach to hate speech detection by combining Computer Vision and Natural Language processing models for abusive context detection. Our study focuses on Twitter messages and, more specifically, on hateful, xenophobic and racist speech in Greek aimed at refugees and migrants. In our approach we combine transfer learning and fine-tuning of Bidirectional Encoder Representations from Transformers (BERT) and Residual Neural Networks (Resnet). Our contribution includes the development of a new dataset for hate speech classification, consisting of tweet ids, along with the code to obtain their visual appearance, as they would have been rendered in a web browser. We have also released a pre-trained Language Model trained on Greek tweets, which has been used in our experiments. We report a consistently high level of accuracy (accuracy score=0.970, f1-score=0.947 in our best model) in racist and xenophobic speech detection.

Globally, there is massive uptake and explosion of data and challenge is to address issues like scale, pace, velocity, variety, volume and complexity of this big data. Considering the recent epidemic in China, modeling of COVID-19 epidemic for cumulative number of infected cases using data available in early phase was big challenge. Being COVID-19 pandemic during very short time span, it is very important to analyze the trend of these spread and infected cases. This chapter presents medical perspective of COVID-19 towards epidemiological triad and the study of state-of-the-art. The main aim this chapter is to present different predictive analytics techniques available for trend analysis, different models and algorithms and their comparison. Finally, this chapter concludes with the prediction of COVID-19 using Prophet algorithm indicating more faster spread in short term. These predictions will be useful to government and healthcare communities to initiate appropriate measures to control this outbreak in time.

Affordances are an important concept in cognition, which can be applied to robots in order to perform a successful human-robot interaction (HRI). In this paper we explore and discuss the idea of emotional affordances and propose a viable model for implementation into HRI. We consider “2-ways” affordances: perceived object triggering an emotion, and perceived human emotion expression triggering an action. In order to make the implementation generic, the proposed model includes a library that can be customised depending on the specific robot and application’s scenario. We present the AAA (Affordance-Appraisal-Arousal) model, which incorporates Plutchik’s Wheel of Emotions, and show some examples of simulation and possible scenarios.

Semantic segmentation of high-resolution remote sensing images provides detailed and precise feature information, playing a pivotal role in decision-making and analysis in sectors such as water management, agriculture, military, and environmental protection. However, most methods merely combine features from various branches directly, without a mechanism for spatial location feature screening, and treat all extracted features as equally important. To overcome these limitations, we introduce a novel spatially adaptive interaction network (SAINet) for dynamic interaction across different features in remote sensing semantic segmentation task. Specifically, we propose a spatial refined module that leverages local context information to filter spatial locations and extract salient regions. Following this, we introduce an innovative spatial interaction module that uses a spatial adaptive modulation mechanism to dynamically select and allocate spatial position weights. This facilitates interaction between local salient areas and global information, allowing the network to focus more on relevant regions. This flexible approach enables the network to capture more informative features, thereby enhancing segmentation accuracy. Experiments conducted on the DeepGlobe, Vaihingen, and Potsdam public datasets confirm the effectiveness and capability of SAINet. Our code and models will be publicly available

As climate changes, our daily life has been much influenced by abnormal meteorological phenomena such as heavy rainfall, heat wave, heavy snowfall, and fine dust. Atmospheric air quality is worsening day by day and indoor air quality is also affected by interconnected daily activities throughout the inside and outside of buildings and houses. Nowadays, pollutants from various sources are emitted, transformed by sunlight, vapor, and ozone and transported into the city from country to country. Due to these reasons, there have been high demands to monitor the transportation of particulate matters and improve air quality. Monitoring of pollutants and identification of type and its concentration enables us to track and control its generation and consequently find out the solution. However, monitoring of pollutants, especially, particulate matter generation and its transportation is still not fully operated in atmospheric air due to its open nature and meteorological factors. Even though indoor air is relatively easy to monitor and control than outdoor in the aspect of specific volume and contaminant source, but it still needs to consider the meteorological parameters because indoor air is not fully separated from the outdoor air flow and contaminants transportation. In this study, optical approach using spectral sensor was attempted to reveal the feasibility of wavelength and chromaticity values of reflected light from specific particles. From the analysis of reflected light of various particulate matters according to different liquid additives, parameter studies were performed to investigate which experimental conditions can contribute to the enhanced selective sensing of particulate matters. Five different particulate matters such as household dust, soil, talc powder, gypsum powder and yellow pine tree pollen were utilized and observed to elucidate the relationship between property of particulate matter and detected light spectrum. Applicable approaches to assist current particle matter sensors and improve the selective sensing were suggested.

This paper presents a study about the integration of Piezoelectric Energy Harvesting Systems (PE-EHS) in building envelopes for powering Fiber Bragg Grating (FBG) sensors, enabling efficient and low-consumption monitoring with the objective to leverage Structural Health Monitoring (SHM). The research includes preliminary tests conducted in a real environment to validate the PE-EHS when fully integrated into the façade, capturing mechanical vibrations generated mainly by wind loads. Based on these activities, the final configuration of PE-EHS is defined to provide a complete system for façade monitoring. This integrated system includes the Piezoelectric Generator (PEG), Supercapacitor (SC), Power Conditioner Circuit (PCC), Fibre Optic Sensing (FOS) Interrogator, and the IoT Gateway trans-mitting measurement data within an Internet of Things (IoT) monitoring platform. This configuration is tailored to address the challenges related to the structural integrity of building envelopes. Results demonstrate a potential for a stand-alone solution in the façade sector, but, raising issues for certain limitations, requiring further investigation. In particular, the study emphasizes constraints related to the energy production of PE-EHS for façade integration. It highlights the necessity to carefully consider these limitations within the broader context of their applicability, providing insights for the informed deployment of Piezoelectric Energy Harvesting technology in building envelope monitoring.

Understanding and preventing the “great obstetrical syndromes” has been the focus of pregnancy research for decades. The combined global impact from these syndromes on maternal and perinatal morbidity and mortality is compounded by increased long-term risk of adverse health effects following pregnancy. Accumulating evidence indicates that conditions such as pre-eclampsia, fetal growth restriction, spontaneous preterm labour, and stillbirth, are syndromes with etiologic heterogeneity that share common pathophysiologic pathways involving placental dysfunction. Recently, the importance of the interaction between fetal and maternal genomes has been extended to include the epigenetic impact of pre-existing maternal pathophysiology on placental development and highlights the importance of nutritional and environmental factors in altered placentation. Validated first-trimester screening algorithms for preeclampsia can identify high-risk women and prophylactic treatment with low-dose aspirin can be initiated. High-risk women can be monitored during the second trimester using clinical assessment, ultrasound, and biochemical markers, that are able to predict complications prior to the development of clinical signs and symptoms. The integration of lifestyle interventions, pregnancy screening, and medical management represents a conceptual advance in pregnancy care. This approach has the potential to significantly reduce pregnancy complications, maternal and perinatal morbidity and mortality, adverse health outcomes later in life, and health-care costs.

Emotion detection holds significant importance in facilitating human-computer interaction, enhancing the depth of engagement. By integrating this capability, we pave the way for forthcoming AI technologies to possess a blend of cognitive and emotional understanding, bridging the divide between machine functionality and human emotional complexity. This progress has the potential to reshape how machines perceive and respond to human emotions, ushering in an era of empathetic and intuitive artificial systems. This paper introduces a novel approach to multimodal emotion recognition, seamlessly integrating speech and text modalities to accurately infer emotional states. Employing CNNs, we meticulously analyze speech using Mel spectrograms, while a BERT-based model processes the textual component, leveraging its bidirectional layers for profound semantic comprehension. The outputs from both modalities are combined using an attention-based fusion mechanism that optimally weighs their contributions. The proposed method undergoes meticulous testing on two distinct datasets: Carnegie Mellon University’s Multimodal Opinion Sentiment and Emotion Intensity (CMU-MOSEI) dataset and the Multimodal Emotion Lines Dataset (MELD). The results demonstrate superior efficacy compared to existing frameworks, achieving an accuracy of 88.4% and an F1-score of 87.9% on the CMU-MOSEI dataset, and a notable weighted accuracy (WA) of 67.81% and a weighted F1 (WF1) score of 66.32% on the MELD dataset. This comprehensive system offers precise emotion detection and introduces several significant advancements in the field.

The study combines two critical aspects of climate research in Nigeria. The first segment focuses on improving predictive performance of global circulation models (GCMs) employing ensemble techniques. ANN ensemble displayed significant improvement in the accuracy of climate projections, particularly in high-altitude geographic areas where traditional models exhibit limitations. Performance evaluation metrics such as Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), and Nash–Sutcliffe Efficiency (NSE) demonstrate the effectiveness of the ANN multimodal ensemble. Building on the improved climate modelling, the second segment delves into a spatial and temporal analysis of rainfall and mean temperature patterns under climate change in Nigeria. By considering two Representative Concentration Pathways (RCPs), namely RCP4.5 and RCP8.5, the study utilizes dynamically downscaled GCMs obtained from the Coupled Model Intercomparison Project phase 5 (CMIP5). The Mann-Kendall test reveals significant spatiotemporal variations in climate trends across the country. The warming trend is particularly pronounced in the northern regions, with RCP8.5 showing more severe impacts compared to RCP4.5. This integrated approach not only highlights the importance of multimodal ensembles in climate modelling but also provides valuable insights into the spatial and temporal patterns of climate change to inform effective adaptation and mitigation strategies for Nigeria.

The collaboration of multimodal transport documents is the foundation for conducting multimodal transport operations. Blockchain technology can effectively address trust issues and information sharing difficulties in current multimodal transport document collaboration. However, in current blockchain-based electronic document research, the bottleneck problem lies in the collaboration of multiparty in multimodal transport under the "one-bill coverage system". We design a blockchain platform architecture for multimodal transport, reshaping collaboration processes for different modes of transport based on the blockchain platform. It proposes a collaboration model for multimodal transport under the "one-bill coverage system" and performs model solving and case analysis. The obtained collaboration strategies are embedded into the platform's application layer. Finally, using Solidity language, it writes smart contracts related to multimodal transport under the "one-bill coverage system " and deploys and executes them on the Remix platform.

Syringomyelia can be associated with multiple etiologies. The treatment of the underlying causes is the first-line therapy, however, a direct approach to the syrinx is accepted as rescue treatment. Any direct intervention on the syrinx requires a myelotomy, posing a significant risk of iatrogenic spinal cord (SC) injury. Intraoperative neurophysiological monitoring (IONM) is crucial to detect and prevent surgically induced damage in neural SC pathways. We retrospectively reviewed the perioperative and intraoperative neurophysiological data and perioperative neurological examinations in ten cases of syringomyelia surgery. All monitored modalities remained stable throughout the surgery in six cases, correlating with no new postoperative neurological deficits. In two patients, significant transitory attenuation, or loss of motor evoked potentials (MEPs), were observed and recovered after a corrective surgical maneuver, with no new postoperative deficit. In two cases, a significant MEP decrement was noted, which lasted until the end of the surgery and was associated with postoperative weakness. A transitory train of neurotonic electromyography (EMG) discharges was reported in one case, the surgical plan was adjusted and the patient showed no postoperative deficit. The dorsal nerve roots were stimulated and identified in the seven cases where the myelotomy was performed via the dorsal root entry zone. Dorsal column mapping guided the myelotomy entry zone in four cases. In conclusion, multimodal IONM is feasible and reliable and may help prevent iatrogenic SC injury during syringomyelia surgery.

Document classification is one of the most critical steps in the document analysis pipeline. There are two types of approaches for document classification, known as image-based and multimodal approaches. The image-based document classification approaches are solely based on the inherent visual cues of the document images. In contrast, the multimodal approach co-learns the visual and textual features, and it has proved to be more effective. Nonetheless, these approaches require a huge amount of data. This paper presents a novel approach for document classification that works with a small amount of data and outperforms other approaches. The proposed approach incorporates a hierarchical attention network(HAN) for the textual stream and the EfficientNet-B0 for the image stream. The hierarchical attention network in the textual stream uses the dynamic word embedding through fine-tuned BERT. HAN incorporates both the word level and sentence level features. While the earlier approaches rely on training on a large corpus (RVL-CDIP), we show that our approach works with a small amount of data (Tobacco-3482). To this end, we trained the neural network at Tobacco-3428 from scratch. Thereby, we outperform state-of-the-art by obtaining an accuracy of 90.3%. This results in a relative error reduction rate of 7.9%.

This paper is an interdisciplinary study of novel applications of techniques and tools of an area of brain science, known as Synesthesia (involving associations and/or confusion between distinct senses), to area of Computer Science known as Immersive Virtual Reality (VR), that makes the subject’s awareness of physical self be diminished by being surrounded in an engrossing artificial environment. Natural Synesthesia has for the last decade been an important emerging area in brain science but is present in only a small proportion of the population. For example a person with Natural Synesthesia, when viewing a grapheme, may perceive a color additionally to be associated to the grapheme. In contrast, Artificial synesthesia (also known as virtual synesthesia or synthetic synesthesia) has been defined as the sensory joining due a cross-modal mapping device, where information of one sense is accompanied by an induced perception in another sense. In particular, we propose use of a multimodal manner of displaying information in VR to increase and concentrate attention. Artificial Synesthesia to synthetically create induced associations between senses, allowing Artificial Synesthesia to be experienced by anyone using a VR system. The paper describes the enhancement of immersive VR by use of Artificial Synesthesia to improve the system’s performance at steering and directing the attention of the user. We describe techniques for an enhanced immersive VR that displays associations between a variety of senses: between colors and characters, also between colors and sounds, and between sounds and the position of tactile sensations. The sense association provided by Artificial Synesthesia allows the system to better capture the user’s attention and better direct that attention. A major application of our work in VR-induced Artificial Synesthesia is to provide an enhanced methodology for controlling the attention of the subject, and to improve the direction of attention of subjects undergoing guided imagery therapies for pain relief. Other potential high-impact applications include improved immersive VR, more programmable human/computer interfaces and other medical therapies.

Hyperspectral imaging data holds great potential for the different stages of the mining life cycle in active and post-mining environments. However, the technology has yet to reach the stage of large-scale industrial implementation and acceptance. While hyperspectral satellite imagery can achieve high spectral resolution, signal-to-noise ratio (SNR) and global availability with break-through satellite systems like EnMAP, EMIT and PRISMA, limited spatial resolution poses chal-lenges for sectors like mining, which require decimetre to centimetre scale resolution for applica-tions such as reconciliation, ore/waste estimates, geotechnical assessments and environmental monitoring. Hyperspectral imaging from drones (referred to herein as Uncrewed Aerial Systems; UASs) offers high spatial resolution data relevant to the camp/ mine scale, with the capability for frequent, user-defined re-visit times. This has been made possible by the miniaturization of hy-perspectral imaging systems. Collection of data in the visible to near and shortwave infrared (VNIR-SWIR) wavelength regions enables the detection of different minerals and surface altera-tion patterns potentially revealing crucial information for exploration, extraction, re-mining, waste remediation, and rehabilitation. In this paper, we provide a review of relevant studies de-ploying hyperspectral imaging in or applicable to the mining sector, especially for the use of hy-perspectral VNIR-SWIR Uncrewed Aerial Systems. Where required, we draw on previous in-sights derived from satellite or ground-based systems. We also discuss UAS survey planning, and sampling considerations for validation and interpretation.

This study was performed to synthesize multimodal radiopharmaceutical designed for the diagnosis and treatment of prostate cancer. To achieve this goal, Super Paramagnetic Iron Oxide (SPIO) nanoparticles were used as a platform for targeting molecule (PSMA-617) and for complexation of two scandium radionuclides, 44Sc for PET imaging and 47Sc for radionuclide therapy. TEM and XPS images showed that the Fe3O4 NPs have a uniform cubic shape and a diameter from 38 to 50 nm. The Fe3O4 core are surrounded by SiO2 and an organic layer. The saturation magnetization of the SPION core was 60 emu/g. However, coating the SPIONs with silica and polyglycerol reduces the magnetization significantly. The obtained bioconjugates were labeled with 44Sc and 47Sc, with a yield higher than 97%. The radiobioconjugate exhibited high affinity and cytotoxicity towards the human prostate cancer LNCaP (PSMA+) cell line, much higher than for PC-3 (PSMA-) cells. High cytotoxicity of the radiobioconjugate was confirmed by radiotoxicity studies on LNCaP 3D-spheroids. In addition, the magnetic properties of the radiobioconjugate should allow for its use in guide drug delivery driven by magnetic field gradient.

This paper summarizes over a decade of collaborative eco-archaeological research along the central coast of California involving researchers from the University of California, Berkeley, tribal citizens from the Amah Mutsun Tribal Band, and California Department of Parks and Recreation archaeologists. Our research employs remote sensing methods to document and assess cultural resources threatened by coastal erosion and geophysical methods to identify archaeological deposits, minimize impacts on sensitive cultural resources, and provide tribal and state collaborators with a suite of data to consider before proceeding with any form of invasive archaeological excavation. Our case study of recent eco-archaeological research developed to define the historical biogeography of threatened and endangered anadromous salmonids demonstrates how remote sensing technologies help identify dense archaeological deposits, remove barriers, and create bridges through equitable and inclusive research practices between archaeologists and the Amah Mutsun Tribal Band. These experiences have resulted in the incorporation of remote sensing techniques as a central approach of the Amah Mutsun Tribal Band when conducting archaeology in their traditional territories.

There have been multiple technological advancements that promise to gradually enable devices to measure and record signals with high resolution and accuracy in the domain of Brain Computer Interfaces (BCI). This review paper provides a comprehensive overview of fNIRS, EEG, and fMRI, including their principles, advantages, limitations, and integration with other modalities for advanced BCIs. The paper highlights the advantages of integrating multiple modalities, such as increased accuracy and reliability, and discusses the challenges and limitations of multimodal integration. Applications of advanced BCIs in clinical, assistive, gaming, and entertainment domains are also discussed. Furthermore, we explain the data acquisition approaches for each of these modalities. We also demonstrate various software programs that help in extracting, cleaning, and refining the data. We conclude this paper with discussion on the available literature on data acquisition approaches and a comparison of the ease of use for each of the modalities.

Along with the popularity of the Internet, people are exposed to more and more ways of micro-videos, and a huge amount of micro-video data has emerged. micro-videos have gradually become the Internet content preferred by the public, and a large number of micro-video apps have also emerged, such as Tiktok and Kwai. Intelligent classification and mining of micro-videos can greatly enhance user experience, improve business operation efficiency and enhance user experience. Through deep intelligent analysis and mining of micro-videos, important information in micro-videos can be extracted to provide an important basis for beautifying videos, content appreciation, video recommendation, content search, etc. In the past, content understanding for short videos often used human work annotation, but in recent years, with the great success of deep convolutional neural networks in image recognition, short video content understanding based on this method has gradually developed. Nowadays, most recognition algorithms extract the feature representation of each frame independently and then fuse them. However, while extracting the feature representation, some low-level semantic features are lost, which makes the algorithm unable to accurately distinguish the category of the video. At present, the algorithm of micro-video recognition based on deep learning has surpassed the iDT algorithm, making these traditional methods fade out of people’s view. In this paper according to the micro-video classification task, a new network model is proposed to concatenate features of each modality into the overall features of various modalities through the network, and then fuse the various modal features with the attention mechanism to obtain the whole micro-video features, which will be used for classification. In order to verify the effectiveness of the algorithm proposed in this paper, experiments are conducted in the public dataset, and it is shown the effectiveness of our model.

With the advent of modern technologies, the healthcare industry is moving towards a more Personalised smart care model. The enablers of such care models are the Internet of Things (IoT) and Artificial Intelligence. These technologies collect and analyse data from persons in care to alert relevant parties if any anomaly is detected in a patient’s regular pattern. However, such reliance on IoT devices to capture continuous data extends the attack surfaces and demands high-security measures. Both patients and devices need to be authenticated to mitigate a large number of attack vectors. The biometric authentication method has been seen as a promising technique in these scenarios. To this end, this paper proposes an AI-based multimodal biometric authentication model for single and group-based users’ device-level authentication that increases protection against the traditional single modal approach. To test the efficacy of the proposed model, a series of AI models are trained and tested using physiological biometric features such as ECG (Electrocardiogram) and PPG (Photoplethysmography) signals from five publicly available datasets from Physionet and Mendeley data repositories. The multimodal fusion authentication model shows promising results with 99.8% accuracy and an Equal Error Rate (EER) of 0.16.

Metabolomic analysis of biological fluids and tissues has become an increasingly routine tool in the biological toolbox. However, challenges remain to be overcome, including developing strategies to maximise coverage of the metabolome without requiring large sample volumes. Here we describe a multimodal strategy that combines data using both LC-MS and GC-MS from a unique vial with a sample of plasma (20µl) or a sample of brain tissue (3mg). Using a split phase extraction the non-aqueous phase was analyzed by reversed phase (RP) LC-MS, whilst the aqueous phase was analyzed using hydrophilic liquid interaction chromatography (HILIC)LC-MS, with both phases also analysed using GC-MS after derivatization of the extract. Analytical performance was assessed in 7 rat cerebellum samples and a pilot study of 40 plasma samples (20 vs. 20: AD vs. healthy controls). The method, which uses four hours of instrument time, measured 20,707 metabolite features in brain samples and 17,266 in plasma samples, from those 44.1% features displayed CV’s below 15% and 75.2% below 30%. The method has potential to resolve subtle biological differences and to correlate metabolite composition directly to clinical outcomes including MMSE, age and ADCS-ADL. This method can acquire in the order of 20K metabolic features when low volumes are available.

The hyper-arid environment is characterized by high inter-annual climatic fluctuations. The yearly average rainfall can change substantially for several years, forming wet or dry sub-periods. Observing rainfall trends over a sub-period can lead to a false perception of a change in the trend but may in fact represent a periodic cycle when examined on a larger time scale. We aimed to better characterize the rainfall regime prevailing in the hyper-arid Arava Valley (Israel/Jordan), and to examine the response of vegetation to annual rainfall. We hypothesized that annual and perennial vegetation would respond differently to wet and dry sub-periods, and that grazing activities will impact vegetation growth. We used a time series of monthly rainfall, from which we calculated Standard Precipitation Index (SPI), and calculated proxies of perennial and annual vegetation over the last four decades using Landsat-derived Normalized Difference Vegetation Index (NDVI). We found no clear trend in rainfall amounts during this period, however we did identify wet and dry sub-periods which were statistically distinct in rain and in vegetation patterns from each other. The highest levels of correlation between rainfall and the NDVI derived proxies were found when examining average rainfall over a period of two- three years for the annual vegetation and over four years for perennial vegetation. Using the Mann-Kendall test, we identified a time lag of two to four years, with the proxies of annual vegetation responding faster than the proxies of perennial vegetation, to shifts between wet and dry sub-period. In addition, we found a consistent difference between natural vegetation cover in Jordan (grazed) and Israel (non-grazed), favoring the development of natural vegetation on the Israeli side. We conclude that integrating between long-term remote sensing satellite imagery and climatic records revealed the greater resilience of perennial vegetation in the hyper-arid region to climatic fluctuation, and enabled us to identify the vegetation’s sensitivity to anthropogenic impact.

This paper comprehensively reviews sensors and sensing devices developed or/and proposed so far utilizing two smart materials: electrorheological fluids (ERFs) and magnetorheological materials (MRMs) whose rheological characteristics such as stiffness and damping can be controlled by external stimuli, an electrical voltage for ERFs and a magnetic field for MRMs, respectively. In this review article, the MRMs are classified into magnetorheological fluid (MRF), magnetorheological elastomer (MRE) and magnetorheological plastomer (MRP). To easily understand the history of sensing research using the two smart materials, the order of this review article is organized in a chronological manner of ERF sensors, MRF sensors, MRE sensors and MRP sensors. Among many sensors fabricated from each smart material, one or two sensors or sensing devices are adopted to discuss on the sensing configuration, working principle and specifications such as accuracy and sensitivity. Some sensors adopted in this article include force sensor, tactile device, strain sensor, wearable bending sensor, magnetometer, display device and flux measurement sensor. After briefly describing what have been reviewed in a conclusion, several challenging future works, which should be done for practical applications as sensors or sensing devices, are remarked in terms of new technologies such as artificial intelligence neural network in which several parameters affecting the sensor signals can be precisely tuned or controlled. It is sure that this review article is very helpful to make creative sensors using not only the proposed smart materials but also several different types of smart materials including shape memory alloys and active polymers.

Chiroptical responses have been an essential tool over the last decades for chemical structural elucidation due to their exceptional sensitivity to geometry and intermolecular interactions. In recent times, there has been an increasing interest for the search of more efficient sensing by the rational design of tailored chiroptical systems. In this Review article, advances on chiroptical systems towards their implementation in sensing applications are summarized. Strategies to generate chiroptical responses are illustrated. Theoretical approaches to assist in the design of these systems are discussed. Development of efficient chiroptical reporters in different states of matter, essential for the implementation in sensing devises, is reviewed. In the last part, remarkable examples of chiroptical sensing applications are highlighted.

An Indo-French campaign was conducted during May 2014 at Gadanki (13.5^oN, 79.2^oE), India, for investigating gravity wave (GW) coupling between the lower and middle atmospheres using a suite of experimental techniques. In this paper we are presenting the sources for those GWs which are observed at airglow layer (~87 km) using (OH) Airglow Imager during the campaign. For this purpose we have developed a background model atmosphere by combining the ERA-Interim, MSIS E 90 model and HWM 07 model data sets. Uniqueness of the present study lies in using near-real time background atmosphere data from simultaneous radiosonde and SVU Meteor Radar covering both source and propagation/ dissipation regions of GWs. We have observed 17 wave events and used GROGRAT model for reverse tracing to locate the source(s). When we searched for the sources near the terminal points or along the ray path, deep convection is found to occur but no strong wind shear. This indicates that the source for the observed GWs is deep convection. We have also simulated small scale waves that can reach mesosphere during different seasons (background conditions). GWs with periods less than 100 minutes and horizontal wavelengths less than 200 km are mostly reaching the mesospheric altitudes. Interestingly, it is found that only few GWs that are generated in the troposphere reach the mesospheric altitudes during the monsoon season, though strong sources like convection and wind shear persist throughout this season. Thus, background wind plays significant role in deciding the propagation of GWs in to upper atmosphere.

Gold nanoclusters (AuNCs) with fluorescence in the Near Infrared (NIR) by both one- and two-photon electronic excitation were incorporated in mesoporous silica nanoparticles (MSNs) using a novel one-pot synthesis procedure where the condensation polymerization of alkoxysilane monomers in the presence of the AuNCs and a surfactant produce hybrid MSNs of 49 nm diameter. This method was further developed to prepare 30 nm diameter nanocomposite particles with simultaneous NIR fluorescence and superparamagnetic properties, with a core composed of superparamagnetic manganese ferrite nanoparticles (MnFe2O4) coated with a thin silica layer, and a shell of mesoporous silica decorated with AuNCs. The nanocomposite particles feature NIR-photoluminescence with 0.6% quantum yield and large Stokes shift (290 nm), and superparamagnetic response at 300 K, with a saturation magnetization of 13.4 emu g-1. The conjugation of NIR photoluminescence and superparamagnetic properties in the biocompatible nanocomposite has high potential for application in multimodal bioimaging.

Acute low back pain (LBP) stands as a leading cause of activity limitation and work absenteeism, and its associated healthcare expenditures are expected to become substantial when acute LBP develops into a chronic and even refractory condition. Therefore, early intervention is crucial to prevent progression to chronic pain whose management is particularly challenging and for which the most effective pharmacological therapy is still controversial. Current guideline treatment recommendations vary and are mostly driven by expertise with opinion differing across different interventions. Thus, it is difficult to formulate evidence-based guidance when relatively few randomized clinical trials did explore the diagnosis and management of LBP while employing different selection criteria, statistical analyses, and outcome measurements. This narrative review aims to provide a critical appraisal of current acute LBP management by discussing the unmet needs and areas of improvement from bench-to-bedside and proposes multimodal analgesia as the way forward to attain an effective and prolonged pain relief and functional recovery in patients with acute LBP.

Eutrophication, a global concern, impacts water quality, ecosystems, and human health. It’s crucial to monitor algal blooms in freshwater reservoirs, as they indicate the trophic condition of a waterbody through Chlorophyll-a (Chla) concentration. Traditional monitoring methods, however, are expen-sive and time-consuming. Addressing this hindrance, we developed models using remotely sensed data from the Sentinel-2 satellite for large-scale coverage, including its bands and spectral indexes, to estimate the Chla concentration on 149 freshwater reservoirs in Ceará, Brazil. Several machine learning models were trained and tested, including k-nearest neighbours, random forests, extreme gradient boosting, the least absolute shrinkage, group method of data handling (GMDH), and sup-port vector machine models. A stepwise approach determined the best subset of input parameters. Using a 70/30 split for the training and testing datasets, the best-performing model was the GMDH, achieving an R2 of 0.91, MAPE of 102.34%, and RMSE of 20.38 g/L, which are values consistent with the ones found in the literature. Nevertheless, the predicted Chla concentration values were most sensitive to the red, green, and near infra-red bands.

The incessant evolution of online platforms has ushered in a multitude of shopping modalities. Within the food industry, however, assessing the delectability of meals can only be tentatively determined based on consumer feedback encompassing aspects such as taste, pricing, packaging, service quality, delivery timeliness, hygiene standards, and environmental considerations. Traditional text data mining techniques primarily focus on consumers' emotional traits, disregarding pertinent information pertaining to the online products themselves. In light of these aforementioned issues in current research methodologies, this paper introduces the Bert BiGRU Softmax model combined with multimodal features to enhance the efficacy of sentiment classification in data analysis. Comparative experiments conducted using existing data demonstrate that the accuracy rate of the model employed in this study reaches 90.9%. In comparison to single models or combinations of three models with the highest accuracy rate of 7.7%, the proposed model exhibits superior accuracy and proves highly applicable to online reviews.

Previous studies to recognize negative emotions (e.g. disgust, fear, sadness) for mental health care have used heavy equipment directly attaching electroencephalogram (EEG) electrodes to the head, making it difficult to use in daily life, and they have proposed binary classification methods to determine whether negative emotion or not. To tackle this problem, we propose a negative emotion recognition system to collect multimodal biosignal data such as five EEG signals in an EEG headset and heart rate, galvanic skin response, and skin temperature in a smart band for classifying multiple negative emotions. It consists of android Internet of Things (IoT) application, an oneM2M-compliant IoT server, and a machine learning server. The android IoT application upload the biosignal data to the IoT server. By using the biosignal data stored in the IoT server, the machine learning server recognizes the negative emotions of disgust, fear, and sadness using a multi-class support vector machine (SVM) model with a radial basis function kernel (RBF). The experimental results showed that the multi-class SVM model achieved 93% accuracy when considering all the multimodal biosignal data. Moreover, when considering only data in the smart band, it could achieve 98% accuracy by optimizing the hyper-parameter of the RBF kernel.

This paper proposes a multimodal fusion 3D target detection algorithm based on the attention mechanism to improve the performance of 3D target detection. The algorithm utilizes point cloud data and information from camera. For image feature extraction, the ResNet50+FPN architecture extracts features at four levels. Point cloud feature extraction employs the voxel method and FCN to extract point and voxel features. The fusion of image and point cloud features is achieved through regional point fusion and regional voxel fusion methods. After information fusion, the Coordinate attention mechanism and SimAM attention mechanism extract fusion features at a deep level. The algorithm's performance is evaluated using the DAIR-V2X dataset. The results show that compared to the Part-A2 algorithm, the proposed algorithm improves the mAP value by 7.9% in BEV view and 7.8% in 3D view at IOU=0.5 (cars) and IOU=0.25 (pedestrians and cyclist). At IOU=0.7 (cars) and IOU=0.5 (pedestrians and cyclist), the mAP value of the SECOND algorithm is improved by 5.4% in the BEV view and 4.3% in the 3D view, compared to other comparison algorithms.

Autonomous driving vehicles rely on sensors for the robust perception of surroundings. Such vehicles are equipped with multiple perceptive sensors with a high level of redundancy to ensure safety and reliability in any driving condition. However, multi-sensor, such as camera, LiDAR and radar, systems bring up the requirements related to sensor calibration and synchronization, which are the fundamental blocks of any autonomous system. On the other hand, sensor fusion and integration have become important aspects of autonomous driving research and directly determine the efficiency and accuracy of advanced functions such as object detection and path planning. Classical model-based estimation and data-driven models are two mainstream approaches to achieving such integration. Most recent research is shifting to the latter, showing high robustness in real-world applications but requiring large quantities of data to be collected, synchronized, and properly categorized. To generalize the implementation of the multi-sensor perceptive system, we introduce an end-to-end generic sensor dataset collection framework that includes both hardware deploying solutions and sensor fusion algorithms. The framework prototype integrates a diverse set of sensors, such as cameras, LiDAR, and radar. Furthermore, we present a universal toolbox to calibrate and synchronize three types of sensors based on their characteristics. The framework also includes the fusion algorithms, which utilize the merits of three sensors , namely, camera, LiDAR and radar, and fuse their sensory information in a manner that is helpful for object detection and tracking research. The generality of this framework makes it applicable in any robotic or autonomous applications, also suitable for quick and large-scale practical deployment.

Improved oil recovery from tight carbonate formations may provide the world with a major source of lower-rate power over several decades. Here we provide an overview of the Arab D formation in the largest oil field on earth, the Ghawar. We investigate the occurrence of microporosity of different origins and sizes using scanning electron microscopy (SEM) and pore casting techniques. Then, we present a robust calculation of the probability of invasion and oil saturation distribution in the nested micropores using mercury injection capillary pressure data available in the literature. We show that large portions of the micropores in Arab D formation would have been bypassed during primary drainage unless the invading crude oil ganglia were sufficiently long. Considering the asphaltenic nature of oil in the Ghawar, we expect the invaded portions of the pores to turn mixed-wet, thus becoming inaccessible to waterflooding until further measures are taken to modify the system’s chemistry.

Many engineered approaches have been proposed over the years for solving the hard problem of performing indoor localisation using smartphone sensors. However, specialising these solutions for difficult edge cases remains challenging. Here we propose an end-to-end hybrid multimodal deep neural network localisation system, MM-Loc, relying on zero hand-engineered features, learning them automatically from data instead. This is achieved by using modality-specific neural networks to extract preliminary features from each sensing modality, which are then combined by cross-modality neural structures. We show that our choice of modality-specific neural architectures is capable of estimating the location with good accuracy independently. But for better accuracy, a multimodal neural network fusing the features of early modality-specific representations is a better proposition. Our proposed MM-Loc solution is tested on cross-modality samples characterised by different sampling rates and data representation (inertial sensors, magnetic and WiFi signals), outperforming traditional approaches for location estimation. MM-Loc elegantly trains directly from data unlike conventional indoor positioning systems, which rely on human intuition.

Multimodal traveling is expected to enhance mobility for users, reduce inequalities of car ownership and reduce emissions. In the same context, ride-sharing aims to minimize negative impacts related to emissions, reduce travelling costs and congestion, and increase passenger vehicle occupancy, and public transit ridership when planned for first/last mile trips. The goal of this study is to present the planning of a multimodal pilot demonstration and the challenges that emerged during and after its implementation in Athens, Greece. The demo aims to enhance the connection of low-density Attica Region areas to public transport (PT) modes, and specifically to the metro, through the provision of demand responsive ride-sharing services. During the demo period two different applications were utilized: the “Travel Companion” app and the “Driver Companion” app, which refer to passengers and drivers of the ride-sharing service. Identification of demo participants is performed through a Stated Preference (SP) experiment. Results of the demo implementation, as well as challenges that were faced show that although participants are willing to try new mobility solutions, the readiness and reliability of the new service are essential attributes towards maintaining existing users and engaging new ones.

With the widespread of embedded sensing capabilities of mobile devices, there has been unprecedented development of context-aware solutions. This allows the proliferation of various intelligent applications such as those for remote health and lifestyle monitoring, intelligent personalized services, etc. However, activity context recognition based on multivariate time series signals obtained from mobile devices in unconstrained conditions is naturally prone to imbalance class problems. This means that recognition models tend to predict classes with the majority number of samples whilst ignoring classes with the least number of samples, resulting in poor generalization. To address this problem, we propose to augment the time series signals from inertia sensors with signals from ambient sensing to train deep convolutional neural networks (DCNN) models. DCNN provides the characteristics that capture local dependency and scale invariance of these combined sensor signals. Consequently, we developed a DCNN model using only inertial sensor signals and then developed another model that combined signals from both inertia and ambient sensors aiming to investigate the class imbalance problem by improving the performance of the recognition model. Evaluation and analysis of the proposed system using data with imbalanced classes show that the system achieved better recognition accuracy when data from inertial sensors are combined with those from ambient sensors such as environment noise level and illumination, with an overall improvement of 5.3% accuracy.

Marine sponges, a well documented prolific source of natural products, harbors numerous microbial communities believed to possess N-acyl homoserine lactones (AHLs) mediated Quorum sensing (QS) as one of the mechanisms of interaction. Bacteria and eukaryotic organisms are known to produce molecules that can interfere with QS signaling, thus affecting microbial genetic regulation and function. In the present study, we established the potential for production of both QS signal molecules as well as QS interfering molecules (QSI) in the same sponge species Sarcotragus spinosulus. A total of eighteen saturated acyl chain AHLs were identified along with six putative unsaturated acyl chain AHLs. Bioassay guided purification led to the isolation of two brominated metabolites with QS-interfering activity. The structures of these compounds were elucidated by comparative spectral analysis of 1HNMR and HR-MS data and was identified as 3-Br-N-methyltyramine (1) and 5,6-dibromo-N,N-dimethyltryptamine (2). The QSI activity of compounds 1 and 2 were evaluated using reporter gene assays for long- and short-chain signals (E. coli pSB1075 and E. coli pSB401) and was confirmed by measuring dose dependent inhibition of proteolytic activity and pyocyanin production in P. aeruginosa PAO1. The obtained results showed the co-existence of QS and QSI in S. spinosulus, a complex network which may mediate the orchestrated function of the microbiome within the sponge holobiont.

This study aims to enhance multimodal transportation systems by mitigating CO2 emissions and enhancing operational efficiency. It introduces a novel centralized cargo concentration approach tailored for regions facing geographical challenges. The method proposes for direct transportation of cargo from its loading origin to export ports, bypassing intermediate hubs. The mathematical model determines the most efficient means of transport for each route, factoring in variables like distance, volume, and cargo type. Research results indicate that, in scenarios with a high concentration of cargo, multiple hubs may not be necessary, which could streamline transportation and logistics operations. Modal preferences vary based on regional dynamics and cargo characteristics, with rail and sea transport emerging as preferable options under specific circumstances, surpassing the efficacy of road transport. The proposed model showcases reductions in both logistics costs and CO2 emissions compared to road-centric scenarios. It underscores the strategic integration of diverse transportation modes as pivotal for enhancing efficiency and sustainability. This study furnishes a framework adaptable to optimizing multimodal transportation systems in regions sharing similar geographical and logistical attributes.

Currently the bimatoprost implant (DURYSTA) is approved for a single administration secondary to concern for progressive corneal endothelial cell (CEC) loss with intracameral administration. In order to better understand the cause of CEC loss and other adverse events (AEs) with intracameral administration, multimodal imaging of the implant utilizing optical coherence topography (OCT), robotic ultrasound and bio microscopy video recording was utilized. Inherently the anterior chamber is a non-confined space. In certain examples the implant assumed a round and spotted structure as it picked up iris pigment as it migrated around the anterior chamber. Accordingly, we explored delivery of the implant in the middle segment behind the iris and in front of the pseudophakic lens. We then evaluated the suitability of various imaging modalities to assess the implant in this location. For middle segment administration, the ArcScan Insight 100 robotic ultrasound provided resolution and repeatability sufficient to monitor the implant behind the iris. This preliminary study hypothesizes that DURYSTA migration in the anterior chamber is the main source of AEs. Through use of a confined location, the middle segment, the implant is unlikely to migrate and cause CEC loss while also delivering drug closer to its site of action, the ciliary body. Further efforts are underway to understand the suitability of the middle segment for serial DURYSTA administration to maximize intraocular pressure lowering while minimizing AEs.

Collaborative robots (cobots) have largely replaced conventional industrial robots in today’s workplaces, particularly in manufacturing setups due to their improved performance and intelligent design. In the framework of Industry 5.0, humans are working alongside cobots to accomplish the required level of automation. However, human-robot interaction has brought up concerns regarding human factors (HF) and ergonomics. A human worker may experience cognitive stress as a result of cobots' irresponsive nature in unpredictably occurring situations, which adversely affects productivity. Therefore, there is a necessity to measure stress to enhance a human worker’s performance in a human-robot collaborative environment. In this study, factory workers’ mental workload has been assessed using physiological, behavioural, and subjective measures. Electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) signals have been collected to acquire brain signals and track hemodynamic activity, respectively. The effect of task complexity, cobot’s movement speed, and cobot’s payload capacity on the mental stress of a human worker have been observed, for a task designed in the context of a smart factory. Task complexity and cobot’s speed have proved to be more impactful. As physiological measures are unbiased and more authentic means to estimate stress, eventually, they may replace the other conventional measures if they prove to correlate with the results of traditional ones. Here, regression and artificial neural networks (ANN) have been utilised to determine the correlation between physiological data and subjective and behavioural measures. Regression has performed better for most of the targets and the best correlation (rsq-adj = 0.654146) has been achieved for predicting missed beeps, behavioural measure, using a combination of multiple EEG and fNIRS predictors. The k-nearest neighbours (KNN) algorithm has been used to evaluate the accuracy of correlation between traditional measures and physiological variables, with highest accuracy of 77.8% achieved for missed beeps as the target. Results show that physiological measures can be more insightful and have the tendency to replace other biased parameters.

The Na+/K+ ATPase is a protein involved in the active transport of ions across the cellular membrane. Ouabain is a cardiotonic glycoside that, by inhibiting the Na+/K+ pump, interferes with cell processes mediated directly by the pump, but also indirectly influences other cellular processes, such as cellular cycle and proliferation, growth, cell differentiation, angiogenesis, migration, adhesion and invasion. We used the SK-BR-3 breast cancer cell line, mesenchymal stem cells (MSCs), and tumor-associated fibroblasts (TAFs) in vitro for determining the changes induced by Ouabain in concentration of 10-5 M, 10-6 M, 10-8 M, 10-9 M on these cellular types, employing specific methods: SEM for morphological evaluation; immunocytochemistry, flowcytometry to assess immunophenotypical markers expression, cell cycle and Annexin V/PI assay for cellular proliferation and viability testing; flowchamber assay to evaluate functional adhesion behavior; RT-PCR for Na+/K+ ATPase subunits molecular expression. Being indicative of the pro-apoptotic and inhibitory effect of Ouabain on tumor invasion and metastasis, the results support the addition of Ouabain to the oncological therapeutic arsenal, trailing the “repurposing drugs” approach.

The present study illustrates a brain-computer interface designed and developed to be wearable, portable, and user-friendly. Eight dry electroencephalographic sensors were adopted to acquire the brain activity associated with motor imagery. Multimodal feedback in extended reality was exploited to improve the detection of neurological phenomena. Twenty-seven healthy subjects used the proposed system in five sessions to investigate the effects of feedback on motor imagery. The sample was divided into two equal-sized groups: the "neurofeedback" group, which performed motor imagery while receiving feedback, and the "control" group, which performed motor imagery with no feedback. Several questionnaires were administrated to participants aiming to investigate the usability of the proposed system and individual’s ability to imagine movements. The highest mean classification accuracy across subjects of control group was about 62 % with 3 % associated type A uncertainty, and 69 % with 3 % uncertainty for neurofeedback group. Moreover, in some cases the results were significantly higher for the neurofeedback group. The perceived usability by all participants was high. Overall, the study highlights the advantages and the pitfalls of using a wearable brain-computer interface with dry sensors. Notably, the results and the perceived usability pave the way for the employment of the proposed system in tele-rehabilitation.

In this paper, we present a multimodal dataset for the analysis of classroom atmosphere, based on the behavior and voice of teachers in teaching scenarios. we propose four visual models, three audio models, and one visual-audio dual-modality model to be tested on our dataset. The results indicate that the CH-CC dataset is feasible and reliable and that the visual modality plays a major role in the analysis of this dataset.

This work proposes a hollow-core photonic crystal fiber-based edible oil sensor in the terahertz (THz) range (e.g., 1.0THz ≤ f ≤ 3.0THz) and different sensing characteristics are numerically analyzed. The suggested sensor’s performance was assessed by means of COMSOL Multiphysics, a commercial program that uses the finite element approach. The computational results indicate that the relative sensitivity is 85.591%, 84.648%, 82.625%, 82.683%, and 79.161%, respectively, at f = 2.2 THz, for several types of sunflower oil, mustard oil, coconut oil, olive oil and palm oil; and the corresponding effective areas are 7.22×10-8 um2, 7.09×10-8 um2, 6.83×10-8 um2,7.09×10-8 um2, 6.5231 ×10-8 um2. In addition, the effective material loss for sunflower oil, muster oil, coconut oil, olive oil, and palm oil has been found to be 0.02561 cm-1, 0.027054 cm-1, 0.030322 cm-1, 0.028854 cm-1 ,0.035427cm-1 respectively. Moreover, the proposed sensor also has low confinement loss are 1.55×10-8 dB/m, 1.63×10-8 dB/m, 1.31×10-8 dB/m, 1.99×10-8 dB/m, 4.0345×10-8 dB/m.This proposed sensor can be fabricated using extrusion and 3D-printing technologies, and due to its augmented detecting capabilities, it can be a vital part of oil sensing devices implemented in real life such as industry fields.

Scene classification plays an important role in the intelligent processing of high-resolution satellite (HRS) remotely sensed image. In HRS image classification, multiple features, e.g. shape, color, and texture features, are employed to represent scenes from different perspectives. Accordingly, effective integration of multiple features always results in better performance compared to methods based on a single feature in the interpretation of HRS image. In this paper, we introduce a multi-task joint sparse and low-rank representation model to combine the strength of multiple features for HRS image interpretation. Specifically, a multi-task learning formulation is applied to simultaneously consider sparse and low-rank structure across multiple tasks. The proposed model is optimized as a non-smooth convex optimization problem using an accelerated proximal gradient method. Experiments on two public scene classification datasets demonstrate that the proposed method achieves remarkable performance and improves upon the state-of-art methods in respective applications.

The study of paraneoplastic syndromes in humans has a long history, whereas for domestic animals – only since the beginning of the 21^th century. There are various forms of paraneoplastic syndromes; many classifications based on pathogenetic, organ-system principles are described. In accordance with the modern classification, paraneoplastic syndromes are divided into endocrinopathies, hematological paraneoplastic syndromes, gastrointestinal, renal, dermatological, neurological, ophthalmological. The purpose of this study was to develop a scientifically based algorithm for diagnosing ophthalmopathies against the background of paraneoplastic syndrome in dogs, including a multifactorial system of “hormonal-drug treatment” of paraneoplastic ophthalmopathies. The work used a comprehensive methodological approach, including the collection of anamnestic data, clinical examination of the animal, examination of the area of the pathological process, hematological studies, cytomorphological studies, visual diagnostic methods with complete oncological and ophthalmological examinations using the necessary equipment. Treatment for cancer included surgical removal of the tumor, chemotherapy and radiotherapy. The choice of treatment protocol depended on the location of the disease, tumor phenotype (if present) and stage of the malignancy. The only chemotherapy was applied using complex protocols in the treatment of dog lymphomas. Treatment of melanoma in dogs was carried out using an integrated approach which combined methods of surgery, chemotherapy, and radiotherapy. The obtained results can be recommended for use in the diagnosis and treatment of paraneoplastic ophthalmopathies in dogs.