1. Introduction
Due to the increasing occurrence of global climate change, the ecological environment is encountering unparalleled difficulties [
1,
2]. This alteration not only has a direct effect on the natural environments where wild animals live, but it also significantly influences their movements during migration, their methods of searching for food, and their techniques for staying alive [
3,
4]. Therefore, it is essential to gain a more profound comprehension of the behavioural tendencies shown by wild animals in relation to climate change [
5].
Animal movement modelling refers to the systematic analysis of animal movement behaviour using quantitative methods [
6]. Nevertheless, conventional modelling techniques that rely on binary location data have inherent constraints in accurately representing the intricacies of animal movement [
7]. Technological improvements have enabled the use of sophisticated deep learning and reinforcement learning methods, which provide unparalleled versatility [
8]. The current approaches for modelling animal behaviour include mathematical theory-based methods, agent-based methods, and intelligence methods based on deep learning [
9,
10,
11]. The step-selection function (SSF) is a common technique in animal locomotion modelling that incorporates environmental variables into the model to simulate movement [
12]. Nevertheless, the SSF usually assumes that the animal’s decision-making process is unchanging and does not vary over time, which may not fully capture the dynamic character of animal behaviour.
Mathematical theory-based approaches serve as essential tools for analysing animal movement behaviour [
13]. Among these models, those that use random walks may effectively replicate animal movement over extended periods of time, although their foundation is mostly in stochastic modelling [
14,
15]. Without more data about an animal’s movements, it is challenging to differentiate behavioural changes resulting from environmental or spatial interactions just through observation and analysis of movement trajectories. Models such as Carlson and Soucek’s [
16] firefly mating behaviour model, Adeva’s [
17] bee foraging activity model, and Molina-Delgado et al.’s [
18] rat behaviour model can accurately replicate animal behaviour in certain situations. However, their capacity to accurately depict the intricate cause-and-effect relationships between behaviour and the environment in real-life settings is restricted. When dealing with situations where there are multiple causal links and uncertainty between behaviour and environment, it is often necessary to use more intricate and thorough models and algorithms. Agent-based methodologies prioritise the creation of models based on the behaviours shown by individual agents [
19]. For instance, Heit et al. [
7] employed a Hidden Markov Model (HMM) together with GPS data to replicate the three-dimensional motion of a puma, overcoming the constraints of conventional two-dimensional models. Anderson et al. [
20] developed an agent-based model that integrates environmental factors to replicate the movement patterns of individual animals, offering a different approach compared to conventional techniques. Agent-based modelling systems have the capability to explore the interactions between animals and their environment. However, their reliance on extensive environmental data makes it challenging to use them in real situations. Deep learning methods primarily utilise designs such as convolutional neural networks (CNNs) and recurrent neural networks (RNNs). CNNs are employed for dimensionality reduction in the feature space, while RNN is a neural network architecture specifically developed for forecasting time-series data [
21]. Long short-term memory (LSTM) is a specific sort of RNN structure that excels at retaining information for extended durations. This characteristic makes it highly proficient in tasks involving the prediction of sequences. LSTM is commonly employed as a distinct form of architecture in recurrent neural networks [
22,
23]. Roy et al. [
17] employed generative adversarial networks (GANs) along with LSTM and CNNs to create generators and discriminators. They conducted experiments using various structure combinations on a dataset of seabird foraging trajectories. Among them, GANs composed of CNN-CNN achieved the best results. However, GANs still fall short in capturing local-scale descriptive statistics, such as step-speed distributions. Wijeyakulasuriya et al. [
21] developed a comprehensive paradigm for forecasting animal locomotion that may be used universally. Their research involved a comparative analysis of single-step and multi-step long-term prediction outcomes using random forests, neural networks, and recurrent neural networks. The findings revealed that deep learning techniques exhibited superior performance in single-step prediction, whereas traditional models demonstrated better results in multi-step long-term prediction. Out of all the deep learning models, LSTM had the highest performance in multi-step long-term prediction. Constructing machine learning and deep learning models is more straightforward compared to classic parametric motion models, as the latter sometimes rely on limiting assumptions. Nevertheless, the interpretability of machine learning and deep learning models is more challenging compared to parametric motion models.
Remote sensing images are visual representations of surface data acquired using remote sensing technologies. The Earth’s surface is remotely viewed and recorded using sensors deployed on platforms such as satellites, aircraft, drones, and other similar devices. Both classic machine learning algorithms and deep learning models face a common challenge: they are unable to incorporate complex environmental variables, particularly unstructured data in the form of remote sensing images. Remote sensing images include abundant information about the surrounding environment, which can be incorporated as environmental variables into animal movement models. The challenge lies in extracting valuable information from remote sensing images in an efficient manner.
Reinforcement learning is a machine learning approach that acquires optimum judgements via iterative interactions with the environment [
24,
25]. This learning technique involves an agent making decisions based on the current state at each time step and modifying its approach depending on feedback rewards from the environment in order to maximise the total rewards over the long run [
26]. The advancement of deep learning-based technologies in remote sensing images has facilitated the creation of agents that can replicate animal movement behaviour [
27,
28,
29,
30]. The progress of these technologies has increased the ease of obtaining animal trajectory data and environmental data. Wildlife monitoring technology, such as GPS collars, tags, and acoustic localisation, are extensively used [
31]. Furthermore, drones and satellite remote sensing images have the capability to provide intricate and precise surface data [
32]. Agent-based animal movement models enable the adoption of an individual-based approach to manipulate important variables, such as internal states, external stimuli, locomotion skills, and navigation abilities [
33,
34]. This allows for a more thorough investigation of the underlying mechanisms that influence animal behaviour. However, such models still face a number of challenges and problems, such as the complexity of animal movement and interaction with the environment [
35]. In order to tackle the difficulties encountered in modelling wildlife behaviour, it is crucial to investigate novel ways for simulating animal behaviour. Specifically, agent models that can include various data sources and replicate animal movement behaviour are of the utmost importance.
The main contributions of this study are as follows:
- 1.
We introduce a new technique called AnimalEnvNet, which combines LSTM, CNN, and Attention Mechanism processes to extract and merge features from trajectory data and remote sensing images. This technique successfully simulates animal behavioural tendencies.
- 2.
By using a deep reinforcement learning framework, we improved the model’s capacity to generalise and adapt by allowing the agents to perceive environmental input and acquire animal behaviour tactics.
- 3.
This work on the subject of animal behaviour modelling addresses the shortcomings of conventional methodologies, offering fresh insights and approaches for investigating the behavioural patterns of animals.
4. Results
4.1. Experimental Design
The starting learning rate in the experiment was set at 0.001. In the experiment, we adopted the configuration shown in
Table 2. To evaluate the generalisation ability of the model, we divided each dataset into a training dataset and a validation dataset. We randomly selected 70% of the data as the training dataset and the remaining 30% as the validation dataset.
4.2. Comparison and Evaluation Results of Different Models
We used the mean absolute error (MAE) to evaluate the similarity between predicted and actual trajectories due to its computational efficiency and simplicity. While the Fréchet distance is a robust measure for trajectory similarity, it requires significantly more computational resources. MAE quantifies the average size of discrepancies between projected values and actual observations inside a model. The calculating formula is as follows:
is the mean absolute error.
n is the sample size.
is the predicted value for the
i-th observation.
is the actual value for the
i-th observation.
This work aims to examine the impact of environmental information on the model. To achieve this, we propose a model called LSTM-basic. Furthermore, we investigate its impact in contrast to cutting-edge GAN models. The LSTM-basic model is employed to extract trajectory features. It comprises two LSTM units: an Attention Mechanism Unit and a Fully Connected Unit. During the model operation, the time and motion parameters are inputted into two distinct LSTM units for individual processing, and then the outputs of both units are combined. The feature data that have been spliced are then subjected to further processing using the Attention Mechanism Unit and the Fully Connected unit, resulting in the generation of the final output. GANs are composed of a generator and a discriminator. The generator generates simulated trajectory data, while the discriminator attempts to differentiate between real trajectory data and generated trajectory data. We choose MSELoss as the loss function for the continuous action space in order to quantify the discrepancies between the model’s predictions and the actual values. During the training process, other components that share the same structure can utilise the identical model weights. It enhances the model’s resilience in handling various data circumstances. Detailed information of the parameters of different models is shown in
Table 3. The training outcomes are displayed in
Table 4.
The LSTM-basic model has an MAE of 102.9, the AnimalEnvnet model has an MAE of 28.4, and the GANs have an MAE of 55.5. The AnimalEnvnet model exhibits the lowest MAE value, suggesting that it has less average error in its predictions and is more accurate in estimating the real values. AnimalEnvNet has both a lower initial MAE and a faster convergence rate compared to the other models. By around round 45, the performance of the model reaches a stable MAE of around 40. In contrast, the LSTM-basic and GAN models stabilise at higher MAE values of approximately 110 and 70, respectively.
The LSTM-basic model has a loss of 0.0021, while the AnimalEnvnet model has a loss of 0.0002. The AnimalEnvnet model demonstrates improved data fitting during the training process, exhibiting reduced discrepancy between its projected and actual values. Regarding training speed, the AnimalEnvNet model, however, reached a stable performance level around the 50th batch. Although AnimalEnvNet requires a longer training period, its ultimate performance is significantly superior, achieving an average training duration of approximately 49 s per batch.
Figure 7 displays a comparison of the MAE results for the models.
Figure 8 displays a comparison of the outcomes obtained from the MSELoss model.
In conclusion, the AnimalEnvNet model achieves a single-step trajectory prediction MAE of 28.4 m. By incorporating environmental information feature modules, AnimalEnvNet demonstrates a significant improvement in single-step prediction accuracy.
To further validate the generalisation capability of the models, we conducted experiments on the African elephant dataset. The results of the training and validation processes are shown in
Figure 9 and
Figure 10. Similar to the giant panda dataset, the AnimalEnvNet model also exhibited the best predictive performance on the African elephant dataset.
Despite the use of different datasets, the AnimalEnvNet model was still able to quickly converge during the training process and maintain the lowest prediction error. In contrast, the GAN model and LSTM-basic model had similar error performances, but the GAN model slightly outperformed the LSTM-basic model. By conducting experiments on two different datasets, it can be seen that the AnimalEnvNet model has strong generalisation capabilities across different trajectory datasets. This indicates that the AnimalEnvNet model has high robustness in capturing temporal and spatial features, performing excellently not only on the giant panda dataset but also maintaining low prediction errors on the African elephant dataset.
4.3. Results of Reinforcement Training for Agent
Table 5 shows the parameters of reinforcement learning training. During the training of reinforcement learning, at the start of each episode, the agent navigates through the environment based on a predetermined policy and obtains rewards at each time step. If the inverse of the reward surpasses 1000, it signifies that the agent has strayed from the genuine activity region of the animals and the story concludes. Each action in the game corresponds to a one-hour time span in the real world. Once a certain number of episodes have passed, the agent collects data from the experience buffer in order to learn. Both the Actor and Critic models undergo repeated updates, using lambda_gae_adv and lambda_entropy to regulate the advantage value estimate and the exploration ratio.
Figure 11 shows the agent’s learning curve during the training procedure. An evident surge is seen after about 20 iterations, indicating that the agent begins to proficiently acquire knowledge and navigate the surroundings. Once the ordinate hits around 12,000, the average reward becomes stable. This indicates that the agent has acquired efficient methods within the environment and is either nearing or attaining performance levels that are close to ideal.
Figure 12 shows the mean number of steps accomplished by the agent during the training procedure. A grand total of 200 batches were taught, with each individual batch comprising training across 4000 episodes. The x-axis shows the training batches for the agent, while the y-axis reflects the average number of steps performed by the agent every episode. The agent’s average completion steps in the first 20 batches is around five steps. The main reason for this is the division of the possible actions into discrete options and the restrictions on how far the agent may advance, which prevents it from satisfying the criteria for ending an episode. As the training advances, starting with the 20th iteration, there is a rise in the mean number of steps accomplished by the agent. After completing 200 training batches, which included exploring over eight hundred thousand episodes, the average number of steps eventually stabilises at 12 steps.
Figure 13 presents visual information on the differences between the real and predicted trajectories. It consists of four subplots labelled A, B, C, and D, each comparing real trajectories (left) and predicted trajectories (right) at different time points. In all subplots, the starting points are marked, and the number of steps is indicated by colour gradients. The trajectory positions are plotted on a coordinate grid with latitude and longitude axes.
In conclusion, the AnimalEnvNet model’s single-step prediction trajectory on this dataset can have an average absolute inaccuracy of around 25.8 m. AnimalEnvNet has enhanced its single-step prediction accuracy by incorporating an environmental information feature module, surpassing the LSTM-basic model that just relies on trajectory information.
4.4. Ablation Study
We conducted an ablation study to analyse the impact of each component of the AnimalEnvNet model. The key components considered include the LSTM module, the CNN module, the Attention Mechanism, and the Feature Fusion with Dimensionality Reduction module. To assess the contribution of each component, we designed the following ablation experiments:
- 1.
AnimalEnvNet: The complete AnimalEnvNet model with all components included.
- 2.
A: Directly combines the outputs of LSTM and CNN without using the Attention Mechanism.
- 3.
B: Replaces the CNN module with simple fully connected layers.
- 4.
C: Replaces the LSTM module with simple fully connected layers.
- 5.
D: Removes the Feature Fusion with Dimensionality Reduction module. Directly concatenates the outputs of LSTM and CNN without feature fusion.
We evaluated the performance of each model using three metrics: MAE, the convergence rate, and the training time. The results of the ablation study are shown in
Table 6.
Figure 13.
Comparison of differences between actual and predicted trajectories. (A–D) represent four different moments.
Figure 13.
Comparison of differences between actual and predicted trajectories. (A–D) represent four different moments.
The MAE of AnimalEnvNet is 28.4 m, indicating the highest prediction accuracy. Removing the Attention Mechanism increases the MAE to 40.2 m, demonstrating the importance of the Attention Mechanism in focusing on relevant features. Removing the CNN module significantly increases the MAE to 55.3 m, highlighting the critical role of spatial feature extraction. The model without the LSTM module has the highest MAE at 60.7 m, indicating the importance of LSTM in capturing temporal dependencies. Removing the Feature Fusion module increases the MAE to 45.1 m, showing the benefit of combining features from different modalities.
In terms of convergence rate, the complete model reaches stable performance within 55 batches, while the models without the Attention Mechanism, CNN, LSTM, and Feature Fusion module require 80, 100, 120, and 90 batches, respectively. This indicates that all components contribute to faster convergence, with LSTM and CNN being the most critical. Regarding training time, the complete model takes an average of 49 s per batch. The simplified models show a reduction in training time due to their reduced complexity, but this comes at the cost of increased MAE and longer convergence times. The ablation study highlights the significance of each component in the AnimalEnvNet model. The LSTM module is crucial for capturing temporal dependencies, the CNN module is essential for spatial feature extraction, the Attention Mechanism enhances the model’s ability to focus on important features, and the Feature Fusion module effectively combines multimodal data. Removing any of these components results in a substantial drop in performance, confirming their importance in the model.
5. Discussion
Simulating animal movement trajectories has long been an important area of study in the field of animal behaviour studies. Traditional approaches, such as random walk models [
44] and HMMs [
45], are often used in these types of research. These approaches are very effective in capturing local behavioural patterns, but they have limits when it comes to capturing complicated time and space changes [
46,
47]. Roy et al. [
48] achieved exceptional results in simulating core feeding trajectories by using GANs. However, GANs generally focus the generation of trajectories that resemble actual data distributions, placing less attention on the acquisition of knowledge and adjustment to environmental and behavioural methods [
49]. AnimalEnvNet has greater training stability compared to trajectory simulation using GANs, since GAN models are susceptible to encountering challenges of training instability. Although GAN models are capable of producing lifelike trajectories, they are limited in their capacity to adjust to changes in the environment and behavioural methods. The AnimalEnvNet, which uses reinforcement learning-trained agents, is capable of precisely imitating animal movement for a maximum of 12 steps, with an error limit of 1000 m. This showcases its ability to adapt to changes in the environment and acquire novel strategies via learning.
While our model has shown some level of success in forecasting animal movement, it fails to provide a clear explanation for the precise impacts of environmental factors on locomotor behaviour. The primary objective of our work is to forecast movement trajectories rather than elucidating the underlying reasons driving movement. However, comprehending the impact of the environment on animal locomotion is a crucial facet of ecological and behavioural investigation. Hence, our methodology has certain constraints in this aspect.
Our model has satisfactory performance in making forecasts for short time intervals, with an error of approximately 28.4 m. However, the accuracy of our predictions drastically diminishes when forecasting over longer time periods, with an error of around 1 km after 12 h. This implies that simulations may have limited efficacy in imitating animal motions over extended periods of time. Forecasting animal movements over extended time periods is a highly intricate undertaking that encompasses numerous aspects, including the stochastic nature of animal behaviour, fluctuations in the environment, and various sources of uncertainty. Hence, it is crucial that we engage in explicit discussions regarding the constraints and anticipated outcomes of models. While the model’s accuracy diminishes when making predictions over longer time periods, it remains valuable for short-term predictions and recognising behavioural patterns. To enhance the model’s long-term forecasting capability, future research may explore the incorporation of additional environmental factors and the utilisation of more advanced time-series analysis techniques. Furthermore, the implementation of the model should be customised to suit unique requirements. For instance, the model’s exceptional accuracy in predicting short-term outcomes would be highly valuable for monitoring animal behaviour and responding to emergencies in wildlife reserves. Ultimately, although our method has certain limits, it remains highly valuable for studying animal locomotor behaviour and making short-term predictions. In future investigations, it is necessary to precisely delineate the practical application scenarios of the model and thoroughly investigate potential enhancements.