Anomaly Detection for Internet of Things Time Series Data Using Generative Adversarial Networks with Attention Mechanism in Smart Agriculture

1Introduction

Internet of Things (IoT) technology, as a paradigm, continues to drive the evolution of modern industries and smart cities. Due to energy challenges in environmental pollution, energy depletion, and water shortage in the whole world, there is an urgent need for digitalization of the agricultural industry to move toward digitalization (Cao et al., 2021). To address these challenges, smart solutions based on real-time monitoring and decision-making is receiving widespread attention . Smart agriculture is an integration of IOT technology and modern agriculture , which mainly takes modern agriculture as an application scenario to achieve scientific cultivation and precise control . For smart agriculture IoT systems, automated management and smart decision of IoT applications are driven by the detailed analysis of data . These data are collected by a large number of sensors that provide information about different environmental conditions. Thus, environmental monitoring and data analysis play an important role in increasing crop yields. . However, IoT devices in smart agriculture are usually exposed to harsh environments and are highly susceptible to damage due to cost control . In addition, the heterogeneous nature of network devices makes it difficult to design protocols, and the transmission of data is easily compromised . Poor communication quality can lead to data loss and misrepresentation. Increasingly complex IoT systems bring technical, complexity and therefore make the design of privacy and security mechanisms more difficult. This can also expose the network to attacks that could lead to data tampering. Missing or misrepresented data is significantly different from normal data in the time series data collected by the sensors . These can be considered as anomalies in the data . Moreover, IoT applications also require an algorithm to analyze these data to facilitate intelligent decision-making. By analyzing the data in detail, the intelligent system can make the most efficient resource scheduling to increase crop yield. Testing for crop growth patterns can help reduce soil depletion, and different weather and soil conditions can affect irrigation decisions . The main idea of data analysis in the smart agriculture scenario is to analyze various sensor data, and the analysis results can reflect the changes in the environment . In particular, data that differ from normal data due to environmental changes can be also designated as anomalies. Therefore, anomaly detection has become an important work of smart agricultural IoT.

2Related work

With the rapid development of computer technology, researchers began to experiment with computer technology to solve anomaly detection. Hawkins (1980) had a widely accepted explanation of anomalies, namely, “in a given data set, anomalous data are that part of the data that is significantly different from the majority of the data.” Current anomaly detection methods can be broadly classified into proximity-based methods, probability-based methods, prediction-based methods, and deep learning-based methods (Aggarwal, 2017). These methods except deep learning are called traditional methods. They used statistical measures to calculate the correlation between the data records. These techniques assumed that the time series is linear and follows a known statistical distribution, which makes them inapplicable to many practical problems (Adhikari and Agrawal, 2013). As the volume and dimensionality of data grow, more deep learning algorithms have been proposed for anomaly detection on complex data. Deep learning-based anomaly detection methods have advantages over these methods in characterizing multidimensional time series data and are more helpful in solving practical problems.

Generally, time series data anomaly detection algorithms are divided into two steps. The data are modeled by different data structures and then the degree of deviation of the test data from the normal data is evaluated based on different forms of metrics (e.g., distance-based and density-based). Data with excessive deviations are judged to be abnormal. The deep learning-based anomaly detection is similar to the above process. Neural networks are used in the data representation phase to learn the data distribution, and reconstruction-based methods are applied in the anomaly calculation phase to measure error. Since the data structure learns the normal data distribution, there will be a large reconstruction error using this model to reconstruct abnormal data. In recent years, there has been an increasing number of studies using GAN for anomaly detection. As a result, a state-of-the-art survey of the anomaly detection for GAN is discussed in the following section.

2.1GAN for image anomaly detection

AnoGAN (Schlegl et al., 2017) was the first work that applied GAN for image anomaly detection. The model was trained with normal data and the final anomaly scores were obtained by calculating errors in the trained generator and discriminator. The reconstruction error was calculated in the generator to calculate the error more efficiently. However, this computation requires finding the inverse mapping from the data space to the latent space and is not synchronized with the training. It can lead to extremely high error computation time. Zenati et al. (2018a) proposed an efficient GAN-based anomaly detection to solve the above problem. They added an extra encoder to GAN to avoid looking for latent vector at each detection. The calculation of the anomaly error was the same as the AnoGAN. Skip-GANomaly proposed by Akçay et al. (2019) introduced an architecture of skip connection to improve image reconstruction. The model improved image reconstruction but did not perform well on all data sets, which was limited by unstable training. Zenati et al. (2018b) pointed out that AnoGAN was inappropriate for real-time anomaly detection or larger data sets. They proposed a bi-directional GAN for image anomaly detection, which simultaneously trained the inverse mapping through an encoder network. The model contained three discriminators which effectively improved training stability.

2.2GAN for time series anomaly detection

The achievements of GAN in image anomaly detection have attracted the attention of researchers, and have been introduced into time series anomaly detection. Li et al. (2018) proposed a GAN-based anomaly detection method (GAN-AD) for time series data, which was used to detect possible anomalous behaviors in complex networks. To capture the correlation of time series data, Long Short-Term Memory networks (LSTM) were used as the basic model to learn normal data distribution patterns. For the evaluation of the error, since the output of the discriminator indicated whether the sample is false or not, it was used directly as the anomaly score to find the anomaly. Li et al. (2019) later extended their study to use a vanilla GAN model to capture multivariate time series model distributions and detect anomalies using reconstruction errors and discriminator outputs. Bashar and Nayak (2020) improved on AnoGAN (Schlegl et al., 2017) and proposed an anomaly detection algorithm for time series data. The model used a convolutional neural network (CNN) as the basic network to capture the correlation between variables. Both of the above models can learn the time correlation of time series data and effectively detect anomalies. However, they also need to find the inverse mapping from the real space to the latent space, which requires an inversion of the generator resulting in a longer computation time.

To address this problem, Geiger et al. (2020) proposed TadGAN based on Zenati et al. (2018a). This model introduced cycle-consistent GAN architectures, which allowed the generator to compute the reconstruction error directly without finding the inverse mapping and reducing the computation time. For the calculation of the anomaly score, the combination of point-wise difference and discriminator was typically considered as the anomaly score. However, using point-wise difference measures alone does not exactly fit the time series data characteristics. The time series can form different smoothing curves, and the shape differences between these curves should be equally considered in the reconstruction error calculation. The TadGAN used curve similarity as a form of calculation of reconstruction error. However, they studied the point-wise difference and curve similarity separately and did not consider them together to meet a realistic definition. Apart from this, the existing articles are insufficient for the study of reconstruction errors.

2.3Anomaly detection in smart agriculture

Research on deep learning-based anomaly detection for IoT systems has yielded excellent results, some of which have been introduced into smart agriculture to address emerging challenges. Most of the research on smart agriculture has focused on the field of anomaly detection in agricultural images, such as the identification of pests and crop diseases. TPest-RCNN proposed by Li et al. (2021) aimed to identify whitefly and thrips in greenhouses. The model was trained on a set of pest images captured by a flytrap and used a transfer learning strategy to achieve improved detection. Liu and Wang (2020) optimized the feature layer of Yolo V3 model by using the image pyramid to achieve multi-scale feature detection and improved the detection accuracy and the speed of Yolo V3 model. Experiments showed that the model can accurately and quickly detect the location and category of tomato pests and diseases.

For time series data generated in agricultural IoT systems, some researchers have focused on anomaly detection of sensor network data. Several papers have offered specifics on anomalies in smart ecosystems (Cook et al., 2019; Hasan et al., 2019; Park et al., 2021). In smart agriculture scenarios, agricultural IoT devices are often exposed to harsh conditions that can lead to failure of the device itself, compromised communications, or malicious attacks, which can lead to data anomalies. Adkisson et al. (2021) proposed an anomaly detection model for smart farming using an unsupervised autoencoder machine learning model. The model used an autoencoder to encode and decode the data, and anomalous data generated a high reconstruction loss value. Ultimately, the test data was determined to be anomalous based on a threshold value. Abdallah et al. (2021) applied autoregressive integrated moving average (ARIMA) and LSTM model to a smart agricultural system and specialized models based on sensor constraints. The transfer learning strategies were introduced into the models to improve the prediction. Anomaly detection for time series data can also be applied in crop harvesting. Moso et al. (2021) proposed a powerful ensemble-based approach for anomaly detection, which was mainly used for data streams generated in smart agriculture. This technology can be applied to crop data sets and identify anomalies that affect crop harvest.

In summary, time series anomaly detection based on deep learning has obtained excellent results, and GAN for anomaly detection has been continuously explored. Various deep learning models have been introduced to solve the problem of time series data for smart agriculture. As the amount and dimensionality of data increase, existing smart agriculture anomaly detection models are unable to handle the data. GAN has been introduced to various fields to process multidimensional time series data with better results. Therefore, we introduce GAN into smart agriculture for anomaly detection and specialize model structure for smart agriculture data characteristics. However, the existing studies of GAN for anomaly detection are limited by the problem of generator inversion and the reconstruction error is calculated in a simple way. To address the problem of GAN for anomaly detection, we design a new architecture and an error calculation method to improve the anomaly detection performance.

3Proposed approach

In this section, we first describe a novel GAN-based anomaly detection model and focus on how it uses an adversarial learning architecture by considering the dependencies between time series data. Then, the internal detailed architecture of the model is shown, which includes the encode–decoder in the generator and the structure of the discriminator. These designs have strong relevance to the goals of improving reconstruction effects and reducing error computation time. To better learn the data distribution, a multi-channel attention mechanism is embedded in the encoder and decoder, which can further improve the reconstruction effect. Finally, we introduce a new error calculation method in this model, which can describe the errors more rationally and improve the detection results effectively.

The core idea of the reconstruction-based anomaly detection method is to encode a data point (time series data in this model) and then decode the encoded data point to reconstruct the data. Anomalous data loses a lot of information during the encoding–decoding process, because what the model should learn is how to reconstruct normal data. Thus, a normal trained model cannot reconstruct abnormal data in the same way as normal data. Large reconstruction errors will arise in the process of reconstructing anomalous data. This means that the reconstructed data has a large difference from the original data. In this paper, GAN model is used to model the data in an attempt to learn the normal distribution of the data.

The basic task of anomaly detection is to identify whether the data to be tested conforms to the distribution of normal data, and data that do not conform to the normal distribution are defined as anomalous (Chalapathy and Chawla, 2019; Kwon et al., 2019). In this work, the completed trained GAN is used for anomaly detection. The test samples are processed in the same data processing manner and then fed into the model in an attempt to reconstruct them. The anomaly score is calculated using a jointly trained generator and discriminator, which consists of the output of the discriminator and the reconstruction error of the generator. For reconstruction errors, we use a new calculation to detect potential anomalies in the data (more details will be described in “Anomaly Detection”).

3.1The proposed GAN framework

The general architecture of our proposed model is shown in Figure 1. The first objective of this model is to learn the normal distribution of a given data set by means of adversarial training. Previous studies have taken random normal vectors in the latent space Z and inputted them into the generator for training (Li et al., 2019; Bashar and Nayak, 2020). The trained generator is able to implicitly capture the multivariate distribution of the training data and learn the mapping of random data to normal data. However, the error between the reconstructed time series data and the real data needs to be calculated in anomaly detection. In order to reconstruct the data, it is necessary to find the random normal data corresponding to the reconstructed data at each calculation. This process requires the inversion of the generator and is time-consuming and computationally resource intensive. To solve this problem, an encoder–decoder architecture is designed as a generator, in which the encoder learns the mapping of normal data to latent vector in the latent space and optimizes the computation time of reconstruction errors.Sensors in smart agricultural systems perform continuous measurement tasks to detect changes in the environment. Therefore, they generate a large amount of multivariate time series data. We use LSTM as the basic model of generator and discriminator to deal with complex multidimensional time series data. For the characteristics of multidimensional time series data, the data streams are not processed separately. The entire dataset is processed concurrently to capture potential interactions between variables. Multivariate time series data are divided into subsequences that are fed into the model through a sliding window mechanism. We set the window size as the super parameter of the model to determine the optimal window length, which can capture the data distribution in different situations according to the characteristics of different data sets. Her

4Results

Figure 2 shows the generator and discriminator loss curves over 50 epochs.

Figure 2 shows improvement in terms of energy as the both discriminator and generator are displayed.

Figure 3 shows the temporal comparison of real and synthetic sequences.

Figure 3 shows how GAN was able to generate a synthetic sample that is statistically the same with real sample. This can be used to replace sudden change in temperature,humidity and voltage,thereby helping in optimization of energy.

5Conclusion

In this paper, we proposed a GAN-based anomaly detection model for multidimensional time series data generated in smart agricultural IoT. This model used the GAN architecture to learn the distribution patterns of normal data and applied reconstruction methods for anomaly detection. Considering the time dependence of time series data and the potential correlation between multidimensional variables, an improved Enhanced LSTM network to form the basis of the GAN was considered in this model. For the problem of generator inversion, the encoder–decoder architecture was adopted as the generator structure of GAN. The co-training of the encoder and decoder eliminated the inversion of the generator for test samples. This effectively reduced the computation time and met the demand for real-time anomaly detection. The performance of anomaly detection has been improved by the use of encoder–decoder architecture. To further improve the reconstruction effect, the encoder–decoder architecture incorporates an attention mechanism, which can extract weights in the time and feature dimensions to help the model reconstruct the samples. For anomaly detection, a new anomaly score calculation was proposed, which took the coupled result of the point-wise difference error and the curve similarity metric as the reconstruction error. The point-wise error and curve similarity were considered together to better fit the definition of realistic error.

Experiments were designed on three smart agriculture-related data sets and these results were compared with four previous anomaly detection algorithms to verify the effectiveness and superiority of the algorithm. The results proved that our method outperformed other methods in most of the metrics, and the error calculation method proposed in this paper can better detect the anomaly. Not only that, our proposed model obtained superior experimental metrics on high-dimensional smart agriculture data sets, which also reflects that GAN can better handle high-dimensional time series data. With the continuous development of smart agriculture, the dimensionality and quantity of data will grow. The model proposed in this paper also provides a new and useful insight for the anomaly detection of high-dimensional time series data in smart agriculture. However, the time window size setting needed to be considered primarily, which may be the reason why this model is lower than one of the counterpart models in terms of F1 score. Thus, how to choose time windows in time series is an important research topic, and the calculation method proposed in this paper also has a strong correlation with the size of time windows, we will continue our work on anomaly calculation methods and time windows in the future.

6Recommendations

1. 1. This study should be encouraged in our rural areas,in order to have a better output of agricultural produce.
2. 2. Government should encourage farmers by procuring GAN model.

References

1. [1]Abdallah M, Lee WJ, Raghunathan N, Sutherland JW, Bagchi S (2021). Anomaly detection through transfer learning in agriculture and manufacturing IoT systems. arXiv. [Preprint]. https://doi.org/10.48550/arXiv.2102.
2. [2]Adhikari R, Agrawal RK (2013). An introductory study on time series modeling and forecasting. arXiv. [Preprint]. https://doi.org/10.48550/arXiv.1302.
3. [3]Adkisson M, Kimmell JC, Gupta M, et al. (2021). Autoencoder-based anomaly detection in smart farming ecosystem, In 2021 IEEE International Conference on Big Data; December 15, 2021.
4. [4]Aggarwal CC (2017). An Introduction to Outlier Analysis.Cham: Springer.
5. [5]Akçay S, Atapour-Abarghouei A, Breckon TP (2019). Skip-ganomaly: skip connected and adversarially trained encoder-decoder anomaly detection. in 2019 International Joint Conference on Neural Networks; July 14, 2019.
6. [6]Bashar MA, Nayak R (2020). TAnoGAN: time series anomaly detection with generative adversarial networks. in 2020 IEEE Symposium Series on Computational Intelligence; December 1, 2020.
7. [7]Berndt DJ, Clifford J (1994). Using dynamic time warping to find patterns in time series. KDD Workshop10, 359–370.
8. [8]Blake C, Merz CJ (1999). Data from: The third international knowledge discovery and data mining tools competition. UCI repository of machine learning databases. Available at
9. [9]Cao X, Yao Y, Li L, Zhang W, An Z, Zhang Z, et al. (2021). iGrow: A smart agriculture solution to autonomous greenhouse control. arXiv. [Preprint]. https://doi.org/10.48550/arXiv.2107.05464
10. [10]Chalapathy R, Chawla S (2019). Deep learning for anomaly detection: a survey. arXiv. [Preprint].
11. [11]Cook AA, Mısırlı G, Fan Z (2019). Anomaly detection for IoT time-series data: A survey. IEEE Internet Things J. 7, 6481–6494. https://doi.org/10.1109/JIOT.2019.2958185
12. [12]Farooq MS, Riaz S, Abid A, Abid K, Naeem MA (2019). A survey on the role of IoT in agriculture for the implementation of smart farming. IEEE Access. 7, 156237–156271. https://doi.org/10.1109/ACCESS.2019.2949703
13. [13]Garg S, Pundir P, Jindal H, Saini H, Garg S (2021). Towards a multimodal system for precision agriculture using IoT and machine learning. in 2021 12th international conference on computing communication and networking technologies (ICCCNT); July 6, 2021.
14. [14]Geiger A, Liu D, Alnegheimish S, et al. (2020). TadGAN: time series anomaly detection using generative adversarial networks. in 2020 IEEE International Conference on Big Data; December 10, 2020.
15. [15]Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, et al. (2014). Generative Adversarial Nets. Advances in Neural Information Processing Systems.Red Hook: NY Curran.
16. [16]Hasan M, Islam MM, Zarif MII, Hashem MMA (2019). Attack and anomaly detection in IoT sensors in IoT sites using machine learning approaches. Internet Things. 7:100059. https://doi.org/10.1016/j.iot.2019.100059
17. [17]Hashimoto M, Ide Y, Aritsugi M (2021). Anomaly detection for sensor data of semiconductor manufacturing equipment using a GAN. Procedia Comp. Sci. 192, 873–882. https://doi.org/10.1016/j.procs.2021.08.090
18. [18]Hawkins DM (1980). Identification of Outliers. Vol. 1.London: Chapman and Hall.
19. [19]Jiang ZK (2020). A Deep Learning Framework for IoT Time-Series Data Analysis and Applications. Master’s Thesis. Changsha (Hunan): Hunan University.
20. [20]Khalil RA, Saeed N, Masood M, Fard YM, Alouini MS, Al-Naffouri TY (2021). Deep learning in the industrial internet of things: potentials, challenges, and emerging applications. IEEE Internet Things J. 8, 11016–11040. https://doi.org/10.1109/JIOT.2021.3051414
21. [21]Kwon D, Kim H, Kim J, Suh SC, Kim I, Kim KJ (2019). A survey of deep learning-based network anomaly detection. Clust. Comput. 22, 949–961. https://doi.org/10.1007/s10586-017-1117-8
22. [22]Li D, Chen D, Goh J, Ng SK (2018). Anomaly detection with generative adversarial networks for multivariate time series.arXiv. [Preprint]. https://doi.org/10.48550/arXiv.1809.04758
23. [23]Li D, Chen D, Jin B, Shi L, Goh J (2019). MAD-GAN: multivariate anomaly detection for time series data with generative adversarial networks, in International Conference on Artificial Neural Networks; September 17, 2019.
24. [24]Li Y, Swersky K, Zemel R (2015). Generative moment matching networks. in International Conference on Machine Learning. 1718–1727. PMLR.
25. [25]Li W, Wang D, Li M, Gao Y, Wu J, Yang X (2021). Field detection of tiny pests from sticky trap images using deep learning in agricultural greenhouse. Comput. Electron. Agric. 183:106048. https://doi.org/10.1016/j.compag.2021.106048
26. [26]Liu J, Wang X (2020). Tomato diseases and pests detection based on improved Yolo V3 convolutional neural network. Front. Plant Sci. 11:898. https://doi.org/10.3389/fpls.2020.00898
27. [27]Malhotra P, Ramakrishnan A, Anand G, et al. (2016). LSTM-based encoder-decoder for multi-sensor anomaly detection. arXiv. [Preprint]. https://doi.org/10.48550/arXiv.1607.00148
28. [28]Moso JC, Cormier S, DeRunz C, Fouchal H, Wandeto JM, et al. (2021). Anomaly detection on data streams for smart agriculture. Agriculture11:1083. https://doi.org/10.3390/agriculture11111083
29. [29]Nassif AB, Talib MA, Nasir Q, Dakalbab FM (2021). Machine learning for anomaly detection: a systematic review. IEEE Access. 9, 78658–78700. https://doi.org/10.1109/ACCESS.2021.3083060
30. [30]Pang G, Shen C, Cao L, Hengel AVD (2021). Deep learning for anomaly detection: A review. ACM Comp. Surveys54, 1–38. https://doi.org/10.1145/3439950
31. [31]Park H, Park D, Kim S (2021). Anomaly detection of operating equipment in livestock farms using deep learning techniques. Electronics10:1958. https://doi.org/10.3390/electronics10161958
32. [32]Pundir M, Sandhu JK (2021). A systematic review of quality of service in wireless sensor networks using machine learning: recent trend and future vision. J. Netw. Comput. Appl. 188:103084. https://doi.org/10.1016/j.jnca.2021.103084
33. [33]Rafii F, Kechadi T (2019). Collection of historical weather data: issues with missing values, in Proceedings of the 4th International Conference on Smart City Applications; October 2, 2019.
34. [34]Salimans T, Goodfellow I, Zaremba W, Cheung V, Radford A, Chen X (2016). Improved techniques for training gans. arXiv. [Preprint]. https://doi.org/10.48550/arXiv.1606.03498
35. [35]Schlegl T, Seeböck P, Waldstein SM, Schmidt-Erfurth U, Langs G (2017). Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. in International Conference on Information Processing in Medical Imaging; June 25, 2017.
36. [36]Taranveer (2019). Data from: The readings with a time span of 1 minute of house appliances in kW from a smart meter and weather conditions of that particular region. Kaggle. Available at: https://www.kaggle.com/taranvee/smart-home-dataset-with-weather-information (Accessed January 10, 2022).
37. [37]USDA-ARS (2016). Data from: The USDA-ARS conservation and production laboratory (CPRL), soil and water management research unit (SWMRU) research weather station. Data.Gov. Available at: https://catalog.data.gov/dataset/data-from-quality-controlled-research-weather-data-usda-ars-bushland-texas (Accessed January 12, 2022).
38. [38]Vilenski E, Bak P, Rosenblatt JD (2019). Multivariate anomaly detection for ensuring data quality of dendrometer sensor networks. Comput. Electron. Agric. 162, 412–421. https://doi.org/10.1016/j.compag.2019.04.018
39. [39]Vyas A, Bandyopadhyay S (2020). Semi-supervised soil moisture prediction through graph neural networks. arXiv. [Preprint]. https://doi.org/10.48550/arXiv.2012.03506
40. [40]Xu Y, Biswal S, Deshpande SR, et al. (2018). Raim: recurrent attentive and intensive model of multimodal patient monitoring data. in Proceedings of the 24th ACM SIGKDD international conference on Knowledge Discovery a