Skip to main navigation Skip to search Skip to main content

Fast Anomaly Detection for IoT Services Based on Multisource Log Fusion

  • Xingguo Jiang
  • , Hong Luo*
  • , Yan Sun
  • , Mohsen Guizani*
  • *Corresponding author for this work
  • School of Computer Science (National Pilot Software Engineering School)
  • Beijing University of Posts and Telecommunications
  • Mohamed Bin Zayed University of Artificial Intelligence

Research output: Contribution to journal › Article › peer-review

16 Scopus citations

Abstract

With the fast development of Internet of Things (IoT), anomaly detection has recently become a common concern for IoT smart applications. The circuit detection mode of all services to detect application anomaly is widely adopted. However, regularly collecting key performance indicators (KPIs) of all services under different clouds is a time-consuming task. Moreover, too many alerts in a short period of time will affect the processing speed of engineers. Recently, some researches of inferring the key service on the call paths can solve the above problems. However, it is still a challenge to locate dynamic and scattered key services accurately. In this article, we propose a service inference-based anomaly detection approach (SIADA), which integrates three sources of logs: 1) call; 2) business; and 3) metric. SIADA leverages the deep graph representation with context-aware multigraph fusion based on a recurrent encoder. This should infer key services and adopt variational autoencoder (VAE) with the flow model to detect multivariate time-series anomalies for key services. We have conducted extensive experiments on the public data set MicroSS. SIADA achieves the best average accuracy of 92% in service inference and the best average F1-score of 0.98 in anomaly detection, which has improved by 12.17% and 6.42% compared with the best SOTA baseline, respectively. Moreover, the total detection time, network transmission, and average alert number are reduced by 42.12%, 81.87%, and 83.5%, respectively.

Original languageEnglish
Pages (from-to)9405-9419
Number of pages15
JournalIEEE Internet of Things Journal
Volume11
Issue number6
DOIs
StatePublished - 15 Mar 2024
Externally publishedYes

Keywords

  • Fast anomaly detection
  • Internet of Things (IoT) service
  • key service inference
  • log fusion
  • service operation

Fingerprint

Dive into the research topics of 'Fast Anomaly Detection for IoT Services Based on Multisource Log Fusion'. Together they form a unique fingerprint.

Cite this