Anomaly detection is the identification of rare events, items, or observations which are suspicious because they differ significantly from standard behaviors or patterns. Anomalies in data are also called standard deviations, outliers, noise, novelties, and exceptions.

In searching data for anomalies that are relatively rare, it is inevitable that the user will encounter relatively high levels of noise that could be similar to abnormal behavior. This is because the line between abnormal and normal behavior is typically imprecise, and may change often as malicious attackers adapt their strategies.[1]

Figure 1. Anomaly Detection

Anomaly Detection is shown in figure 1. Anomaly detection (aka outlier analysis) is a step in data mining that identifies data points, events, and/or observations that deviate from a dataset’s normal behavior. Anomalous data can indicate critical incidents, such as a technical glitch, or potential opportunities, for instance, a change in consumer behavior. Machine learning is progressively being used to automate anomaly detection. [2]

Anomalous data can be brought about by mistakes made by people, instruments, populations naturally deviating from the norm, fraud, unforeseen behavioral changes, or systemic flaws. Depending on the type of data gathering anomalies, one might take into account the three anomaly detection methods and algorithms:

• Unsupervised Clustering
• Supervised Classification
• Semi-supervised Detection [3]

anomaly detection used for:

Now let’s see how anomaly detection can be used in practice.

• Intrusion detection
• Fraud detection
• Health monitoring
• Defect detection

Anomaly detection is identifying data points in data that don’t fit the normal patterns. It can be useful to solve many problems including fraud detection, medical diagnosis, etc. Machine learning methods allow to automate anomaly detection and make it more effective, especially when large datasets are involved. Some of the common ML methods used in anomaly detection include LOF, autoencoders, and Bayesian networks.[4]

References:

1. https://avinetworks.com/glossary/anomaly-detection
2. https://www.anodot.com/blog/what-is-anomaly-detection/
3. https://deepchecks.com/question/what-are-the-three-3-basic-approaches-to-anomaly-detection/
4. https://serokell.io/blog/anomaly-detection-in-machine-learning

Cite this article:

Gokula Nandhini K (2023), Anomaly Detection, Anatechmaz, pp.66