[2J6-GS-2] Machine learning: Advancement reinforcement learning (2)

[2J6-GS-2-02] Development of Abnormality Predictor Prediction Detection Method Using Bayesian Inverse Reinforcement Learning

〇Dinesh Dinesh^1,2, Tomoaki Kimura¹, Masaru Sogabe², katsuyoshi Sakamoto¹, Koichi Yamaguchi¹, Tomah Sogabe^1,2 (1. UEC, 2. Grid Inc.)

Keywords:Inverse reinforcement leanring, Gauss regression

Anomaly detection is widely applied in a variety of domains, involving, for instance, smart home systems, network traffic monitoring, IoT applications, and sensor networks. In a safety-critical environment, it is crucial to have an automatic detection system to screen the streaming data gathered by monitoring sensors and report abnormal observations if detected in real-time. Oftentimes, stakes are much higher when these potential anomalies are intentional or goal-oriented. We assume that abnormality detection is a great challenge, especially without labels, to maximize the confidence level of the decision and minimize the stopping time concurrently. We propose an end- to-end framework for sequential anomaly detection using inverse reinforcement learning (IRL), whose objective is to determine the decision-making agent’s underlying function which triggers agent detection behavior. The proposed method takes the sequence of state of a target source and other meta information as input. The agent’s normal behavior is then understood by the reward function, which is inferred via a Bayesian approach for IRL. We use a neural network to represent a reward function. We firstly checked the Bayesian IRL reward using a Gym classic game environment and We also present results using the Numenta Anomaly Benchmark (NAB), a benchmark containing real-world data streams with labeled anomalies.