Data Embedding Scheme for Efficient Program Behavior Modeling With Neural Networks

Abstract

As modern programs grow in size and complexity, the importance of program behavior modeling is emerging in various areas. Because of the large amount of data generated by a target program and the difficulty of runtime analysis, previous works in these areas employ deep learning. However, they did not sufficiently consider the input of a target program, since, in our view, program behavior is a history of computational steps consisting of a function and its input arguments. A naive, intuitive way to embed the value of $x$ as it is in a vector representation creates a tremendously large vector size. Instead, we found that all the values inducing the same runtime behavior can be represented as one identical characteristic value (CV). In this paper, we show that not only can a characteristic value sequence replace the argument input, but it is also efficient to use it as an input vector for a neural network. This efficiency comes from modeling the whole program with multiple LSTM-RNN models and reducing the input space of the neural network. To demonstrate the effectiveness of this replacement, we performed experiments on the problem of program behavior anomaly detection. Our results show that our model achieves better detection performance compared to previous models and similar detection performance even with smaller model sizes. We also provide a visualization of the embedded vectors extracted from the embedding layer in the neural network model to prove that the CV sequence well represents the arguments.