آشکار سازی کد مخرب ناشناخته توسط تکنیک های دسته بندی روی الگوهای OpCode

ABSTRACT Detecting unknown malicious code by applying classification techniques on OpCode patterns

Modern computer and communication infrastructures are highly susceptible to various types of attacks. A common method of launching these attacks is by means of malicious software (malware) such as worms, viruses, and Trojan horses, which, when spread, can cause severe damage to private users, commercial companies and governments. The recent growth in high-speed Internet connections enable malware to propagate and infect hosts very quickly, therefore it is essential to detect and eliminate new (unknown) malware in a prompt manner [1].

Anti-virus vendors are facing huge quantities (thousands) of suspicious files every day [2]. These files are collected from various sources including dedicated honeypots, third party providers and files reported by customers either automatically or explicitly. The large amount of files makes efficient and effective inspection of files particularly challenging. Our main goal in this study is to be able to filter out unknown malicious files from the files arriving to an anti-virus vendor every day. For that, we investigate the approach of representing malicious files by OpCode expressions as features in the classification task.

Several analysis techniques for detecting malware, which commonly distinguished between dynamic and static, have been proposed. In dynamic analysis (also known as behavioral analysis) the detection of malware consists of information that is collected from the operating system at runtime (i.e., during the execution of the program) such as system calls, network access and files and memory modifications [3, 4, 5, 6, 7]. This approach has several disadvantages. First, it is difficult to simulate the appropriate conditions in which the malicious functions of the program, such as the vulnerable application that the malware exploits, will be activated. Secondly, it is not clear what is the required period of time needed to observe the appearance of the malicious activity for each malware.

In static analysis, information about the program or its expected behavior consists of explicit and implicit observations in its binary/source code. The main advantage of static analysis is that it is able to detect a file without actually executing it and thereby providing rapid classification [8].

Static analysis solutions are primarily implemented using the signature-based method which relies on the identification of unique strings in the binary code [2]. While being very precise, signature-based methods are useless against unknown malicious code [9]. Thus, generalization of the detection methods is crucial in order to be able to detect unknown malware before its execution. Recently, classification algorithms were employed to automate and extend the idea of heuristic-based methods. In these methods the binary code of a file is represented, for example, using byte sequence (i.e., byte n-grams), and classifiers are used to learn patterns in the code in order to classify new (unknown) files as malicious or benign [1, 10]. Recent studies, which we survey in the next section, have shown that by using byte n-grams to represent the binary file features, classifiers with very accurate classification results can be trained, yet there still remains room for improvement.

In this paper, which is an extended version of [11], we use a methodology for malware categorization by implementing concepts from the text categorization domain, as was presented by part of the authors in [12]. While most of the previous studies extracted features which are based on byte n-grams [12, 13], in this study, we use OpCode n-gram patterns, generated by disassembling the inspected executable files, to represent the files. Unlike byte sequence, OpCode expressions, extracted from the executable file, are expected to provide a more meaningful representation of the code. In the analogy to text categorization, using letters or sequences of letters as features is analogous to using byte sequences, while using words or sequences of words is analogous to the OpCode sequences.

Another important aspect when using binary classifiers for the detection of unknown malicious code is the imbalance problem. The imbalance problem refers to scenarios in which the proportions of the classes are not equal. Previous studies presented evaluations based on test collections having similar proportions of malicious and benign files in the test collections. These proportions do not reflect real-life situations in which malicious code is significantly lower than 50% and therefore might report optimistic results. As a case in point, a recent McAfee survey [14] indicates that about 4% of search results from the major search engines on the web contain malicious code. Additionally, Shin et al. [15] found that above 15% of the files in the KaZaA network contained malicious code.

We rigorously evaluate the framework that is suggested in this paper, using a test collection containing more than 30,000 files, in order to determine the optimal settings of the framework. Additionally, we investigate the imbalance problem and evaluate through various malicious-benign proportions, the best settings for a training set given a test set.

Another aspect in the maintenance of such a framework is the importance of updating the training set with new known malicious files. This is intuitively important, because the purpose of malicious files changes over time and accordingly the patterns within the code. Moreover, these malicious files are written in varying frameworks which result in differing patterns. However, it is not clear to what extent it is essential to retrain the classifier with the new files. For this purpose we designed a chronological experiment, based on a dataset including files from the years 2000 to 2007, trained each time on files untill year k and tested on the following years.

The rest of the paper is organized as follows. We begin in section 2 with a survey of previous relevant studies. Section 3 describes the methods we used, including concepts from text categorization, data preparation, and classifiers. In sections 4 and 5 we present the evaluation and the evaluation results. Lastly, section 6 discusses the results and future work.