Improving Security Through Analysis of Log Files IntersectionsAuthor(s): Kazimierz Kowalski | Mohsen Beheshti
Journal: International Journal of Network Security ISSN 1816-353X
Volume: 7; Issue: 1; Start page: 24; Date: 2008;
Keywords: Data mining | intrusion prevention | log files | security architectures
The paper discusses our research in development of general and systematic methods for intrusion prevention. The key idea is to use data mining techniques to discover repeated patterns of system features that describe program and user behavior. Server systems customarily write comprehensive activity logs whose value is useful in detecting intrusion. Unfortunately, production volumes overwhelm the capacity and manageability of traditional approach. This paper discusses the issues involving large-scale log processing that helps to analyze log records. Here, we propose to analyze intersections of firewall log files with application log files installed on one computer, as well as intersections resulting from firewall log files with application log files coming from different computers. Intersections of log files are substantially shorter than full logs and consist of records that indicate abnormalities in accessing single computer or set of computers. The paper concludes with some lessons we learned in building the system.