2. The theory behind LogIDS


Intrusion detection, much like any other sectors of the computer security game, is still in its infancy, even if great progress were made in the recent years. But despite these progress, we are still very far from the Hollywood cliché of the network admin being a simple security guard looking at a screen and "seeing" network intrusions that made me laugh so often in various movies. Every seasoned network administrator and security consultant knows that there is no silver bullet for security, and that you can't just stay there and "watch" your network. You have to apply various security tools, apply patches, perform audits, review logs, on top of maintaining the systems and performing operations. But what if this was all about to change? Or at least a good portion of it?

The work around the things presented in this document comes directly from what I've learned and applied in computer security over the last few years and from my previous experiences as a network administrator under different PC-based operating systems (DOS, Windows, OS/2, Novell). LogIDS, LogAgent, ComLog and the SIDTk all come from the same will to increase the overall security of Windows-based networks at an affordable cost in money and human resources, and provide enhanced intrusion detection capabilities than what was previously the "industry standard". All this work has been inspired with various degrees from different topics such as host bastioning and firewalling, intrusion detection, forensics and incident response, honeypots, along with the work of several individuals that would be too long to name here but who pioneered computer security on the UNIX platform years ago. I have tried to bring to the Windows environment the same philosophy that inspire so many *NIX administrators out there.

So, you might ask, what is the common link between these various topics and LogIDS? Well, first of all, I could say that they are all related, in one way or another, at increasing computer and network security. Some of these methods are pro-active (establishing a defence), some are reactive (analysing an incident that occurred) and even educational (honeypot technologies). Even if we could discuss about each of these topics in great details separately, I would prefer to focus here on the "common link" I suggested above. Let's look at protection and detection technologies, like firewalls, antivirus, host and network based IDS, they are good at protecting some aspects of your architecture, but cannot handle the whole security needs all by themselves. Also, each of these tools produces logs that must be manually analyzed and interpreted, which is tedious and always "after the fact". Besides, all of these logs are all separated from each other, and makes it hard to link things together. For one thing, I hope LogIDS will redefine the term "log correlation". On the other hand, forensics and incident response, do not rely completely on, but are greatly advantaged, by the presence of valuable log data, in a condition that is maintained in a place where we can attest of its credibility.

Honeypots, in a sense, are a variation of the forensics and incident response problematic, in the way that it is heavily focused on the analysis of compromised hosts. Where it gets interesting, though, is that the compromised machines where designed to get broken into, which allows for a level of monitoring that is out of the question in a production environment, for technical and legal purposes. One of the great advantages of this setup is that it lets the community learn from real-life incidents, monitoring the activities of unsuspecting black-haters. Even in the highly monitored environment that is a honeypot, it became clear that certain circumstances required some new strategies or some new tools to develop, profiting to the security community as a whole, while learning the mechanics of different real-life attacks. The analysis of a compromised honeypot also involves, but is not limited to, the analysis of various log files, which are generally richer and more diversified than what is met in an average production environment.

LogIDS, along with the other tools mentioned above, are built from the learning of each of these fields. The strategy behind it is to implement various level of security controls, some of which are probably already present in your network, have these security measures deployed on every host on the network, centralise the logs produced by these various tools , and analyse as soon as possible the log data as it arrives on the central host. Some tools provided by SecurIT Informatique Inc. fit in the log evidence producing category of tools (ComLog keeps a log of cmd.exe sessions and the SIDTk, which contains several intrusion detection modules that produces logs when invalid items are found). LogAgent specializes in monitoring and forwarding of ASCII log files, like the one produced by a great number of security tools around, along with the events from the Event Viewer, all of this in real-time (or as close as it can get). This allow for the easy integration of other security solutions not covered by the tools I provide (and there are many!) that could enhance your network security, such as network-based IDS like Snort, a personal firewall, an antivirus, and just about any other kind of security tool I cannot think of here.

This gives us a scenario where we have a good deal of log information to treat in order for all this to make sense. Some of this data are mere warnings of attempted illicit activities that were easily dealt with by the reporting software (like a cleaned virus, a blocked port scan), while some of this data was traditionally looked only after a confirmed incident occurred, was produced during intrusions but could not stop it (like Snort, for example), or was simply unavailable (in the case of ComLog, which is very precious in the context of forensics analysis). The combination of using LogAgent with a variety of tools provide you with the ability to have all of this data available as it is produced in a central location (or several), which is much better than, let's say, a weekly analysis. The problem is to present an interface where to make sense of all this heterogeneous data, and effectively detect intrusions in the case that one part of your security architecture drops the ball.

LogIDS is my attempt at solving this problematic. I tried to move away from the traditional interface, where we are simply presented with a list of log entries, that we have to decipher as they fly. LogAgent is a simple version of this crude log display interface, but let's be frank, most of all the log analysis software (real-time or not) are more or less a variation of the same interface of log lines after log lines. Some of them will let you click on a log to enter in the details and link it to a database, but in my opinion, even if I think these features are mildly cool, I think they are cumbersome to navigate between the screens when time is critical and you have to wait after the web server's response (who may not be able to reach you because you're flooded, for example). And most of all, most of these tools are single-application oriented only, meaning they are designed to monitor either the Event Viewer, either Snort, either ZoneAlarm, but nothing offers a really united view of what is going on.

So I designed an interface that lets you represent at least a logical representation of your network environment, where you have not 1, but as many as you have defined items text windows to display log data. I hope this representation will help to better understand the interactions between the various systems when they are involved in a complex attack. LogIDS also provides with an easy to configure filter module to allow for pretty much all kind of ASCII log files, and a rules module to modify LogIDS behaviour at your will. It is designed specifically so you can adapt it to your environment, giving you all the flexibility to automate some of the analysis aspects with the rules file. Further most, as we will see later, even if an intrusion occurs, LogIDS will still detect it and unless you're either blind, deaf or dead, there should be no doubt in your spirit about if you've been hacked or not when it happens. The combination of several security tool is a great way to eliminate false-positive, or at least to correctly identify them as such, while keeping focus on real-positives. In the case you do get hacked into, the difference now is that you know about it as soon as it occurs, and lets you take rapid action to solve the issue. We will see in the following chapters how to configure LogIDS and how it can help achieve these goals. Also, I thought it was important to have an Open Source version of such a tool, because it would not help LogIDS nor the security community to keep it under shadows: it is clear to me that this tool has potential to grow far more than I alone can conceive. Still, I hope this tool will also be recognized as a key piece in a security architecture in the commercial market, which will help sustain the long term the future development of LogIDS.

1. Introduction
3. Centralising your logs