Since LogIDS is a log-based intrusion detection system, it is important to feed it with logs. Log management is a crucial part of the security game, as it is often the key to understand what is going on within the network. They are also one of the first place to look at when performing forensics analysis, even before looking at the inspected machine's hard drive. Security has traditionally been deployed at the network periphery, such as a firewall and a intrusion detection system at the internet gate, with a DMZ for public servers. But the rate of security incidents, including full-scale network intrusion, is constantly increasing. It should now be very clear that the single combo of firewall and antivirus is not sufficient to the task, even when mixed with network-based intrusion detection, which often generates false positives and false negatives that tend to lessen the network administrator dedication to it. But the security of the internal network is often left unaccounted for, limited probably to user accounts and ACLs on files and directories. One must know that these measures can be circumvented, and thus cannot be fully be relied upon. In fact, it is a well agreed-upon philosophy in the security community that security is best achieved by applying a multi-layer approach, by applying security in depth.
One of the strategies to achieve this is to apply security measures to every host on the network, not just the servers or the gates to the Internet. This is often already done for antivirus, but security could be greatly enhanced by applying personal firewalls to every host, making every host's local network be itself and the various internal servers it interacts with, making every other desktop on the network a part of the external (protected) network. This will help preventing colleagues from hacking each other, or at least provide factual data if an internal network scan occurs. It should also help in reducing junk traffic between the hosts, making network-based intrusion detection systems more efficient and reporting less false-positives. In turn, it could allow for tighter configuration, and also allow less false-negatives. Another idea would be to install Snort on every host, running as a service, in non-promiscuous mode. Cumulated with the tools from the SIDTk, you come up with a pretty solid security configuration deployed on all your hosts for relatively cheap. For optimal results, the OS and applications configuration should be tightened, but that is more complicated/costly, and turning auditing ON on NT/2K/XP systems. If you have other ideas in order to improve this security model, let me know and I'll add it here.
Now, to deal with all these logs, you have to put them all in the same place. You can probably do so by specifying the path to an available share on a remote server, like \\server\share$\ for example. In this case, the share share$ would be the \Log directory that LogIDS is monitoring. But with some software, you do not have the ability modify the destination path of the log file. Even worse, some applications do not specify identification information about the host reporting, like the ip address or the host name, which is vital for LogIDS purposes. Some other tools may not apply a date-time stamp on the log. In the case of the Event Viewer logs, they can't be easily centralised without polling your server at regular intervals, which is not efficient nor effective. In order to solve all these problems, and to ease log management, I created LogAgent, which is now at version 5.0.
LogAgent is available both in Open Source (free) and Pro (commercial) versions. LogAgent 5.0 also comes with the SIDTk, which can be run as independent modules as a Windows service. LogAgent is able to monitor just any kind of ASCII log files (with the exception of IIS, but this can be worked around by using URLScan's log instead) and the Event Viewer logs, converted to ASCII, in real-time. You can also specify if you want to append to your logs information like the ip address, the host name, the username, the date and time, and lets you monitor your logs in real time and forward them to any number of local or remote locations you want. You can also watch your logs on the screen using the console, or send them to a printer. For desktop use, the Pro version can be run as a service. LogAgent can effectively and efficiently (it consumes very little CPU time) manage the logs of all the applications mentioned in this chapter, and even more. I'm curious at hearing what people monitor with LogAgent. By monitoring many log files on your machine with LogAgent, you may end up with duplicate fields on some logs (ip address reported by LogAgent and by the log-generating application, for example). Don't worry with this, LogIDS is specially built for dealing with this with ease.
So, to recapitulate, in order to function and react on new log items, these logs have to be put into the logids\log\ directory. Once processes, the logs are then cleaned from this directory and put into the logids\log\backup\ directory for archiving. Note that log lines subjected to the BACKUP rule with LogAgent 5.0 Pro will also end up in logids\log\backup\, conveniently putting all your logs at the same place even if not all submitted to the console's watch. The easiest way to get your logs into logids\log\ is to share this directory on your network, by making sure the account LogAgent (or your other security applications if sent directly from the app) runs under have sufficient access to this folder to write data, and use the \\servername\sharename UNC convention as a destination for your logs within your security apps or from within LogAgent.
2. The theory behind LogIDS
4. Defining your network map