Development of Intrusion Detection System Software

Development of Intrusion Detection System Software INTRODUCTION Heavy reliance on the Internet and worldwide connectivity has greatly increased that can be imposed by attacks plunged over the Internet against systems. It is very difficult to prevent such attacks by the only use of security policies, firewall or other mechanism because system and application software always contains unknown weaknesses or many bugs. In addition, complex, often unforeseen, interactions between software components and or network protocols are continually exploited by attackers. Successful attacks inevitably occur despite the best security precautions. There for intrusion detection system has become an essential part of the system because they can detect the attacks before they inflict widespread damage. Some approaches detect attacks in real time and can stop an attack in progress. Others provide after-the-fact information about attacks and can help repair damage, understand the attack mechanism, and reduce the possibility of future attacks of the same type. More adv anced intrusion detection systems detect never-before-seen, new, attacks, while the more typical systems detect previously seen, known attacks [1]. MOTIVATION The speed of growth of Internet is very fast without any end. With this growth the threat of attacks is also increasing. Because as we all know that theft can be occurred over the Internet from all over the world. So we require a system which can detect the attack or theft before there is some loss of information and reputation of organization or any individual. There are many solutions has been provided by the researchers and from many companies like firewall, intrusion detection system and IPS to stop the attacks. But still it is very hard to detect the attacks like DoS and worm propagation before they widespread, because regularly thousands of attacks are being developed and for a signature based intrusion detection system it is very hard to detect these kinds of new attacks with perfect accuracy. Mostly intrusion detection system generates many false alarms. These false alarms can affect the other processing of the network. If somehow any attacker gets to know that there is an intrusion detection system in the network then, the attacker will want to disable the intrusion detection system. His/her first target will be the intrusion detection system before attacking the network. So there should be proper security policies for deploying the IDS to take proper advantages of it. PROJECT OBJECTIVES Security is the main concern for any network. Every day thousands of attacks are created so that alarms and logs should be generated properly for reducing their effect. intrusion detection system and IPS are mostly used devices for providing these kinds of solutions. But there are many issues like performance and accuracy. So the main objective of the project is to develop a signature based intrusion detection system for DoS attacks with better scalability and performance i.e. intrusion detection system with minimum false alarms and with better throughput. In this study the example of TCP SYN flood attack will be taken for implementing and evaluating the performance and scalability of the developed intrusion detection system. Second Objective of this study is to discuss the policies for implementing the intrusion detection system securely. And these policies shall also be evaluated. Intrusion detection system intrusion detection systems (IDS) are software or hardware systems that automate the process of monitoring the events occurring in a computer system or network, analyzing them for signs of security problems. As network attacks have increased in number and severity over the past few years, intrusion detection systems have become a necessary addition to the security infrastructure of most organizations [2, 48]. There are many different types of intrusion detection system and they can be characterized by different monitoring and analysis approaches. Each approach has different advantages and disadvantages. All approaches can be described in terms of generic process model for intrusion detection systems. Many intrusion detection systems can be described in terms of three fundamental functional components information source, analysis, and response [2]. OVERVIEW Chapter 1 In this chapter we will give a brief introduction of whole project, what is the motivation for selecting this project. What are the main objectives of this project? And what is the main problem which will be considered in this project. Chapter 2 is all about the literature review. In this chapter many different aspects of the intrusion detection system will be discussed like why we require intrusion detection system, different type of intrusion detection system, need for intrusion detection system, about attacks different types of attacks and many other different facts about intrusion detection system which can help to improve the knowledge about intrusion detection system. Chapter 3 will focus on the analysis and designing part of the intrusion detection system. How a computer system can be designed. What s the system engineering and different type’s models will be discussed. CHAPTER 2 NEED FOR INTRUSION DETECTION SYSTEM Internet is carrying more traffic than ever before and still growing in the size without any end. Along with the explosive growth comes an increased threat from Internet related attacks. The Internet allows theft to occur from anywhere of the world [14]. Many threats impact on the operation of your computer network. Natural threats such as flood fire and tornadoes, causes unexpected disruptions. Most companies have well-defined procedure to handle these natural attacks. Security procedures designed to combat hacker attacks, an unsecured network will definitely be attacked. The only question is when the attack will occur [14]. COMPUTER ATTACKS AND VULRANABILITIES intrusion detection systems have been adopted by many organizations because the organizations know that intrusion detection systems are necessary component of the security architectures. But still intrusion detection system is not too much popular, most organizations lack experienced intrusion detection system operators. intrusion detection system can be most effective if the human operates it. But before developing a signature based intrusion detection system the knowledge of the attacks is must. Signatures is a set of rules that sensor uses to detect typical intrusive activities. These rules are based on various criteria i.e. IP protocol parameters, transport protocol parameter and packet data [12]. THE PHASES OF THE ATTACKS Attack can be divided into three different phases. The first phase is defining the goal for attack. The second phase is the reconnaissance attack, also known as the information gathering. After collecting the information the attacker proceed to the third phase, the attacking phase [12]. FIRST PHASE: GOALS OF ATTACK Before attacking a network or system, an attacker sets her goals or objectives. When attacking network the attacker can have various goals: Data manipulation System access Elevated privileges Denying availability of the network resources MOTIVATION Revenge Political activism Financial gain Attackers attempt to disrupt network to discredit the particular organization’s image [12]. RECONNAISSANCE BEFORE THE ATTACK Collecting the information is the attacker’s second step in launching an attack against the network. Successful reconnaissance is also important for successful attack. Attackers use two main mechanisms to collect the information about the network. Public data source Scanning and probing An attacker sometime starts his knowledge search by examining public information available about company. By using these kind of information the attacker can determine that where the business is located, the business partners, the value of the company assets and much more. And through scanning, the attackers use remote reconnaissance to find specific resource on the network. The goal of the information gathering is to pinpoint weak points on the network where an attack is likely to succeed. By pinpointing specific weakness on the network, the attacker can launch an attack in the future that generates minimal traffic or noise on the network. This greatly reduces the likelihood of detection during the actual attack [12]. For example: ping sweep, vertical scan, horizontal attack, DNS query, block scan and many more. THE ACTUAL ATTACK After an attacker maps the network, he researches known vulnerabilities for the system that he detected. The attacker’s goal at this stage is to gain access to resources of the network i.e. Unauthorized data manipulation, system access, or privilege escalation. ATTACK METHODOLOGY Regardless of the motivation or personal preferences, an attacker has several attack methodologies from which to choose [12]: Ad hoc (random) Methodological Surgical strike (lightning quickly) Patient (slow) AD HOC (Random) An ad hoc attack methodology is unstructured. An attacker using this methodology is usually disorganized and those types of attacks frequently fail. It is difficult to comprehensively locate targets on the network. METHODOLOGICAL It provides a well-defined sequence of steps to attack a network. First, the attackers use the reconnaissance to locate the targets. Next the attacker locates the exploits for known vulnerability on the target. Finally when he satisfies with his toolkit he starts attacking system on the target network. SURGICAL STRIKE (Lightning Quick) Many times the attacker uses an automated script against a network. The entire attack is completed in a few seconds. Before the system administrator or security analysts have time to react and make any decision. PATIENT (Slow) It refers to how quickly the attacker executes his attacks. Usually the one uses a patient (slow) methodology to avoid detection. Many intrusion detection systems have difficulty detecting attacks that occurs over long period of time. BACK DOORS Viruses and worms provide a vehicle for an attacker to wreak havoc on your network and potentially the Internet. However, the spread of viruses and worms is much harder to determine in advance. Viruses and worms are much harder to determine in advance. Trojan horse program enables an attacker to establish back door on systems. However Trojan horse requires some type of transport vehicle [12]. DENIAL OF SERVICE TECHNIQUES The purpose of DoS attacks is to deny legitimate access to the network resources. These attacks include everything from simple one-line commands to sophisticated programs written by knowledgeable hackers. There are different types of DoS attacks some of them are- Network resource overload Host resource starvation Out-of-band attacks Distributed attacks NETWORK RESOURCE OVERLOAD One common way to deny the network access is by overloading a common resource necessary for network components to operate. The main common resource that can be attacked in the network bandwidth in several ways generating lots of traffic, distributing the attack across numerous hosts, and using a protocol flaws that amplifies the attack by soliciting help from many different hosts on the target [12]. Example- Smurf and Fraggle attack. HOST RESOURCE STARVATION The resources available at the hosts are also known as the attack point as well. One such resource is the buffer that a host uses to track TCP connections. OUT-OF-BOUNDS ATTACKS The first out-of-bounds attack category uses over-sized packet, it overflows the allocated buffer and causes the system crash. An over-sized packet attack is ping of death. DISTRIBUTED ATTACKS The latest trend in DoS attacks is for an attacker to compromise numerous hosts and then use all these compromised hosts to provide a massive against a specific target. These types of attacks are known as the distributed denial of service attack (DDoS). DISTRIBUTION EFFECT To disrupt the victims communication very badly, the attacker must compromise an agent machine that has more network resources than the victim. Locating and breaking into such a machine may prove difficult, if the target of the attack is well-provisioned site [16]. Distribution brings number of benefits to the attackers: By using distribution techniques, the attacker can multiply the resources on the attacking end, allowing him to deny service to more powerful machines at the target end [16]. To stop a simple DoS attack from a single agent, a defender needs to identify that agent and take some action that prevents it from sending such a large volume of traffic. In many cases, the attack from a machine can be stopped only if the machine’s human administrator, or network operator, takes action. If there are thousands agents participating in the attack, however, stopping any single one of them may provide little benefit to the victim. Only by stopping most or all of them can the DoS effect be palliated [16]. If the attacker choose agents that are spread widely throughout the Internet, attempts to stop the attack are more difficult, since the only point at which all of the attack traffic merges is close to the victim. This point is called aggregation point. Other nodes in the network might experience no telltale signs of the attack and might have difficulty distinguishing the attack traffic from legitimate traffic [16]. In DoS attack executed from a single agent, the victim might be able to recover by obtaining more resources. For example, an overwhelmed Web server might be able to recruit other local servers to help handle the extra load. Regardless of how powerful a single agent might be, the defender can add more capacity until he outstrips the attacker’s ability to generate load. This approach is less effective in defending against DDoS attacks. If the defender doubles his resources to handle twice as many requests, the attacker merely needs to double the number of agents- often an easy task [16]. TCP-SYN ATTACK The SYN-flooding attack is a Distributed denial-of-service method disturbing hosts that run TCP server processes. The attack take benefit of the state retention TCP performs for some time after receiving a SYN segment to a port that has been put into the listen state. The basic idea is to utilize this behavior by causing a host to retain enough state for bogus half-connections that there are no resources to establish new genuine connections [51, 52]. A TCP implementation may allocate to LISTEN state to be entered with either all, some, or none of the pair of IP addresses and port numbers specified by the application. In many common applications like web servers, none of the remote host’s information is pre known or preconfigured, so that a connection can be established with any client whose details are unidentified to the server ahead of time. This type of â€Å"unbound† LISTEN is the goal of SYN flooding attacks due to the way it is typically implemented by operating systems [51, 52]. For success, [51, 52] the SYN flooding attack relies on the victim host TCP implementation’s behavior. In particular, it assumes that the victim allocates state for every TCP SYN segment when it is received and that there is perimeter on the amount of such state than can be kept at any time. The [51, 52] SYN flooding attack does not attempt to overload the networks recourses or the end host memory, but merely attempts to exhaust the backlog of half-open connections associated with the port number. The goal is to send a quick barrage of SYN segments from IP addresses (often spoofed) that will not generate replies to the SYN-ACKs that are produced. By keeping the backlog full of bogus half-opened connections, legitimate requests will be rejected. Three important attack parameters for success are the size of the barrage, the frequency with which barrages2 are generated, and the means of the selecting IP addresses to spoof. Usually, [51, 52] systems implements a parameter to the typical listen () system calls that allows the application to suggest a value for this limit, called the backlog. 1 To be effective, the size of the barrage must be made large enough to reach the backlog. Ideally, the barrage size is no larger than the backlog, minimizing the volume of the traffic the attacker must source. Typical default backlog values vary from half-dozen to several dozen, so the attack might be tailored to the particular value determined by the victim host and application. On machines intended to be servers, especially for a high volume of the traffic, the backlogs are often administratively configured to higher. Another aspect makes both DoS and DDoS attacks hard to handle: Defenses that work well against many other kinds of attacks are not necessarily effective against denial of service. For years, system administrators have been advised to install a firewall and keep its configuration up to date, to close unnecessary ports on all machines, to stay current with patches of operating systems and other important software, and to run intrusion detection system to discover any attacks that have managed to penetrate the outer bastions of defense [16]. Unfortunately, these security measures often will not help against denial of service. The attack can consist of traffic that the firewall finds acceptable. intrusion detection systems are of limited value in dealing with DoS, since, unlike break-ins and thefts, DoS attacks rarely hide themselves [16]. WHAT IS INTRUSION DETECTION SYSTEM? intrusion detection systems gather information from a computer or network of computers and attempt to detect intruders or system abuse. Generally, an intrusion detection system will notify a human analyst of a possible intrusion and take no further action, but some newer systems take active steps to stop an intruder at the time of detection [4]. The goal of intrusion detection is seemingly simple: to detect intrusions. However, the task is difficult, and in fact intrusion detection systems do not detect intrusions at all—they only identify evidence of intrusions, either while they’re in progress or after the fact. Such evidence is sometimes referred to as an attacks â€Å"manifestation.† If there is no manifestation, if the manifestation lacks sufficient information, or if the information it contains is untrustworthy, then the system cannot detect the intrusion [5]. intrusion detection systems are classified into two general types known as signature based and heuristic based. Pfleeger and Pfleeger describe signature-based systems as â€Å"pattern-matching† systems that detect threats based on the signature of the attack matching a known pattern. Heuristic based systems, which are synonymous with anomaly-based systems, detect attacks through deviations from a model of normal behavior [6]. intrusion detection systems that operate on a single workstation are known as host intrusion detection system (HIDS), while those that operate as stand-alone devices on a network are known as NIDS. HIDS monitor traffic on its host machine by utilizing the resources of its host to detect attacks. NIDS operate as a stand-alone device that monitors traffic on the network to detect attacks. NIDS come in two general forms; signature based NIDS and heuristic based NIDS [7]. PROCESS MODEL FOR INTRUSION DETECTION SYSTEM intrusion detection systems can be described in terms of three fundamental functional components [2, 48]: Information Sources the different sources of event information used to determine whether an intrusion has taken place. These sources can be drawn from different levels of the system, with network, host, and application monitoring most common. Analysis the part of intrusion detection systems that actually organizes and makes sense of the events derived from the information sources, deciding when those events indicate that intrusions are occurring or have already taken place. The most common analysis approaches are misuse detection based (signature based) and anomaly detection. Response the set of actions that system takes once it detects intrusions. These are typically grouped into active and passive measures, with active measures involving some automated intervention on the part of the system, and passive measures involving reporting intrusion detection system findings to humans, who are then expected to take action based on those reports. INFORMATION SOURCE The most common way to classify intrusion detection system is to group them by information source. Some intrusion detection systems analyze network packets, captured from network backbones or LAN segments, to find attackers [2]. It can be describe by dividing three different parts. NETWORK BASED INTRUSION DETECTION SYSTEM NIDS are intrusion detection systems that capture data packets traveling on the network media (cables, wireless) and match them to a database of signatures. Depending upon whether a packet is matched with an intruder signature, an alert is generated or the packet is logged to a file or database [8, 48]. Network-based intrusion detection systems often consist of a set of single-purpose sensors or hosts placed at various points in a network. These units monitor network traffic, performing local analysis of that traffic and reporting attacks to a central management console. As the sensors are limited to running the intrusion detection system, they can be more easily secured against attack. Many of these sensors are designed to run in â€Å"stealth† mode, in order to make it more difficult for an attacker to determine their presence and location [2, 48]. HOST INTRUSION DETECTION SYSTEM  or HIDS Host-based intrusion detection systems or HIDS are installed as agents on a host. These intrusion detection systems can look into system and application log files to detect any intruder activity. Some of these systems are reactive, meaning that they inform you only when something has happened. Some HIDS are proactive; they can sniff the network traffic coming to a particular host on which the HIDS is installed and alert you in real time [8, 48]. These types of intrusion detection systems run on host to reveal inappropriate activities on these hosts. The HIDSs are used for detecting the attacks from the inside and outside network. They provide snap shot about the existing system files and connect them to the previous. If the important system files were modified or deleted, the warning is sent to the administrator for inspection. The HIDS example is notice able on the machines with significant task; these machines do not expect the change of their configuration [9, 48]. APPLICATION-BASED INTRUSION DETECTION SYSTEM Application-based intrusion detection systems are a special subset of host-based intrusion detection systems that analyze the events transpiring within a software application. The most common information sources used by application-based intrusion detection systems are the application’s transaction log files. The ability to interface with the application directly, with significant domain or application-specific knowledge included in the analysis engine, allows application-based intrusion detection systems to detect suspicious behavior due to authorized users exceeding their authorization. This is because such problems are more likely to appear in the interaction between the user, the data, and the application [2, 48]. INTRUSION DETECTION SYSTEM ANALYSIS There are two primary approaches to analyzing events to detect attacks: misuse detection and anomaly detection. Misuse detection in which the analysis targets something known to be â€Å"bad†, is the technique used by most commercial systems. Anomaly detection, in which the analysis looks for abnormal patterns of activity, has been, and continues to be, the subject of a great deal of research. Anomaly detection is used in limited form by a number of intrusion detection systems. There are strengths and weaknesses associated with each approach, and it appears that the most effective intrusion detection systems use mostly misuse detection methods with a smattering of anomaly detection components [2, 48]. ANOMALY BASED DETECTION Anomaly detection uses models of the intended behavior of users and applications, interpreting deviations from this â€Å"normal† behavior as a problem. A basic assumption of anomaly detection is that attacks differ from normal behavior. For example, we can model certain users’ daily activity (type and amount) quite precisely. Suppose a particular user typically logs in around 10 Am., reads mail, performs database transactions, takes a break between noon and 1 Pm., has very few file access errors, and so on. If the system notices that this same user logs in at 3 Am., starts using compilers and debugging tools, and has numerous file access errors, it will flag this activity as suspicious. The main advantage of anomaly detection systems is that they can detect previously unknown attacks. By defining what’s normal, they can identify any violation, whether it is part of the threat model or not. In actual systems, however, the advantage of detecting previously unknown attacks is paid for in terms of high false-positive rates. Anomaly detection systems are also difficult to train in highly dynamic environments [5]. MISUSE DETECTION Misuse detection systems essentially define what’s wrong. They contain attack descriptions (or â€Å"signatures†) and match them against the audit data stream, looking for evidence of known attacks. One such attack, for example, would occur if someone created a symbolic link to a UNIX system’s password file and executed a privileged application that accesses the symbolic link. In this example, the attack exploits the lack of file access checks [5, 10]. The main advantage of misuse-based systems is that they usually produce very few false positives: attack description languages usually allow for modeling of attacks at such fine level of detail that only a few legitimate activities match an entry in the knowledge base. However, this approach has drawbacks as well. First of all, populating the knowledge base is a difficult, resource intensive task. Furthermore, misuse based systems cannot detect previously unknown attacks, or, at most, they can detect only new variations of previously modeled attacks. Therefore, it is essential to keep the knowledge base up-to-date when new vulnerabilities and attack techniques are discovered. Figure 2 shows how the misuse detection based intrusion detection system works is [11]. RESPONSE OPTION FOR INTRUSION DETECTION SYSTEM Once intrusion detection systems have obtained event information and analyzed it to find symptoms of attacks, they generate responses. Some of these responses involve reporting results and findings to a pre-specified location. Others involve more active automated responses. Though researchers are tempted to underrate the importance of good response functions in intrusion detection systems, they are actually very important. Commercial intrusion detection systems support a wide range of response options, often categorized as active responses, passive responses, or some mixture of the two [2]. IMPORTANCE OF THE INTRUTION DETECTION SYSTEM Usually we place a burglar alarm on the doors and windows of our home. We are installing an intrusion detection system (intrusion detection system) for our house. The intrusion detection systems used to protect our computer network operate in similar fashion. An intrusion detection system is a software and possibly hardware that detects attacks against our network. They detect intrusive activities that enter into our network. We can locate intrusive activity by examining network traffic, host logs, system calls, and other areas that signal an attack against our network [14]. There are different benefits that an intrusion detection system provides. Besides detecting attacks, most intrusion detection systems also provide some type of response to the attacks, such as resetting TCP connections [14]. DESIRABLE CHARACTERSTICS OF INTRUSION DETECTION SYSTEM There are different characteristics for an ideal intrusion detection system, which are listed below [many references]: An ideal intrusion detection system must run with minimum human supervision. An ideal intrusion detection system must be easy to deploy. An ideal intrusion detection system must be able to detect attacks intrusion detection system must not produce false negative alarms. intrusion detection system must not produce false positive alarms. intrusion detection system must report intrusion as soon as possible after the attacks occur. intrusion detection system must be general enough to detect different types of attacks. An ideal intrusion detection system must be fault tolerant; it must be able to recover from crashes and must restore previous state, either accidental or caused by malicious activities. An ideal intrusion detection system must impose minimal overhead on the system. An ideal intrusion detection system must be configurable to implement the securities policies of the system. THE PERIMETER MODEL AND DoS The perimeter model is an architecture commonly used by today’s organizations to protect critical infrastructures. This security model divides network architectures into two distinct groups; trusted and entrusted. The trusted group is often the finite internal infrastructure, whilst the entrusted group consists of infinite external networks. In this model two types of devices are used; firewall to control the traffic entering and leaving the trusted domain, and intrusion detection system to detect misbehavior of trust with in the trusted area boundary [18]. WHERE IDS SHOULD BE PLACED IN NETWORK TOPOLOGY Depending upon network topology, the intrusion detection system can be positioned one or more places. It’s also depends upon what type of intrusion activities should be detected: internet external or both. For example if the external intrusion activities should be detected, and only one router is connected to the internet, the best place for an intrusion detection system may be just inside the router or firewall. If there are many different paths to the internet, then the intrusion detection system should be placed at every entry point. However, if the internal attacks should be detected then the intrusion detection system should be placed in every network segment 2. Placement of the intrusion detection system really depends upon security policies 3 [8]. Note that more intrusion detection systems mean more work and more maintenance costs. Which defines that what should be protected from the hackers [8]? IDS AGAINST DENIAL-OF-SERVICE ATTACKS (DoS) The goal of a DoS attack is to disrupt some legitimate activity, such as browsing, web pages, an on line radio and many more. The denial of service is achieved by sending message to the target that interferes with its operation and makes it hang, crash, reboot or do useless work [16]. A denial-of-service attack is different in goal, form, and effect than most