During the development of network intrusion detection, there are mainly two schools of thought: intrusion detection based on logs and intrusion detection based on traffic. However, because the data source of intrusion detection based on logs comes from the operation logs of various systems, the log formats are diverse, the standardization level is low, and the completeness of the log record content is also varied. Therefore, it is difficult to standardize intrusion detection products based on logs. Therefore, most security vendors generally focus on intrusion detection products based on traffic. The author will discuss various issues that arise in deployment practice for such detection products.
1. Selection of Traffic Detection Products
After many years of development, network security intrusion detection based on traffic may be the security product with the most types of equipment models among security vendors. There are also some differences in the naming and classification methods of various security vendors, which leads to a wide variety of network intrusion detection devices based on traffic, making it dazzling to the eyes. The author combines the specific needs of the client company and the product catalog of the supplier company, and roughly classifies various products into the following 8 categories: DDoS detection, IDS/IPS, application firewall (WAF), WebIDS, APT detection, threat intelligence detection, situation awareness, and full traffic deep analysis. Below, the author will discuss the characteristics and precautions of various products, for everyone's reference when selecting types.
DDoS defense: It can be used to detect and clean up distributed denial-of-service attack traffic such as SYN_Flood, ACK_Flood, HTTP_GET in the network, and is generally deployed at the outermost layer of the internet access area. However, from the actual deployment and use, the author believes that the local deployment of DDoS defense equipment by most enterprises has little effect. The main reasons are: first, it cannot withstand ultra-large traffic. When ultra-large traffic DDoS attack traffic really reaches the enterprise data center exit, hundreds of megabytes of export bandwidth are already in a congested state, and it is difficult for DDoS defense equipment to have an effect on local cleaning. In early 2019, the famous hacker organization 'Anonymous' threatened to launch DDoS attacks on hundreds of Chinese bank networks. Subsequently, a large bank indeed suffered from attacks exceeding tens of Gbps in a few days, and the effect of the boundary DDoS defense equipment was negligible during the defense process. Second, ordinary traffic does not need to be defended. With the improvement of server performance and the application of load balancing technology, business servers themselves also have the ability to resist ordinary DDoS attack traffic. Third, there is a risk of mistakenly blocking normal business traffic. Although many DDoS defense equipment claim to have intelligent detection and accurate identification capabilities, many enterprises still base their judgments on thresholds in actual application, which leads to mistakenly blocking normal business traffic during periods of sudden business outbursts (such as promotions, flash sales, etc.). Therefore, the author believes that the main role of DDoS defense detection in practical application is to meet compliance requirements.
IDS/IPS: The author believes that IDS can be regarded as the ancestor of traffic-based intrusion detection, once considered as one of the three swords of network security protection. It mainly detects attack behaviors in traffic based on attack features or abnormal behaviors. It is generally believed that IDS operates at layer 4 of the TCP/IP protocol model and can detect attacks such as port scanning, trojans, worms, denial-of-service attacks, buffer overflow attacks, etc. However, with the development of technology, the author finds that many IDS can also detect real-time alerts for SQL injection attacks, CGI access attacks, IIS server attacks, and other violations. It is evident that this also includes many application layer attack detection. However, from the actual detection effect, this application layer detection capability is just slightly better than nothing. IPS is an extension of the IDS to the firewall interdiction function, to make up for the problem that IDS cannot effectively intercept detected attacks.
Web Application Firewall (WAF): A detection product focusing on web application security, which can protect against common OWASP TOP10 attack behaviors such as SQL injection, XSS, webshell, command injection, middleware vulnerabilities, and can intercept external attacks detected.
WebIDS: Also a detection product focusing on web application security, WebIDS does not have interception functions, and there are fewer players in the market. From the author's experience, WebIDS is relatively intuitive in displaying HTTP request packets and response packets of web attacks, and the search function is also relatively convenient.
Threat Intelligence Detection: Compared to traditional IDS products, the initial promotion was mainly by a few threat intelligence vendors. Relying on powerful cloud-based threat intelligence libraries and intelligence generation capabilities, it has added detection methods based on malicious domain names, malicious IP addresses, malicious files, and other IOC (intelligence indicators) to detection products, and can also provide some traceability capabilities.
APT Detection: Compared to traditional IDS, WAF, and other devices, APT detection is generally a comprehensive achievement of various detection capabilities, such as feature-based detection, anomaly-based detection, sandbox detection, web attack detection, threat intelligence detection, context correlation analysis, and attack backtracking. The author personally thinks that APT detection is just a sales buzzword of manufacturers. Compared to advanced persistent threats (APT), it is a bit exaggerated to achieve APT detection only based on traffic. However, the detection capability is also somewhat improved compared to other products. By the way, two points are mentioned: first, traditional IDS products are also integrating threat intelligence libraries. Second, the detection method based on IOC itself has significant problems, such as the author having encountered an IOC indicator based on the reputation of file hash codes, which identified all uploaded image files of the application system as malicious files, leading to a large number of false positives.
Tactical Awareness: The author believes that if APT detection is a buzzword, then tactical awareness is a misdirection, which is fundamentally not the real security situation awareness of enterprises. It is just an addition of a large screen display function on the traditional detection capability, reflecting only the security status of the current traffic mirror. Real tactical awareness must be an integrated summary of various perception device (Sense) basic data for comprehensive analysis and correlation analysis, thereby generating indicators reflecting the current security situation of the enterprise. Of course, even so, the current situation of tactical awareness is not satisfactory. In addition, if the company's budget is limited, there are not enough hands, and the leaders are particularly fond of tactical awareness, purchasing a traffic-based tactical awareness product is also a good choice.
Full Traffic Deep Analysis: A full traffic network analysis device for long-term data storage and retrospective analysis of communication data in the network. Compared to previous products or systems, intrusion detection capability is no longer its most important function; convenient and quick full packet traffic analysis is the main function, which can analyze and decompose data packets at each layer of traffic like Wireshark. Full traffic deep analysis devices are indispensable for confirming and retrospective analysis of attack behaviors.
2. Deployment Location Selection
In practical work, people often have trouble deciding where to deploy traffic detection devices. In many corporate practices, traffic-based intrusion detection devices are generally deployed at the internet exit of the data center. When deploying further, people often do not know how to choose? Here, the author offers a way of thinking, which is to think from the perspective of the attacker, considering the possible attack paths of the attacker, and deploying detection points on the possible paths of attack. The following figure is the author's view from the perspective of the attacker, which shows that there are six possible paths to enter the internal network of the enterprise, including the internet exit of the data center, the third-party external connection area, the office network and the data center exit, and the branch connection area, etc.
For traffic monitoring, corresponding traffic monitoring devices need to be deployed at the boundaries of these areas. In actual deployment, the best deployment point can be selected according to the characteristics of the enterprise network planning, such as deploying traffic monitoring devices at the points where the branches are connected in the case of many branches.
3. Deployment Mode Selection
The main deployment modes of traffic detection devices at present are divided into physical series, logical series, and bypass mirroring deployment. Readers can communicate the deployment mode with the network department according to the characteristics of the traffic detection device. However, the basic principle is that the bypass mirroring deployment mode generally cannot achieve interception functions. Therefore, if DDoS, WAF, IPS, and other devices want to enable interception functions, physical series or logical series methods need to be considered.
From what the author has learned, there are still very few enterprises that dare to connect WAF, IPS, and other devices in series to the network link and enable the interception mode. False interception can cause the greatest risk that the security department has to bear, as a thousand correct and effective interdictions cannot compensate for the impact of a single false interception. Without a good包容 environment and incentive policies, only enabling the monitoring mode is the self-protection mode that the security team has to adopt.
4. Challenges Faced by Traffic Detection Devices
(1) Large Alarm Volume
It's really not known until experienced, and one is startled after experiencing it; the alarm volume of a single device in a day can reach the level of millions, and the number of alarms is beyond the capacity of human analysis. The main reasons for such a large number of alarms are: 1. The quantity of real attacks is very large. A large number of vulnerability scans, crawlers, and other activities in the network lead to a very large background attack traffic. Considering the continuity of business, interception methods are generally rarely adopted in the process of network security protection. This is why the daily analysis workload of security operation personnel is so huge. 2. False alarms from detection devices themselves. In daily operations, whether it is IDS, WAF, or APT products, the false alarm rate is high, and this is something that all detection products need to improve. The author has not found any product that does not have false alarms. 3. Unstandardized development of business systems. In actual monitoring, we have observed that some systems transmit SQL statements as parameters, and some login backends use weak passwords, leading to a large number of alarms.
In response to the problem of massive log volume, existing detection devices all have certain attack classification functions (high-risk, medium-risk, low-risk, compromised, severe, attempted, etc.), which alleviate some workload for security analysts and allow them to focus on high-risk or compromised alerts. In addition, for companies with significant security investments, the author is aware of the following approaches: 1. Many enterprises now adopt self-built big data analysis platforms or SOC methods to centralize the logs of detection devices, then use aggregation, classification, and other analysis methods to deduplicate and merge a large amount of logs to reduce the amount of manual data analysis during operation, but even so, the amount of data after analysis is still large. 2. For false positives, it can be requested that manufacturers optimize the strategy, as the products of manufacturers are generalized products, and some localization improvements are needed for actual use. 3. For the construction of application systems, it can be requested that the system负责人 make system modifications.
(2) Fear of Large-scale Blocking
As mentioned in the text, IPS and WAF have already mentioned that the security team is afraid to enable the blocking mode, as it may cause business interruption. Even though big data analysis, correlation analysis, and other technical means that have been strongly promoted in the industry are not really enabled in the blocking mode. The merit of one million successful blocks cannot compensate for the fault caused by one incorrect block leading to business interruption. Good news does not travel far, while bad news spreads like wildfire; security analysts can only silently bear such pain and analyze real successful attack events from the vast sea of alarm data. In this regard, the author believes that enterprises should allow a certain degree of tolerance for security defense, so that security defense measures can fully exert their effectiveness.
(3) Traffic Encryption
With the gradual increase in corporate security awareness, full-site HTTPS is increasingly becoming the best practice in the industry. This is indeed a good thing, as it has improved the overall security level of enterprises. However, for traffic detection devices, this has undoubtedly added another layer of difficulty to the already complex detection environment. The author is aware of the following approaches: 1. Deploy WAF, webIDS, and other devices after the decryption traffic offloading devices in the data center; the traffic passing through these devices is decrypted data. 2. Implement decryption within detection devices themselves, but this method requires attention to the following points: First, the detection device must be connected in the network chain, and side devices cannot use this method. Second, decryption certificates need to be imported into the detection device, which increases the risk of certificate leakage. Third, self-decrypting by detection devices will greatly consume device performance, and generally requires cluster deployment. Fourth, the encryption and decryption process will increase link latency, affecting the overall throughput of the system, so it should be used with caution for high-concurrency business.
The above-mentioned are all traditional data centers. For the currently widely promoted cloud computing scenarios, due to the fact that traditional WAF, IDS, and other box devices cannot be connected to the cloud, traditional solutions also face significant challenges. This is something the author is exploring as well.
(4) Incomplete Deployment Locations
With the continuous development of corporate business, the export boundaries of corporate data centers will become more and more numerous and complex. For example, there are operator专线, internet VPN专线, and ordinary internet access for third-party external connections. The enterprise will also open various access interfaces for different branches. At the same time, the penetration attack strategies of attackers are also expanding, with the emergence of supply chain, third-party jump board attacks, and other methods. The traditional way of thinking that the internet positive export of the data center is the only defense deployment point has become outdated. Under the condition that the budget input allows, it is still necessary to increase the deployment density of traffic detection devices to increase the detection probability of attacks. At the same time, it is necessary to ensure that the traffic is fully covered, and the traffic of devices at the same deployment location is consistent. One device with full traffic is more reliable than ten devices with incomplete traffic. If necessary, it can be used to check the coverage of the traffic through the asset list to see if it meets the requirements.
(5) Loss of source address
In many network security monitoring devices, the source IP address detected is no longer the real client IP address, but the enterprise internal network address or CDN address. This is mainly because the traffic is NAT converted when passing through CDN or other internal network devices (firewalls, load balancers, etc.). The solutions I know mainly include: 1. Use the CDN's origin transmission method to pass the real source address of the client. 2. In the enterprise network, try to use the routing forwarding mode instead of the NAT forwarding mode. 3. If NAT forwarding mode is used, the source IP address can be passed in using the HTTP protocol's X-Forwarded-For.
(6) The problem of heterogeneous deployment
In the practice of network security detection, there is often a problem of heterogeneous deployment, mainly thinking about preventing the failure of a single device's detection capability from leading to the missed reporting of threats. In my opinion, this approach is not problematic, but in specific practice, some units may adopt a single-point heterogeneous deployment model. This model still has the coexistence of false positives and false negatives, and although heterogeneous deployment has some complementarity, the effect is not significant. Finally, the vast majority of alarms seen at the same location are similar, and the few differences may even be false positives. Therefore, my suggestion is that if heterogeneous deployment is to be adopted, it should also be a multi-point in-depth deployment model. During the attack process, the attack methods adopted by attackers at different network locations are different. For example, at the perimeter, it may be an attack using the Weblogic deserialization vulnerability, while inside the internal network, it may be an attack using host vulnerabilities for buffer overflow, weak password brute force, and other attacks.
In general, from the perspective of manufacturers or clients, security detection products based on traffic are relatively easy to implement and deploy solutions. In the future, there will definitely be more products coming into the market and being used in corporate security operations, such as the recently emerged data security products based on traffic.
The original author of this article is shengl99, and this article belongs to the FreeBuf Original Reward Program. Reproduction is prohibited without permission.
评论已关闭