Deep content inspection

Deep Content Inspection (DCI) is a form of network filtering that examines an entire file or MIME object as it passes an inspection point, searching for viruses, spam, data loss, key words or other content level criteria. Deep Content Inspection is considered the evolution of Deep Packet Inspection with the ability to look at what the actual content contains instead of focusing on individual or multiple packets. Deep Content Inspection allows services to keep track of content across multiple packets so that the signatures they may be searching for can cross packet boundaries and yet they will still be found. An exhaustive form of network traffic inspection in which Internet traffic is examined across all the seven OSI ISO layers, and most importantly, the application layer.^[1]

Background

Traditional inspection technologies are unable to keep up with the recent outbreaks of widespread attacks.^[2] Unlike shallow inspection methods such as Deep Packet Inspection, where only the data part (and possibly also the header) of a packet are inspected, DCI-based systems are exhaustive, such that network traffic packets are reassembled into their constituting objects, un-encoded and/or decompressed as required, and finally presented to be inspected for malware, right-of-use, compliance, and understanding of the traffic’s intent. If this reconstruction and comprehension can be done in real-time, then real-time policies can be applied to traffic, preventing the propagation of malware, spam and valuable data loss. Further, with DCI, the correlation and comprehension of the digital objects transmitted in many communication sessions leads to new ways of network performance optimization and intelligence regardless of protocol or blended communication sessions.

Historically, DPI was developed to detect and prevent intrusion. It was then used to provide Quality of Service where the flow of network traffic can be prioritized such that latency-sensitive traffic types (e.g., Voice over IP) can be utilized to provide higher flow priority.

New generation of Network Content Security devices such as Unified Threat Management or Next Generation Firewalls (Garner RAS Core Research Note G00174908) use DPI to prevent attacks from a small percentage of viruses and worms; the signatures of these malware fit within the payload of a DPI’s inspection scope. However, the detection and prevention the new generation of malware such as Conficker and Stuxnet is only possible, through the exhaustive analysis provided by DCI.

The Evolution of DPI Systems

Computer networks send information across a network from one point to another, the data (sometimes referred to as the payload) is ‘encapsulated’ within an IP packet, which looks as follows:

Sample encapsulation of application data from UDP to a Link protocol frame

_{*The IP Header provides address information - the sender and destination addresses, while the TCP/UDP Header provided other pertinent information such as the port number, etc.}

As networks evolve, inspection techniques evolve; all attempting to understand the payload. Throughout the last decade there have been vast improvements including;

Packet Filtering

Historically, inspection technology examined only the IP Header and the TCP/UDP Header. Dubbed as ‘Packet Filtering’, these devices would drop sequence packets, or packets that are not allowed on a network. This scheme of network traffic inspection was first used by firewalls to protect against packet attacks.

Stateful Packet Inspection

Stateful packet inspection was developed to examine header information and the packet content to increase source and destination understanding. Instead of letting the packets through as a result of their addresses and ports, packets stayed on the network if the context was appropriate to the networks’ current ‘state’. This scheme was first used by Checkpoint firewalls and eventually Intrusion Prevention/Detection Systems.

Deep Packet Inspection: Implementations, Limitations and Typical Work Arounds

Deep Packet Inspection is currently the predominant inspection tool used to analyze data packets passing through the network, including the headers and the data protocol structures. These technologies scan packet streams and look for offending patterns.

To be effective, Deep Packet Inspection Systems must ‘string’ match Packet Payloads to malware signatures and specification signatures (which dictate what the request/response should like) at wire speeds. To do so, FPGAs, or Field Programmable Gateway Arrays, Network Processors, or even Graphics Processing Units (GPUs)^[3] are programmed to be hardwired with these signatures and, as such, traffic that passes through such circuitry is quickly matched.

While using hardware allows for quick and inline ‘matches, DPI systems have the following limitations including;

Hardware limitations:, since DPI systems implement their pattern matching (or searches for ‘offending’ patterns) through hardware, these systems are typically limited by:

• The number of circuits a high-end DPI chip can have; as of 2011, this of a high end DPI system can, optimally, process around 512 request/response per session.
• The memory available for pattern matches; as of 2011, high-end DPI systems are capable of matches of up to 60,000 unique signatures

Payload limitations: Web applications communicate content using binary-to-text encoding, compression (zipped, archived, etc.), obfuscation and even encryption. As such payload structure is becoming more complex such that straight ‘string’ matching of the signatures is no longer sufficient. The common workaround is to have signatures be similarly ‘encoded’ or zipped which, given the above ‘search limitations’, cannot scale to support every application type, or nested zipped or archived files.

Deep Content Inspection

First Generation - Secure Web Gateway or Proxy-based Anti-Virus Solutions

Proxies have been deployed to provide internet caching services to retrieve objects and then forward them. Consequently, all network traffic is intercepted, and potentially stored. These graduated to what is now known as secure web gateways, proxy-based inspections retrieve and scans object, script, and images.

Deep Content Inspection, the process through which network content is exhaustively examined, can be traced back as early as 2006, when the open-source, cross-platform antivirus software ClamAV provided support for caching proxies, Squid and NetCache. Using the Internet Content Adaptation Protocol (ICAP), a proxy will pass the downloaded content for scanning to an ICAP server running an anti-virus software. Since complete ‘objects’ were passed for scanning, proxy-based anti-virus solutions are considered the first generation of DCI.

This approach was first pioneered by BlueCoat, WebWasher and Secure Computing Inc. (now McAfee), eventually becoming a standard network element in most enterprise networks.

Limitations: While proxies (or secure web gateways) provide in-depth network traffic inspection, their use is limited as they:

• require network reconfiguration which is accomplished through – a) end-devices to get their browsers to point to these proxies; or b) on the network routers to get traffic routed through these devices
• are limited to web (http) and ftp protocols; cannot scan other protocols such as e-mail
• and finally, proxy architectures which are typically built around Squid, which cannot scale with concurrent sessions, limiting their deployment to enterprises.

Second Generation – Gateway/Firewall-based Deep Content Inspection

The Second generation of Deep Content Inspection solutions were implemented in firewalls and/or UTMs. Given that network traffic is choked through these devices, full deep content inspection is possible. However, given the expensive cost of such operation, this feature was applied in tandem with a DPI system and was only activated on a-per-need basis or when content failed to be qualified through the DPI system. This approach was first pioneered by NetScreen Technologies Inc. (acquired by Juniper Networks Inc.) and Fortinet Inc..

Third Generation – Transparent Deep Content Inspection

The third, and current, generation of Deep Content Inspection solutions are implemented as fully transparent devices that perform full application level inspections at wire speed in order to understand the communication session’s intent¬¬¬—in entirety— by scanning both the handshake and payload. Once the digital objects (executables, images, JavaScript’s, .pdfs, etc.) in a Data-In-Motion session can be comprehended, many new insights into the session can be gained not only with simple pattern matching and reputation search, but also with unpacking, and behaviour analysis. This approach was first pioneered by Wedge Networks Inc..

Content

Content based focus, instead of a packet or application viewpoint. Understanding content and its intent is the highest level of intelligence to be gained from network traffic. This is important as information flow is moving away from Packet, towards Application, and ultimately to Content.

Example Inspection Levels

• Packet: Random Sample to get larger picture
• Application: Group or application profiling. Certain applications, or areas of applications, are allowed / not allowed or scanned further.
• Content: Look at everything. Scan everything. Understand the intent.

Inspection

Opening Traffic and understanding it. Then you can apply services to it.

Some Service/Inspection examples:

• Anti-Virus
• Data Loss
• Unknown Threats
• Analytics
• Code Attacks/Injection
• Content Manipulation

Applications of Deep Content Inspection

DCI is currently being adopted by enterprises, service providers and governments as a reaction to increasingly complex internet traffic with the benefits of understanding complete file types and their intent. Typically, these organizations have mission-critical applications with rigid requirements.^[4]

Obstacles to Deep Content Inspection

Network Throughput

This type of inspection deals with real time protocols that only continue to increase in complexity and size. One of the key barriers for providing this level of inspection, that is looking at all content, is dealing with network throughput. Solutions must overcome this issue while not introducing latency into the network environment. They must also be able to effectively scale up to meet tomorrow's demands and the demands envisioned by the growing Cloud Computing trend. One approach is to use selective scanning; however, to avoid compromising accuracy, the selection criteria should be based on recurrence. The following patent USPTO# 7,630,379^[5] provides a scheme as to how Deep Content Inspection can be carried out effectively using a recurrence selection scheme. The novelty introduced by this patent is that it addresses issues such as content (E.g., an mp3 file) that could have been renamed before transmission.

Accuracy of Services

Dealing with the amount of traffic and information and then applying services requires very high speed look ups to be able to be effective. Need to compare against full services platforms or else having all traffic is not being utilized effectively. An example is often found in dealing with Viruses and Malicious content where solutions only compare content against a small virus database instead of a full and complete one.

Notes

1. ^ "Deep Content Inspection vs. Deep Packet Inspection", Wedge Networks Inc., August 2, 2011, accessed August 23, 2011.
2. ^ Adhikari, Richard . "Seeking Tomorrow's Security Solutions Today, Part 1", Tech News World, July 21, 2011, accessed August 23, 2011.
3. ^ Sarang, Dharmapurikar. "Deep Packet Inspection - Which Implmentation Platform". http://www.netsec.colostate.edu/ccw08/Slides/sarang.ppt. Retrieved 31 August 2011. 
4. ^ Racoma, Angelo J. "Wedge Networks BeSecure Uses Deep Content Inspection to Protect Against Malware", CMS Wire, May 19, 2011, accessed August 1, 2011.
5. ^ Morishita, et al.. "US Patent 7,630,379". http://www.freepatentsonline.com/7630379.pdf. Retrieved December 8, 2009.