ModSecurity Trustwave
This blog has moved! Please update your
bookmarks to

ModSecurity Blog: March 2008

New Blog is Up and Running

We've just completed a move to the new blogging platform. Not only is our blog now visually integrated with the rest of the web site, but we've enabled comments and trackbacks, which should make our blogging from now on much more interesting. The old posts are going to remain where they are. The old feed has been updated to that of the new blog, so if you are a subscriber you don't need to do anything.

Web Application Monitoring Data Model

A data model is the foundation of web application monitoring and, thus, key to successful utilisation of web application firewalls. We don't get to design the model; we can only deduct it from the information provided to us from the underlying technology. What we can do is build on it, and, for that reason, it is very important to understand what we have to work with.

An ideal model is one that helps structure the information available to us, allows us to enrich it with additional pieces of data and generally helps us raise events based on the information it contains.

The major parts of a web application monitoring data model are as follows:

  • Connection - corresponds to one TCP connection.
  • Request - corresponds to one HTTP request.
  • Response - corresponds to one HTTP response.
  • IP Address - the IP address of the client, retrieved from the TCP connection.
  • Session - application session.
  • User - authenticated user; in most cases this translates to the application user, but some sites still use HTTP authentication, and some might use both.
  • Site - perhaps more accurately called Protection domain, or Application. None of these terms is perfect, but I generally prefer to use Site. In our model, Site refers either to the functionality behind an entire domain name (e.g., or only a subset of one (
  • Country - the country where the request originates.
  • City - the city where the request originates.
  • Custom - any number of custom attributes. For example, you might want to have different policies for different departments within your organisation. To achieve this, you will map client IP addresses to department names, which you will then use to determine policies.

Most of the components are easy to construct, mapping from the structures used in programming, but there are a few places where the technology does not support the view, or where what we are given is not what we want to see:

  • Some work is needed to be able to distinguish sessions. There are different session identifier techniques to consider (e.g. in the URI, in a parameter, in a cookie). While there is a number of platforms that have standardised session management, there is also a large number of applications using their own schemes, so in general some custom work will be needed.
  • More so in the case of user identification. Building on session identification one needs to identify a successful login event in the traffic in order to determine the session username.
  • The IP address may not be accurate. It may be that of an intermediary, and not of the client itself. Such cases can sometimes be identified (as is the case with HTTP proxies) , but not always (e.g. if a transparent HTTP proxy is used). The problem is that, unless you control the proxy, you can only rely on the IP address you got from the TCP stack; the information extracted from HTTP requests headers is not to be trusted.

Note: This post is part of the Web Application Firewall Concepts series.

Web Application Firewall Use Case: Continuous Security Assessment

After some deliberation, I have decided to add Continuous security assessment as a standalone item on my web application firewall use cases list. Although some could argue the functionality is already covered in the Web intrusion detection and prevention section, it is not obvious—you have to understand this field in order to know it is there.

Continuous security assessment is not specific to web application firewalls—it's been used for the network layer for years—but web application firewalls are more useful for web applications (than IDS tools are for network applications), simply because there's essentially one data format to analyse (if you can call a bunch of loosely related specifications used by web applications "one" data format). With web applications, you get to see clear picture of application input and output. Therefore, with proper parsing and analysis in place, it's almost like having a direct communication channel with the application itself.

The focus here is not on the attacks and other suspicious client behaviour, which comes out of the stock IDS/IPS functionality, but on everything else:

  • application defects,
  • information leakage,
  • configuration errors,
  • change control

and so on. The advantage is that you can detect some very subtle issues, only possible because of the depth of analysis.

Just as an extreme example, there are quite a few web applications out there where SQL injection (less often) and XSS (surprisingly common) exist by design—their clients are allowed and expected to send such low-level stuff and have it executed by the server. These types of problem can be detected early and with little effort, and because assessment never stops, you get the alert as soon as they are manifested.

Note: This post is part of the Web Application Firewall Concepts series.

Web Application Firewall Use Cases

There are many reasons to use a web application firewall. Most people tend to focus on prevention and blocking when the term is brought up, but that is just one of the possible uses. Three years ago, almost to the day, I wrote this post to argue how one needs a WAF to serve as a part of the overall defence strategy. My opinion remains unchanged today, but I have since expanded the list of use cases for web application firewalls. Here they are:

  1. Web intrusion detection and prevention. Applying the principles of external traffic monitoring (IDS) and prevention (IPS) to HTTP and the related technologies, which are used to build web applications. Through your WAF you will look for signs of generic web application attacks (negative security model), or deploy a learning engine to construct a model of your site and reject all invalid traffic, not just attacks (also known as positive security model).
  2. Continuous security assessment. The idea with this case is to emphasize the fact web application firewalls actually understand web applications pretty well. Armed with this knowledge, they can do more than detect attacks; they can observe from signs of weaknesses, information leaks, configuration errors, and similar problems before an attempt to exploit them is made.
  3. Virtual (or just-in-time) patching. When you need to deal with a specific problem in your web site, which exists either in your code or in the code of the product you are relying on. The focus in this use case is on writing custom rules to deal with custom issues.
  4. HTTP traffic logging and monitoring. Do you know what is actually going on in your web applications? Who are your users and how are they using your systems? Are you being attacked and how?
  5. Learning. My original list did not include learning as a WAF feature but, in retrospective, I realised I was failing to convey a very important functionality and differentiator over intrusion detection and prevention systems. The point is that WAFs understand web applications, so in some cases they are able to move away from intrusion detection (as in traffic monitoring) to a default deny security model.
  6. Web application hardening. If you deploy your WAF as a reverse proxy then you can get it to modify the traffic stream to fix some of the design faults of your application or the HTTP protocol.

I will expand on each use case in my future posts.

Update (10 July 2008): I've decided to remove the "Network building blocks" use case from the list. While I still believe this is an important use case, I now think the usefulness should be attributed to reverse proxies. And, if a WAF happens to be implemented as a reverse proxy, that it will include the advantages and the disadvantages of that model.

Note: This post is part of the Web Application Firewall Concepts series.

Web Application Firewall Concepts

I went through all my ModSecurity Blog posts yesterday, partly to admire myself for blogging consistently for almost 5 years and partly to understand what is that I talked about during this time. While I knew that most of my posts were pretty technical (after all, I did start my new blog to focus on other things) imagine my surprise when I realised I didn't properly cover the one thing this blog is supposed to cover: web application firewalls!  The emphasize is on the word "properly": I provided a great deal of technical information but not enough content that would explain why one would deploy a web application firewall and how. This stuff had went into my conference talks and the Web Application Firewall Evaluation Criteria project, but I forgot to discuss the topics here. In an effort to fix this I am starting a series of blog posts called Web Application Firewall Concepts. Each post will be reasonably brief and cover one aspect of the technology, and I will continually update this post to serve as a table of contents. Posts in this series:

  1. Use Cases
    1. Web intrusion detection and prevention
    2. Continuous Security Assessment
    3. Virtual (or just-in-time) patching
    4. HTTP traffic logging and monitoring
    5. Learning
    6. Web application hardening
  2. Deployment models
    1. Inline
    2. Out of line
    3. Embedded
  3. Data Model
    1. Model construction
    2. Persisting information across requests
    3. Distinguishing sessions
    4. Distinguishing users
  4. Analysis Model
    1. Negative security
    2. Positive security
    3. Anomaly scoring
    4. Learning
    5. Evasion
    6. Impedance mismatch
  5. Traffic logging
  6. Special protection techniques
    1. Cookie protection
    2. Cross-Site Request Forgery
    3. Brute force attacks
    4. Denial of Service attacks
    5. PDF UXSS protection


November 2010
Sun Mon Tue Wed Thu Fri Sat
1 2 3 4 5 6
7 8 9 10 11 12 13
14 15 16 17 18 19 20
21 22 23 24 25 26 27
28 29 30


Atom Feed



Recent Entries