ModSecurity Trustwave
This blog has moved! Please update your
bookmarks to http://blog.spiderlabs.com/modsecurity/.

ModSecurity Blog: March 2007

Regular Expression Development Tools

Since ModSecurity is based on regular expressions. Writing rules requires developing and testing such expressions. The following tools can help you in analyzing and testing them:

  • The Regex Coach is simple and powerful. You simply type your expression in the upper box and the text to match in the lower box. Any matches, if found, are highlighted in the text. In between the boxes you can control the regular expression flags such as “ignore case” or “global match”. The Regex Coach does not stop there, it provides insight into the regular expression matching process by showing a tree view of the regular expression and letting you follow the matching process, step by step. A weak point is that sometimes the Regex Coach has difficulty in handling the regular expressions we use in the ModSecurity Enhanced Rule Set.
  • Expresso - An alternative tool that works well is Expresso. While free, it is not typical of open source software. It uses the .NET framework and you may be required to pay for certain other components. It has good peripheral features such as a library of regular expressions and the ability to save your test work in a project file, but is more complex than the Regex Coach and seems to have less information about the internals of regular expressions.

2.1/1.x Rule Differences For Identifying Missing/Empty Headers and Variables

There are certain scenarios where you might want to create white-listed ModSecurity rulesets which enforce that certain headers/variables are both present and not empty. This Blog entry highlights an important difference between the 1.X and 2.X ModSecurity Rules Languages in this regard.

ModSecurity 2.X Rules
There are some good examples of this in Core Rules file - modsecurity_crs_21_protocol_anomalies.conf file. Take for example the following entries -

SecRule &REQUEST_HEADERS:Host "@eq 0" "skip:1,log,auditlog,msg:'Request Missing a Host Header',id:'960008',severity:'4'"
SecRule REQUEST_HEADERS:Host "^$" "log,auditlog,msg:'Request Missing a Host Header',id:'960008',severity:'4'"

The 1st rule uses the "&" operator to enable counting the number of variables in a collection. So, this 1st rule will identify if the Host header is not present in the request. The 2nd rule uses the RegEx "^$" that will match if the Host header is present, but is empty.

ModSecurity 1.X Rules
The older ModSecurity 1.X rules language operated a bit differently. If you wanted to use a similar rule as the one above, you could use this rule -

SecFilterSelective HTTP_Host "^$"

This rule means two different things -
1) The Host header is missing and/or
2) The Host header is empty

Which is better?
On the surface, you might think "The 1.X rules way is better since you only need 1 rule..." however you need to realize that anytime you have rules or directives that implicitly enforce certain capabilities you run the risk of having false positives as it could match things that you didn't want them to. For instance, what if you have a situation where certain web clients (such as mobile devices) legitimately include some headers, however they are empty? Do you want to automatically block these clients? With the ModSecurity 1.X Rule Language, you would have to remove the entire rule. With the ModSecurity 2.X Rule Language, however, you are able to create rules to more accurately apply the logic that you desire.

ModSecurity Console: Purpose and Deployment

If you have more then 1 ModSecurity installation, you have undoubtedly run into issues with consolidating, analyzing and responding to alert messages. Yes, you can always reconfigure Apache to send its access/error logs through Syslog onto a remote, central logging server however the data being forwarded is a very small subset of the entire transaction. It is only a warning message and not enough information to conduct proper incident response to determine if there was a false positive or if it was a legitimate attack. In order to determine this information, you need access to the ModSecurity Audit log files. Unfortunately, the original "Serial" format of the audit log was multi-line with all records held within one file. This made remote logging difficult. What was really needed was to have a mechanism to send logs onto a centralized logging host made specifically for processing ModSecurity Alert data. This is the purpose of the ModSecurity Console.

Although you can deploy the ModSecurity Console on the same host that is running ModSecurity, the recommended solution is to deploy the Console on a stand-alone server and then reconfigure your various ModSecurity hosts to send their logs across the network to the Console. This provides two benefits:

1) Management of the audit logs is centralized and thus makes analysis easier, and
2) If the web server were to be compromised, the audit logs would not be held locally where an attacker could delete them.

DEPLOYMENT
There are essentially 3 components that need to be configured and working correctly for this deployment to work:

1) The Console itself must be up and running and have profiles setup for each Sensor.
2) The ModSecurity configuration files on the individual installations must have the proper Audit logging directives configured, and
3) The modsec-auditlog-collector.pl script must be configured with the correct information.

The following steps will show you how to set these all up.

STEP 1: Install the Console software on the central log host
Follow the installation steps outlined in the README.txt file that comes with the software. Once you have installed the ModSecurity Console software on your logging host, you should then log into the web interface. Let's say that the Console host has an IP address of - 192.168.1.100. You should then login by going to - https://192.168.1.100:8888/. You will then get a Basic Auth login popup box. Login with "admin" as username and "admin" as password.

STEP 2: Create sensor profiles
You need to create Sensor profiles on the ModSecurity console. The term Sensor simply means a remote installation of ModSecurity. The Sensor profile is created to uniquely identify the incoming audit logs from each other and to optionally implement password/IP access controls. Click on the "Sensors" link at the top of the Console page and then click on the "Add Sensor" button. Next, fill in the appropriately Sensor Details such as Sensor Name, IP Address, Sensor Password and then click on Submit. You need to remember this information when you are editing the modsec-auditlog-collector.pl script on each Sensor host. Let's say for this sensor, the IP Address is 192.168.1.101, Username is alpha1 and the Password is sensor1.

STEP 3: Reconfigure the Audit Directives on each Sensor to use Concurrent Logging
The next step is to update a few of the audit directives in the configuration file of the individual ModSecurity installs.

Edit these directives so they look like this -

SecAuditEngine RelevantOnly
SecAuditLogRelevantStatus "^(?:5|4\d[^4])"
SecAuditLogType Concurrent
SecAuditLogParts ABCDEFGHZ
SecAuditLogStorageDir /path/to/apache/logs/data/
SecAuditLog /path/to/apache/logs/audit.log

STEP 4: Test local concurrent logging
Once this is configured, you should restart Apache, send a few requests that will trigger a ModSecurity rule and then check the SecAuditLog file for the new, concurrent data. The format should look something similar to this -

192.168.2.101 192.168.2.11 - - [15/Jul/2005:11:56:52 +0100] "POST /form.php HTTP/1.1" 403
3 "http://192.168.2.101:8080/form.php" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.8) 
Gecko/20050511 Firefox/1.0.4" G3yTd38AAAEAAAM7BLwAAAAA "-" /20050715/20050715-1156/20050715-
115652-G3yTd38AAAEAAAM7BLwAAAAA 0 1031 md5:dc910f6d647d47b32ae6e47326f0ca42

The bolded section of the log entry is the location of the individual audit log file under the SecAuditLogStorageDir location. If you then looked at the following location - /path/to/apache/logs/data/20050715/20050715-1156/20050715-
115652-G3yTd38AAAEAAAM7BLwAAAAA - you would find the entire audit log record. Once you have confirmed that Concurrent logging is working correctly in local mode, the next step is to setup the modsec-auditlog-collector script with the correct information.

STEP 5: Update the modsec-auditlog-collector.pl script
You can download the modsec-auditlog-collector.pl code from within the web interface of the ModSecurity Console. Once you are logged in, click on the "About" link and then use "this Perl script" link. Once you download the script, you should upload it to your ModSecurity host and place it in the /path/to/apache/bin/ directory. You then need to do the following:

1) Chmod the script to make sure that it is executable by Root.
2) Verify that the path to Perl is correct in the script.
3) Edit the following lines within the script -

my $CONSOLE_URI = "/rpc/auditLogReceiver";
my $CONSOLE_HOST = "192.168.1.100";
my $CONSOLE_PORT = "8886";
my $CONSOLE_USERNAME = "alpha1";
my $CONSOLE_PASSWORD = "sensor1";

It is important that you specify the correct IP address for your ModSecurity Console host and that you also specify the correct Sensor Username and Password that you created within the ModSecurity Console. If any of this information is not correct, the modsec-auditlog-collector.pl script will not successfully tranfer the audit log files. After updating the modsec-auditlog-collector.pl script, the next step is to update one of the ModSecurity audit directives again.

STEP 6: Update the SecAuditLog Directive
Edit the ModSecurity configuration file once again and edit the SecAuditLog directive like this -

SecAuditLog "|/path/to/apache/bin/modsec-auditlog-collector.pl \
/path/to/apache/logs/data/ /path/to/apache/logs/audit.log"

What this is doing is using the "|" piped log feature to send the audit log data directly to the modsec-auditlog-collector.pl script. It also has two arguments, the first one being the path to the directory location specified by SecAuditLogStorageDir and the original SecAuditLog file that holds the audit log meta-data. Once you have updated this this directive, you should then restart Apache.

STEP 7: Testing Remote Logging
The final step is to verify that the remote ModSecurity Sensor will now send its Concurrent audit log data to the central Console. In order to do this, you should send another request to the ModSecurity Sensor that will trigger one of the Core Rules. This will then result in an audit log being created, passed to the modsec-auditlog-collector.pl script and then sent across the network to the Console.

In the Console, you should see the remote audit log file show up under the "Alerts" page in the Console. You can then click on the alert file link for all of the details.

Conclusion
Hopefully, this information will assist ModSecurity users with deploying their ModSecurity Consoles.

Anurag Agarwal writes: Reflection on Ivan Ristic

Anurag Agarwal has started profiling prominent personalities from the web applicaation security space. He started with Amit Klein, then RSnake, Jeremiah Grossman, and myself. Needless to say I am very flattered to be in such good company. There's more stuff coming. A bio of Sheeraj Shah is coming soon.

ModSecurity ASCIIZ Evasion

It has been brought to our attention that a fault in the ModSecurity parsing code has been discovered and published. (No, we have not been contacted by the author, either before or after the publication. We learned about the problem from a third party.) The fault makes it possible to sneak an attack payload past a class (not all, as you will see below) of ModSecurity rules.

The rules that use variables that refer to request parameters (e.g. ARGS) can be evaded in certain circumstances, as follows: 1) parameters must be transported in an application/x-www-form-urlencoded request body, 2) an un-encoded NULL byte (ASCIIZ) must be embedded in the payload and 3) the parser used by the web application must do things differently. (Apparently, PHP < 5.2.0 cannot be attacked in this way but >= 5.2.0 can. Of course, the problem is not specific to PHP, but at this point we don't know exactly how other environments are impacted. Unless you know differently you should treat the situation as if the evasion is possible.) The ASCIIZ byte will cause the parser to terminate prematurely, missing some data.

The logging features are not affected.

A ModSecurity update will be released to deal with this issue. However, an update is not the only way to deal with the problem. it is possible to use ModSecurity itself to detect and prevent evasion. Simply add the following rule to your rule set:

SecRule REQUEST_BODY "@validateByteRange 1-255" \
"log,deny,phase:2,t:none,msg:'ModSecurity ASCIIZ Evasion Attempt'"

All your other rules can remain intact. The above rule should not result in false positives under normal circumstances.

Please note that, not having been notified in advance, there was not enough time to research all aspects of this evasion technique as thoroughly as we would like. We are sending this email to notify our users and help them mitigate the problem quickly. The information presented here may end up being a work in progress, in which case we will follow up as soon as we learn more.

And, for the record, I am not at all happy with the fact the issue was not disclosed to us in advance. We take security very seriously; a responsible disclosure would have allowed us to have an updated version of ModSecurity available for download at the same time as the disclosure.

Update: This was fixed a while ago, with the release of ModSecurity v2.1.1.

Calendar

November 2010
Sun Mon Tue Wed Thu Fri Sat
1 2 3 4 5 6
7 8 9 10 11 12 13
14 15 16 17 18 19 20
21 22 23 24 25 26 27
28 29 30

Feeds

Atom Feed

Search

Categories

Recent Entries

Archives