We are starting a new blog post series here on the ModSecurity site called "Advanced Feature of the Week" where we will be highlighting many of ModSecurity's really cool capabilities.  These are the features that seldom used or fully understood by the average ModSecurity user however can provide detection of sophisticated attacks if used properly.  It is our goal with these blog posts to help shed light on these unique features and to provide some real-world, in-the-trenches gotchas for successful usage of these features.

This blog post series will have the following major topic sections -

1) ModSecurity Reference Manual Information

Provide reference manual data.

2) Use Within the OWASP Core Rule Set (CRS)

Outline if/when/how the CRS is utilizing this feature.

3) So What?

Will provide some context as to why you as a user should even care about this capability.  What advanced attack/vulnerability is this attempting to catch.

This week's feature is on the use of the @validateByteRange operator.

SecRule ARGS:text "@validateByteRange 10, 13, 32-126"

ASCII Byte Range Chart

Use of @validateByteRange in the OWASP ModSecurity CRS (from the end of the modsecurity_crs_20_protocol_violations.conf file) -

#
# Restrict type of characters sent
# NOTE In order to be broad and support localized applications this rule
# only validates that NULL Is not used.
#
# The strict policy version also validates that protocol and application 
# generated fields are limited to printable ASCII. 
#
# -=[ Rule Logic ]=-
# This rule uses the @validateByteRange operator to look for Nul Bytes.
# If you set Paranoid Mode - it will check if your application use the range 32-126 for parameters.
#
# -=[ References ]=-
# http://i-technica.com/whitestuff/asciichart.html
#

SecRule ARGS|ARGS_NAMES|REQUEST_HEADERS|!REQUEST_HEADERS:Referer "@validateByteRange 1-255" \
 "phase:2,rev:'2.0.8',pass,nolog,auditlog,msg:'Invalid character in request',id:'960901',tag:'PROTOCOL_VIOLATION/EVASION',tag:'WASCTC/WASC-28',tag:'OWASP_TOP_10/A1',tag:'OWASP_AppSensor/CIE3',tag:'PCI/6.5.2',severity:'4',t:none,t:urlDecodeUni,setvar:'tx.msg=%{rule.msg}',tag:'http://i-technica.com/whitestuff/asciichart.html',setvar:tx.anomaly_score=+%{tx.notice_anomaly_score},setvar:tx.protocol_violation_score=+%{tx.notice_anomaly_score},setvar:tx.%{rule.id}-PROTOCOL_VIOLATION/EVASION-%{matched_var_name}=%{matched_var}"

SecRule TX:PARANOID_MODE "@eq 1" "chain,phase:2,rev:'2.0.8',pass,nolog,auditlog,msg:'Invalid character in request',id:'960018',tag:'PROTOCOL_VIOLATION/EVASION',tag:'WASCTC/WASC-28',tag:'OWASP_TOP_10/A1',tag:'OWASP_AppSensor/CIE3',tag:'PCI/6.5.2',severity:'4',t:none,t:urlDecodeUni,tag:'http://i-technica.com/whitestuff/asciichart.html'"
 SecRule REQUEST_URI|REQUEST_BODY|REQUEST_HEADERS_NAMES|REQUEST_HEADERS|!REQUEST_HEADERS:Referer|TX:HPP_DATA \
 "@validateByteRange 32-126" \
 "t:none,t:urlDecodeUni,setvar:'tx.msg=%{rule.msg}',setvar:tx.anomaly_score=+%{tx.notice_anomaly_score},setvar:tx.protocol_violation_score=+%{tx.notice_anomaly_score},setvar:tx.%{rule.id}-PROTOCOL_VIOLATION/EVASION-%{matched_var_name}=%{matched_var}"

%u3008scr%u0131pt%u3009%u212fval(%uFF07al%u212Frt(%22XSS%22)%u02C8)%u2329/scr%u0131pt%u2A

〈scrıpt〉ℯval(＇alℯrt(”XSS”)ˈ)〈/scrıpt〉

<script>eval('alert("XSS")')</script>

So, this brings us to today's advanced ModSecurity feature - @validateByteRange.  By restricting the allowed character byte ranges, you can help to identify when unexpected character code points are used.  If the payload above is sent, you should receive an alert message similar to the following -

Message: Found 3 byte(s) in REQUEST_URI outside range: 32-126. [file "/usr/local/apache/conf/modsec_current/base_rules/modsecurity_crs_20_protocol_violations.conf"] [line "251"] [id "960018"] [rev "2.0.6"] [msg "Invalid character in request"] [severity "WARNING"] [tag "PROTOCOL_VIOLATION/EVASION"] [tag "WASCTC/WASC-28"] [tag "OWASP_TOP_10/A1"] [tag "OWASP_AppSensor/CIE3"] [tag "PCI/6.5.2"] [tag "http://i-technica.com/whitestuff/asciichart.html"]

Since Black Hat and DEFCON we have been busying building teams and aligning objectives over here at Trustwave's SpiderLabs. We are committed to driving innovation into the development of ModSecurity for the future.

Here are are few things that we having going on right now:

• Website Updates - We are in the process of refreshing the website to reflect its new home within SpiderLabs but also clean out any dusty content and replace it with fresh new material. Look for phases of changes over the next several weeks.
• Recruiting - We are actively looking to hire full-time developers for ModSecurity. Those developers will be part of the SpiderLabs research team. If you or anyone you know is interested, please let us know.
• Internal Development Team - We have formed a team of people within SpiderLabs that will be actively contributing to ModSecurity. This team is comprised of many very experienced individuals including many OWASP, Black Hat and DEFCON researchers/speakers in the area of web application security. This team is excited to breath new life into this project.
• End User Survey - In the next few weeks, we will be sending out a survey to the ModSecurity mailing lists to ask for feedback on features to include in the next release.
• Blog Updates - Each week we'll be writing about an Advance Feature in ModSecurity that the community and end users may not be aware of. The idea is here is that with increased usage of these advanced, we'll get better feedback on how to improve the product.
• Twitter - We have just linked this blog to the ModSecurity Twitter account. Follow us!

As many of you already know, about a month ago Trustwave acquired Breach Security, the then owner and sponsor of the ModSecurity project. Trustwave's Spider Labs will now be leading the project. Within Spider Labs and the highly capable people there, ModSecurity will be able to thrive.

And so, it is with mixed feelings that I also announce that I am stepping down from the project. I have enjoyed working on ModSecurity with Breach for the past 3 1/2 years, but it is time for me to move on to another exciting project (more on that later). While I may be stepping down as a leader, I am not leaving, just changing roles. I plan to stay an active member of the community and to contribute where I can. I expect to see a lot of innovation from ModSecurity in the near future and I look forward to see where ModSecurity will go with all its new resources.

Cheers to ModSecurity's new home in Spider Labs and to a successful future!

P.S. -- I'll be around at Black Hat/DEFCON next week in Vegas if anyone wants to hook up.

ModSecurity Community,

We will be hosting a ModSecurity happy hour during Black Hat USA. It is open to anyone who contributes, uses or wants to learn more about the project. You'll also get a chance to meet the members of Trustwave's SpiderLabs team that will be supporting the ModSecurity project moving forward. 

• What: ModSecurity Happy Hour
• Where: munchbar @ Caesar's Palace LV (next to Sports Bar & Pure Night Club)
• When: Wednesday, July 28th, 2010 - 4:00-6:00 PM

There will be appetizers, drinks, and Nintendo Wii action!

Look forward to meeting you next week!

Senior Vice President, SpiderLabs

Trustwave

There was a recent blog article stating that ModSecurity can be bypassed by adding invalid characters to Base64 encoded data. Well, this is somewhat correct, but I am not sure I'd call it a bypass. It is really "Impedance Mismatch" as it depends on the decoder you are using in your app. PHP's decoder is ignoring characters (RFC-2045) and ModSecurity is doing what Apache does for HTTP Basic Auth and not allowing the extra characters (RFC-4648)

The article's example is roughly this:

1) Take an attack string: <script>alert(1)</script>
2) Base64 encode it to: PHNjcmlwdD5hbGVydCgxKTwvc2NyaXB0Pg==
3) Now add an illegal character: P.HNjcmlwdD5hbGVydCgxKTwvc2NyaXB0Pg==
4) Notice that most decoders will not work, but PHP's will (act surprised)

PHP will apparently just skip the invalid characters (RFC-2045) and so something like this (article's example, not mine) will of course fail:

SecRule ARGS:b64 "alert" "t:base64decode,log,deny,status:501"

The Base64 decoder in ModSecurity is based off the RFC-4648 implementation of Base64. There are many other variants. Well, as it turns out it is important to know a bit more about your platform on which your app is based and the above trivial rule is just not going to cut it.

For PHP and possibly others you will need to go a little further and validate the character set first using a positive rule. Something like this is going to be required for the article's example:

SecRule ARGS:b64 "!^[A-Za-z0-9\+/]*={0,2}$" \
  "phase:2,t:none,log,deny,status:403,msg:'Invalid Base64 Encoding'"
SecRule ARGS:b64 "alert" \
  "phase:2,t:none,t:base64decode,log,deny,status:403,msg:'Badness in b64'"

And now you get some better coverage:

# For PHNjcmlwdD5hbGVydCgxKTwvc2NyaXB0Pg==
ModSecurity: Access denied with code 403 (phase 2). Pattern match "alert" 
at ARGS:b64. [file "test.conf"] [line "3"] [msg "Badness in b64"] 
[hostname "myhost"] [uri "/foo"] [unique_id "S8-4-X8AAQEAACGOJcoAAAAA"]

# For P.HNjcmlwdD5hbGVydCgxKTwvc2NyaXB0Pg==
ModSecurity: Access denied with code 403 (phase 2). Match of 
"rx ^[A-Za-z0-9\\\\+/]*={0,2}$" against "ARGS:b64" required. 
[file "test.conf"] [line "1"] [msg "Invalid Base64 Encoding"] 
[hostname "myhost"] [uri "/foo"] [unique_id "S8-5BX8AAQEAACJSKBYAAA@i"]

Though I am picking on PHP a bit here, this may be true in many other areas if you have decoders/parsers that accept out-of-the-norm data. You really do have to know your apps to write targeted rules like the example in this article. You cannot detect encodings like Base64 generically and I would not expect to find such a rule as this in a generic rule set such as ModSecurity's CRS.

Edited: Added details on which RFCs I was referring to and removed blame on PHP after further investigation as it really is just an issue with multiple variants of base64.

ModSecurity versions 2.5.8 and 2.5.9 have been released to fix two vulnerabilities which could be used to cause a denial of service (DoS).  The first vulnerability is fixed in version 2.5.8 and the second (as it was disclosed after version 2.5.8 was already frozen) is fixed in version 2.5.9.  Because of this, the 2.5.8 release should be disregarded in favor of 2.5.9.  Both vulnerabilities, however, have workarounds until ModSecurity can be upgraded/patched.

Vulnerability in the PDF XSS engine

The ModSecurity PDF XSS protection engine prior to version 2.5.8 would allow a NULL pointer to be dereferenced when a non-GET request was used to generate a PDF.  This would cause Apache to crash and allow for a denial of service (Low Risk).

As a workaround, you should disable PDF XSS protection (the default), such as in the following example, until an update to ModSecurity is in place.

SecPdfProtect Off

Vulnerability in the multipart request parser

The ModSecurity multipart request parser prior to version 2.5.9 would allow a NULL pointer to be dereferenced under some circumstances when a zero length part header name is used.  This would cause Apache to crash and allow for a denial of service (Med-High Risk).

As a workaround, you should disable multipart/form-data parsing and add a rule to detect exploitation, as in the following example, until an update to ModSecurity is in place.

###
# BEGIN TEMPORARY WORKAROUND
#
# Force URLENCODED processing so that the vulnerability is bypassed
# and the REQUEST_BODY target variable is populated, but at the 
# expense of no longer being able to parse multipart form arguments.
SecRule REQUEST_HEADERS:Content-Type "multipart/form-data" \
 "phase:1,t:none,t:lowercase,nolog,pass,ctl:requestBodyProcessor=URLENCODED"
#
# Deny any multipart requests with a zero length header name
SecRule REQUEST_HEADERS:Content-Type "multipart/form-data" \
 "chain,phase:2,t:none,t:lowercase,deny,msg:'ModSecurity DoS attempt - NULL part header name'"
SecRule REQUEST_BODY "(?m:^--.*\\x0d\\x0a?:.*\\x0d\\x0a?--)" "t:none"
#
# END TEMPORARY WORKAROUND
###

More details can be found in the CHANGES file within the released source packages.  Special thanks to those at Red Hat and Internet Security Auditors for disclosing the vulnerabilities.

Both of these are excellent examples of how ModSecurity can be used to provide a virtual patch before an upgrade and/or proper patch can be applied -- even a vulnerability within ModSecurity itself.

Deciding to leave a job is rarely easy, but leaving your job and the project you've nurtured for six years is very difficult. In a few weeks' time I will no longer be working for Breach Security and, as a consequence, I will be joining the ModSecurity community as a user. I leave ModSecurity in the capable hands of Brian Rectanus, who has been taking care of it for some time now.

It's been an exciting journey, starting ModSecurity and growing it for all these years. It's been a pleasure working with the community and helping others solve their problems. It's been hard at times, but that made it all the more interesting.

Finally, I would like to take this opportunity to thank every single ModSecurity user, because we, all together, made ModSecurity what it is. I couldn't have done it without you.

I spent the last week importing ModSecurity's source code repository into subversion at Source Forge. I am proud to announce that a read-only version of ModSecurity's subversion repository is now publicly available. In addition to this, Atlassian has graciously given the ModSecurity project a license to use their software. We are currently using Jira to track reported issues, FishEye to track and analyze the code repository and have plans to add Confluence (a wiki) in the near future. Together, these tools allow users to test and apply fixes immediately as well as monitor the progress of reported issues and the progress of ModSecurity as a whole.

This is a big step forward for the ModSecurity project. The project is becoming much more open and feels more like an Open Source project should. Please take some time to look at the new tools. Comments are always welcome.

This year, the OWASP's Summer of Code event contains one project that's of particular interest to me (and possibly to you, consider that you're following this blog): Securing WebGoat Using ModSecurity. If you've even seen WebGoat (a learning sandbox that contains samples of many application security issues) then you know how difficult it might be to secure. It's true; it's probably the worst-case scenario for ModSecurity, because it's not your typical application, and it contains a number of problems that require the understanding of application state to exploit (and thus protect too). So we should all congratulate Stephen Evans for pulling through and reaching the end of the project.

I find the project interesting because it stretches the boundaries of what ModSecurity can do. For example, one of the features Stephen relied heavily on is the Lua scripting language, which is currently marked as experimental. As we are currently working on the design for ModSecurity 3, Stephen's feedback is going to be very useful to us.

If this sort of thing is indeed of interest to you then you need to know that Stephen will be sharing his experiences during the OWASP Summit in Portugal next week. If you'll be attending the summit (or even if you can schedule a trip on short notice) do consider attending.

Intrusion detection is a well-known network security technique--it introduces monitoring and correlation devices to networks, enabling administrators to monitor events and detect attacks and anomalies in real-time. Web intrusion detection does the same but it works on the HTTP level, making it suitable to deal with security issues in web applications. This session will start with an overview of web intrusion detection and web application firewalls, discussing where they belong in the overall protection strategy. The second part of the talk will discuss ModSecurity and its capabilities. ModSecurity is an open source web application firewall that can be deployed either embedded (in the Apache HTTP server) or as a network gateway (as part of a reverse proxy deployment). Now in it's sixth year of development, ModSecurity is mature, robust and flexible. Due to its popularity and wide usage it is now positioned as a de-facto standard in the web intrusion detection space.

This is the same talk I presented at ApacheCon Europe 2008 in Amsterdam and the OWASP AppSec US 2008 conference in New York, and it's getting better every time I do it. This is going to be my first ApacheCon in the US and I am really looking forward to it. It's the place to be if you are involved in any way with the projects of the Apache Software Foundation.