ModSecurity Blog

Just a quick note to let everyone know that a 2-day ModSecurity training class was added to the upcoming Blackhat USA 2009 conference in Las Vegase - http://www.blackhat.com/html/bh-usa-09/train-bh-usa-09-categories-appsec.html.  This class will be focused on the ModSecurity Rules Language and using it for virtual patching purposes.  We will have a hands-on lab on day 2 that covers the OWASP Securing WebGoat with ModSecurity project data.  This will expand upon the Blackhat Federal presentation that I gave earlier this year - http://www.blackhat.com/html/bh-dc-09/bh-dc-09-speakers.html#Barnett.  So, if you want to get your hands dirty using the latest and greatest features of ModSecurity, including Content Injection and Lua, then this class is for you!

ModSecurity versions 2.5.8 and 2.5.9 have been released to fix two vulnerabilities which could be used to cause a denial of service (DoS).  The first vulnerability is fixed in version 2.5.8 and the second (as it was disclosed after version 2.5.8 was already frozen) is fixed in version 2.5.9.  Because of this, the 2.5.8 release should be disregarded in favor of 2.5.9.  Both vulnerabilities, however, have workarounds until ModSecurity can be upgraded/patched.

Vulnerability in the PDF XSS engine

The ModSecurity PDF XSS protection engine prior to version 2.5.8 would allow a NULL pointer to be dereferenced when a non-GET request was used to generate a PDF.  This would cause Apache to crash and allow for a denial of service (Low Risk).

As a workaround, you should disable PDF XSS protection (the default), such as in the following example, until an update to ModSecurity is in place.

SecPdfProtect Off

Vulnerability in the multipart request parser

The ModSecurity multipart request parser prior to version 2.5.9 would allow a NULL pointer to be dereferenced under some circumstances when a zero length part header name is used.  This would cause Apache to crash and allow for a denial of service (Med-High Risk).

As a workaround, you should disable multipart/form-data parsing and add a rule to detect exploitation, as in the following example, until an update to ModSecurity is in place.

###
# BEGIN TEMPORARY WORKAROUND
#
# Force URLENCODED processing so that the vulnerability is bypassed
# and the REQUEST_BODY target variable is populated, but at the 
# expense of no longer being able to parse multipart form arguments.
SecRule REQUEST_HEADERS:Content-Type "multipart/form-data" \
 "phase:1,t:none,t:lowercase,nolog,pass,ctl:requestBodyProcessor=URLENCODED"
#
# Deny any multipart requests with a zero length header name
SecRule REQUEST_HEADERS:Content-Type "multipart/form-data" \
 "chain,phase:2,t:none,t:lowercase,deny,msg:'ModSecurity DoS attempt - NULL part header name'"
SecRule REQUEST_BODY "(?m:^--.*\\x0d\\x0a?:.*\\x0d\\x0a?--)" "t:none"
#
# END TEMPORARY WORKAROUND
###

More details can be found in the CHANGES file within the released source packages.  Special thanks to those at Red Hat and Internet Security Auditors for disclosing the vulnerabilities.

Both of these are excellent examples of how ModSecurity can be used to provide a virtual patch before an upgrade and/or proper patch can be applied -- even a vulnerability within ModSecurity itself.

More Than A Blocking Device

Unfortunately, most people have pigeon-holed WAFs as only "Attack Blocking Devices" and that is but one use-case option.  This Blog post will highlight another interesting use-case/benefit of a web application firewall - capturing full HTTP transactional logging when web applications generate error conditions.  Due to the fact that a WAF is constantly monitoring live communications between clients and web applications, it is able to identify if or when a web application generates errors.  This may of course be caused by an actual attack in the reqest payload, however I can speak from personal experience in working with both Breach customers and the ModSecurity users in stating that many times applications bomb out due to either a back-end communication problem or from abnormal, non-malicious client input.

Why Are These Errors Occuring?

A typical scenario is that a normal client is interacting with web application and they receive a 500 level responst status code from the web application and they have no idea why.  If this were to happen to a large number of clients and/or if these were reported to the site owners, the next step would be for administrators to try and trouble-shoot and figure out what happened to cause the failure.  The sad reality is that web application exception handling and logging mechanisms are sorely lacking.  How is an administrator supposed to conduct analysis to identify what happened when all they have are terse or cryptic error message such as the following Apache error_log entry - 

[Sat Jan 11 08:52:42 2009] [notice] child pid 24906 exit signal Segmentation fault (11)

Review of the access_log data shows the following -

202.99.11.91 - [11/Jan/2009:08:52:42 +0200] "POST /service/soap/GetItemRequest HTTP/1.1" 500 21364 "-" "Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.8.1.13) Gecko/20080325 Fedora/2.0.0.13-1.fc8 Firefox/2.0.0.13"

This request seems to match up with the time stamp of the error_log entry, however unfortunately, the request uses the POST request method which means that there was data passed in the request body that is not logged by the Common Log Format (CLF).  Good luck figuring out what caused the problem with this data... 

Detailed server-side error and debug logging mechanisms are commonly disabled due to performance/disk space reasons, in favor of allowing the web application to include detailed stack dump data within the error pages presented to clients.  This is not an ideal situation as non-malicious clients probably won't know what to do with these technical details, while malicious clients may leverage the data to fine tune an attack.

Capturing Critical Transaction Data

The audit logging capabilties of web application firewalls allow them to address the two deficiencies outlined above: 1) Capturing the full request data, and 2) Capturing the full application response.  By inspecting the full transactional data, it can be determined if the inbound request was malicious or not and any detailed application error message can be reviewed.

Shortening Trouble-shooting Time

In my previous life, I was the head of operational security for a government client's web applications.  The approach that I found successful when introducing a web application firewall to operational personnel was to demonstrate to them how the auditing capabilities could help them to expedite their trouble-shooting steps.  Believe me, when your job requirements are tied to system uptime, this can be a life saver.

Helping QA/Security Testing 

If the cause of the application error was traced to some abnormality in the client's request, then it is a good idea to utilize that same request data as a part of future QA unit tests to ensure that the updated software correctly handles it. 

The web application firewall audit logs that capture "real" attacks lauched at your site are also very useful for security testing future software.  If you have ever been responsible for QA/Security testing of web applications, then you are familiar with the with the challenges of developing a comprehensive set of attack or abuse test cases.  These audit logs can help your organization create a repository of attack data.

Data Sharing Is The Key

They key take away from this post is that web application firewalls gather a wealth of information that could be utilized by other business units.  Therefore, it is critical that processes are setup to share this information.

In a previous post I showed how you can use both ModSecurity and Apache together to identify/modify SessionIDs that are missing the HTTPOnly flag.  I received some feedback where people were asking how to accomplish the same thing but for the "Secure" cookie flag which instructs the browser to *only* send the SessionID back over an SSL connection. 

If you are only interested in addressing the missing "Secure" cookie flag, then you can simply take the example from the previous post and edit it slightly to swap out "httponly" with "secure".  If, however, you want to try and address both of these issues together, then you will need to change the rule set approach a bit so that it works correctly.  This is because there are now three different scenarios you have to account for -

• Missing HTTPOnly flag
• Missing Secure flag (if the SessionID is being sent over an SSL connection)
• Missing both HTTPOnly and Secure flags

With this in mind, here is an updated rule set that will handle both missing HTTPOnly and Secure cooking flags.

#

# First we want to capture Set-Cookie SessionID data for later inspection

SecRule RESPONSE_HEADERS:/Set-Cookie2?/ "(?i:(j?sessionid|(php)?sessid|(asp|jserv|jw)?session[-_]?(id)?|cf(id|token)|sid))" "phase:3,t:none,pass,nolog,setvar:tx.sessionid=%{matched_var}"

 

#

# We now check the saved SessionID data for the HTTPOnly flag and set an Apache

# ENV variable if it is missing.

SecRule TX:SESSIONID "!(?i:\;? ?httponly;?)" "phase:3,t:none,setenv:httponly_cookie=%{matched_var},pass,log,auditlog,msg:'AppDefect: Missing HttpOnly Cookie Flag.'"

 

#

# Next we check the saved SessionID data for the Secure flag (if this is an SSL session)

# and set an Apache ENV variable if it is missing.

SecRule SERVER_PORT "@streq 443" "chain,phase:3,t:none,pass,log,auditlog,msg:'AppDefect: Missing Secure Cookie Flag.'"

 SecRule TX:SESSIONID "!(?i:\;? ?secure;?)" "t:none,setenv:secure_cookie=%{matched_var}"

 

#

# The final check is to see if BOTH of the HTTPOnly and Secure cookie flags are missing

# and set an Apache ENV variable if they are missing.

SecRule TX:SESSIONID "!(?i:\;? ?httponly;?)" "chain,phase:3,t:none,pass,log,auditlog,msg:'AppDefect: Missing HttpOnly and Secure Cookie Flag.'"

 SecRule SERVER_PORT "@streq 443" "chain,t:none"

  SecRule TX:SESSIONID "!(?i:\;? ?secure;?)" "t:none,setenv:secure_httponly_cookie=%{matched_var}"

 

#

# This last section executes the Apache Header command to

# add the appropriate Cookie flags 

Header set Set-Cookie "%{httponly_cookie}e; HTTPOnly" env=httponly_cookie

Header set Set-Cookie "%{secure_cookie}e; Secure" env=secure_cookie

Header set Set-Cookie "%{secure_httponly_cookie}e; Secure; HTTPOnly" env=secure_httponly_cookie

These rules will both alert and fix these cookie issues.  You may want to switch the actions to "nolog" so that you are not flooded with alerts.

If you are unfamiliar with what the HTTPOnly cookie flag is or why your web apps should use it, please refer to the following resources -

• Mitigating Cross-site Scripting With HTTP-only Cookies - http://msdn.microsoft.com/en-us/library/ms533046.aspx
• OWASP HTTPOnly Overview - http://www.owasp.org/index.php/HTTPOnly

The bottom line is this - while this cookie option flag does absolutely nothing to prevent XSS attacks, it does significanly help to prevent the #1 XSS attack goal which is stealing SessionIDs.  While HTTPOnly is not a "silver bullet" by any means, the potential ROI of implement it is quite large.  Notice I said "potential" as in order to provide the intended protections, two key players have to work together -

• Web Applications - whose job it is to append the "HTTPOnly" flag onto all Set-Cookie response headers for SessionIDs, and
• Web Browsers - whose job it is to identify and enforce the security restrictions on the cookie data so that javascript can not access the contents.

The current challenges to realizing the security benefit of the HTTPOnly flag is that universal adoption in both web apps and browsers is still not there yet.  For example, depending on your web app platform, you may not have an easy mechanism to implementing this feature.  For example - in Java you could following the example provided here on the OWASP site - http://www.owasp.org/index.php/HTTPOnly#Using_Java_to_Set_HTTPOnly, however this doesn't work well for the JSESSIONID as it is added by the framework.  Jim Manico has been fighting the good fight to try and get Apache Tomcat developers to implement his patch to add in HTTPOnly support - http://manicode.blogspot.com/2008/08/httponly-in-tomcat-almost.html.  The point is that with so many different web app development platforms, it isn't going to be easy to find support for this within every web app that you have to secure...

As for browsers - they too have sporadic, non-consistent adoption of HTTPOnly.  It was for this reason that the OWASP Intrinsic Security group has started an RFC Spec for HTTPOnly - http://groups.google.com/group/ietf-httponly-wg.  Hopefully this group will get some traction with the various browser developers.

So, at this point you might be asking yourself - Ryan, that is interesting news and all, but why is this being posted on the ModSecurity site?  What can a web application firewall do to help with this issue?  I would then in turn reply - Great question, I am glad that you asked. ;)

One of my pet peevs with the web application security space is the stigma that is associated with a WAF.  Most everyone only focuses in on the negative security and blocking of attacks aspects of the typical WAF deployment and they fail to realize that WAFs are a highly specialized tool for HTTP.  Depending on your circumstances, you may not ever intend to do blocking.  There are many other use-cases for WAFs and how they can help, in this case as a tactical response tool to help address an underlying vulnerability  In this case, we could monitor when back-end/protected web applications are handing out SessionIDs that are missing the HTTPOnly flag.  This could raise an alert that would notify the proper personnel that they should see if editing the web language code is possible to add this feature in.  A rule to do this with ModSecurity would look like this -

# Identifies SessiondIDs without HTTPOnly flag
#
SecRule RESPONSE_HEADERS:/Set-Cookie2?/ "!(?i:\;? ?httponly;?)" "chain,phase:3,t:none,pass,log,auditlog,msg:'AppDefect: Missing HttpOnly Cookie Flag.'"
  SecRule MATCHED_VAR "(?i:(j?sessionid|(php)?sessid|(asp|jserv|jw)?session[-_]?(id)?|cf(id|token)|sid))" "t:none"

While this rule is pretty useful for identifying and alerting of the issue, many organizations would like to take the next step and try and fix the issue.  If the web application does not have a way to add in the HTTPOnly cookie flag option internally, you can actually leverage ModSecurity+Apache for this purpose.  ModSecurity has the ability to set environmental data that Apache reads/acts upon.  In this case, we can modify our previous rule slightly to use the "setenv" action and then we add an additional Apache "header" directive that will actually overwrite the data with new Set-Cookie data that includes the HTTPOnly flag -

# Identifies SessiondIDs without HTTPOnly flag and sets the "http_cookie" ENV
# Token for Apache to read
SecRule RESPONSE_HEADERS:/Set-Cookie2?/ "!(?i:\;? ?httponly;?)" "chain,phase:3,t:none,pass,nolog"
  SecRule MATCHED_VAR "(?i:(j?sessionid|(php)?sessid|(asp|jserv|jw)?session[-_]?(id)?|cf(id|token)|sid))" "t:none,setenv:http_cookie=%{matched_var}"

# Now we use the Apache Header directive to set the new data

Header set Set-Cookie "%{http_cookie}e; HTTPOnly" env=http_cookie

The end result of this ruleset is that ModSecurity+Apache can transparently add on the HTTPOnly cookie flag on the fly to any Set-Cookie data that you define.  Thanks goes to Brian Rectanus from Breach for working with me to get the Header directive syntax correct.

One note of warning - make sure that you understand how the web application is handling setting SessionIDs meaning if they are created server-side vs. client-side (in javascript).  This rule set will work fine if the SessionIDs are generated server-side.  If they are created client-side, however, this will disrupt session management.

Hopefully the data presented here will help people who would like to have the security benefit of this flag however are running into challenges with implementing it within the app.

Deciding to leave a job is rarely easy, but leaving your job and the project you've nurtured for six years is very difficult. In a few weeks' time I will no longer be working for Breach Security and, as a consequence, I will be joining the ModSecurity community as a user. I leave ModSecurity in the capable hands of Brian Rectanus, who has been taking care of it for some time now.

It's been an exciting journey, starting ModSecurity and growing it for all these years. It's been a pleasure working with the community and helping others solve their problems. It's been hard at times, but that made it all the more interesting.

Finally, I would like to take this opportunity to thank every single ModSecurity user, because we, all together, made ModSecurity what it is. I couldn't have done it without you.

I spent the last week importing ModSecurity's source code repository into subversion at Source Forge. I am proud to announce that a read-only version of ModSecurity's subversion repository is now publicly available. In addition to this, Atlassian has graciously given the ModSecurity project a license to use their software. We are currently using Jira to track reported issues, FishEye to track and analyze the code repository and have plans to add Confluence (a wiki) in the near future. Together, these tools allow users to test and apply fixes immediately as well as monitor the progress of reported issues and the progress of ModSecurity as a whole.

This is a big step forward for the ModSecurity project. The project is becoming much more open and feels more like an Open Source project should. Please take some time to look at the new tools. Comments are always welcome.

This year, the OWASP's Summer of Code event contains one project that's of particular interest to me (and possibly to you, consider that you're following this blog): Securing WebGoat Using ModSecurity. If you've even seen WebGoat (a learning sandbox that contains samples of many application security issues) then you know how difficult it might be to secure. It's true; it's probably the worst-case scenario for ModSecurity, because it's not your typical application, and it contains a number of problems that require the understanding of application state to exploit (and thus protect too). So we should all congratulate Stephen Evans for pulling through and reaching the end of the project.

I find the project interesting because it stretches the boundaries of what ModSecurity can do. For example, one of the features Stephen relied heavily on is the Lua scripting language, which is currently marked as experimental. As we are currently working on the design for ModSecurity 3, Stephen's feedback is going to be very useful to us.

If this sort of thing is indeed of interest to you then you need to know that Stephen will be sharing his experiences during the OWASP Summit in Portugal next week. If you'll be attending the summit (or even if you can schedule a trip on short notice) do consider attending.

Intrusion detection is a well-known network security technique--it introduces monitoring and correlation devices to networks, enabling administrators to monitor events and detect attacks and anomalies in real-time. Web intrusion detection does the same but it works on the HTTP level, making it suitable to deal with security issues in web applications. This session will start with an overview of web intrusion detection and web application firewalls, discussing where they belong in the overall protection strategy. The second part of the talk will discuss ModSecurity and its capabilities. ModSecurity is an open source web application firewall that can be deployed either embedded (in the Apache HTTP server) or as a network gateway (as part of a reverse proxy deployment). Now in it's sixth year of development, ModSecurity is mature, robust and flexible. Due to its popularity and wide usage it is now positioned as a de-facto standard in the web intrusion detection space.

This is the same talk I presented at ApacheCon Europe 2008 in Amsterdam and the OWASP AppSec US 2008 conference in New York, and it's getting better every time I do it. This is going to be my first ApacheCon in the US and I am really looking forward to it. It's the place to be if you are involved in any way with the projects of the Apache Software Foundation.

I will be giving the updated version of our ModProfiler presentation this Sunday (14th) at the OWASP Israel 2008 conference. ModProfiler has seen a release or two since Black Hat (where it was announced) so I can now speak with more experience about some of the challenges of automated positive model generation.

Ofer (who is organising the conference, by the way) will be speaking about What's Hot in 2008, which is always an interesting and well-attended talk.

The conference schedule looks great, but, judging from the presenters' names alone, I doubt many will be in English. (Not that it matters, as I will have to spend the afternoon working, back at Breach Security's offices in Herzliya.) This conference is the first time that I've seen anyone discussing database firewalls (let alone an open source one) at an application security conference: Yuli Stremovsky will be discussing GreenSQL. We should have that sort of thing more often. In case you're interested, another similar project is Open Database Security and Monitoring Solutions (ODSMS).