What is Request Filtering?

Request filtering refers to blocking unwanted HTTP requests from reaching your website. Usually with an ASP.NET website or web application, the web application processes the request, and returns a response. Some requests however, we don’t even want to bother processing, and can filter them out immediately on sight. Common examples of this might be file uploads which state (in the HTTP headers) that they’re about to upload gigabytes of data - in this case the server will aggressively block the request rather than wait until all the data has been uploaded to return an error message!

Why build a Kentico module for it?

The idea initially occurred to me because of two seemingly clashing Kentico "best practice" guidelines:

1. You should log all 404 responses.
2. You should not have 404 responses in your Event Log.

This is fine if the only 404s you're getting on your site are from broken links, but that's not always the case. Requests for URLs that return a 404 can come from many sources, and many of them are simply outside of our control. For example, there are a huge number of automated bots or scripts out there which relentlessly crawl the web, testing every combination of URL to try to find a vulnerability in your website.

So we decided we needed to be able to filter out the "bad" 404s, and only log the "good" 404s that we could do something about.

A few other reasons to build the module right into Kentico:

1. Very few Kentico users have access to an advanced Web Application Firewall or the ability to install and configure Request Filtering rules on their web server directly.
2. Changing Request Filtering rules often requires direct access to the web server, or if configured through a web.config file, will interrupt the functioning of the site when configuration is changed.
3. Configuring request filtering for a content-managed site is closely related to other admin functions such as viewing Event Log reports for 404s and exceptions (such as dangerous requests).
4. We had a need for more complex yet user-friendly request filtering rules (simple regex anyone?).

How does the Kentico Request Filtering module work?

• You configure Request Filtering rules through a custom module, or app, in the Kentico admin interface.
• These rules are loaded into memory when the application starts, or whenever they are changed.
• An ASP.NET HTTPModule executes on every request, and checks if this request matches any of your request filtering rules.
• If there’s a match, that request is blocked immediately, with an empty 403 HTTP response. No database queries are executed and nothing is logged in Kentico - the rest of the application is blissfully unaware that the request was even made.

Pros and Cons

Benefits include:

• Powerful rules such as regular expression matching, meaning one rule could remove huge amounts of unwanted traffic.
• A much cleaner event log, with only “real” 404s to review.
• Improved performance (compared to 404s, errors or redirects), as filtered requests hardly affect the server at all.

Issues or risks include:

• Very powerful, so easy to accidentally break functionality in your site.
• Definitely not a solution for all of your 404s - sometimes a 404 is exactly what’s required!j
  Please check out my new expanded post on Handling 404s in your Kentico Event Log for more options.

Can Haz?

The module has been published as a Nuget package (instructions on how to install packages here).