HAProxy – IIS and X-Forward-For Header
IIS and X-Forwarded-For Header
So, you’re using IIS and you want to track your clients by IP address in your IIS logs. Unfortunately, out of the tin, this is not directly supported. The X-Forwarded-For (XFF) HTTP header is an industry standard method to find the IP address of a client machine that is connecting to your web server via an HTTP proxy, load balancer etc. Fortunately, depending on the version of IIS being used, there are a number of ways to enable this.
A – IIS 7 & later :
Microsoft do now have a solution – it’s called IIS Advanced Logging. This is an installable IIS feature and can be downloaded here. Once installed on the IIS server, you’ll see an extra option called ‘Advanced Logging’ for the sites in IIS.
Once installed, follow the steps below to add the X-Forwarded-For log field to IIS:
1. From your Windows Server 2008 or Windows Server 2008 R2 device, open IIS Manager
2. From the Connections navigation pane, click the appropriate server, web site, or directory on which you are configuring Advanced Logging. The Home page appears in the main panel
3. From the Home page, under IIS, double-click Advanced Logging
4. From the Actions pane on the right, click Edit Logging Fields
5. From the Edit Logging Fields dialog box, click the Add Field button, and then complete the following:
-in the Field ID box, type X-Forwarded-For
-from the Category list, select Default
-from the Source Type list, select Request Header
-in the Source Name box, type X-Forwarded-For
-click the OK button in the Add Logging Field box, and then click the OK button in the Edit Logging Fields box
6. Click a Log Definition to select it. By default, there is only one: %COMPUTERNAME%-Server. The log definition you select must have a status of Enabled
7. From the Actions pane on the right, click Edit Log Definition
8. Click the Select Fields button, and then check the box for the X-Forwarded-For logging field
9. Click the OK button
10. From the Actions pane, click Apply
11. Click Return To Advanced Logging
12. In the Actions pane, click Enable Advanced Logging
Now, when you look at the logs the client IP address is included.
B – IIS 6 :
Unfortunatey the Microsoft solution mentioned above is not available for IIS 6. luckily there are a number of solutions available to address this limitation – some that cost money and others that have been released as open source. One excellent example that we’ve tested with our products is F5′s X-Forwarded-For ISAPI filter. It’s avaialable in both in 32 & 64 bit versions.
1. Download the zipped archive from here and extract to an appropriate folder
2. Navigate to the relevant version (32 or 64 bit)
3. Copy F5XForwardedFor.dll to a suitable location on your server, e.g. C:ISAPIfilters
4. Make sure you have ISAPI Filters enabled on your IIS server
5. Open IIS Manager, right-click the site and select Properties
6. Select the ISAPI Filters tab
7. Click ‘add’, then in the popup enter a suitable name and select the DLL file stored in step 3
8. Restart your website
That’s it – you should now start seeing the IP address of the client PC’s in your IIS logs rather than the IP of the load balancer.
———————————————————————————————————————————–
IIS X-Forward-For ISAPI Filter
A recent customer issue came up where they were load balancing servers but we unable to get the true client address logged in their IIS logs. They had their servers fronted by a BIG-IP and when clients would make requests the address passed to the server was the internal address of the BIG-IP and not that of the client.
This is a common issue with proxies and fortunately there is a standard for forwarding client information. It is the HTTP X-Forwarded-For header which is handled by most proxies. So, I set out to find an existing ISAPI filter to replace the c-ip (client ip) log value in IIS with the contents of the X-Forwarded-For header (if it exists). I was amazed to find that I couldn’t find a single instance of any open source (or even commercial) filter that would do this.
So, I dug out Visual Studio and whipped up a filter that does just that. It’s very basic and contains no user configuration so all you need to do is plug it into your Web Applications list of ISAPI Filters within the IIS Administration and you’re set to go.
We’ve released the source under the iControl End User License Agreement (available in any iControl SDK download). You can download it in the CodeShare section of DevCentral. If you find a way to optimize this filter, please let me know and I’ll update the sources here.
After 24-hours of posting, a customer already returned some performance testing on the filter indicating that it only effected the traffic by less than 1 percent. I’m sure there are ways to optimize the memory allocation in the filter to speed this up a bit more, but I’ll leave that for the community to work on.
Oh, and it should be noted that the X-Forwarded-For header isn’t supported the same way across all proxy products so you’ll want to make sure you test this out before using it. It is expecting the header to only contain an IP Address as it does a straight substitution on the value in the c-ip section of the log entry.
Enjoy!
Setting up a Reverse Proxy using IIS, URL Rewrite and ARR
Today there was a question in the IIS.net Forums asking how to expose two different Internet sites from another site making them look like if they were subdirectories in the main site.
So for example the goal was to have a site: www.site.com expose a www.site.com/company1 and a www.site.com/company2 and have the content from “www.company1.com” served for the first one and “www.company2.com” served in the second one. Furthermore we would like to have the responses cached in the server for performance reasons. The following image shows a simple diagram of this:
This sounds easy since its just about routing or proxying every single request to the correct servers, right? Wrong!!! If it only it was that easy. Turns out the most challenging thing is that in this case we are modifying the structure of the underlying URLs and the original layout in the servers which makes relative paths break and of course images, Stylesheets (css), javascripts and other resources are not shown correctly.
To try to clarify this, imagine that a user requests using his browser the page at http://www.site.com/company1/default.aspx, and so based on the specification above the request is proxied/routed to http://www.company1.com/default.aspx on the server-side. So far so good, however, imagine that the markup returned by this HTML turns out to have an image tag like “<img src=/some-image.png />”, well the problem is that now the browser will resolve that relative path using the base path on the original request he made which was http://www.site.com/company1/default.aspx resulting in a request for the image at http://www.site.com/some-image.png instead of the right “company1” folder that would be http://www.site.com/company1/some-image.png .
Do you see it? Basically the problem is that any relative path or for that matter absolute paths as well need to be translated to the new URL structure imposed by the original goal.
So how do we do it then?
Enter URL Rewrite 2.0 and Application Request Routing
URL Rewrite 2.0 includes the ability to rewrite the content of a response as it is getting served back to the client which will allow us to rewrite those links without having to touch the actual application.
Software Required:
- IIS 7.0 or IIS 7.5 (included in Windows Vista, Windows 2008, Windows 7 or Windows 2008 R2).
- Install URL Rewrite 2.0
- Install Application Request Routing
Steps
- The first thing you need to do is enable Proxy support in ARR.
- To do that just launch IIS Manager and click the server node in the tree view.
- Double click the “Application Request Routing Cache” icon
- Select the “Server Proxy Settings…” task in the Actions panel
- And Make sure that “Enable Proxy” checkbox is marked. What this will do is allow any request in the server that is rewritten to a server that is not the local machine will be routed to the right place automatically without any further configuration.
- Configure URL Rewrite to route the right folders and their requests to the right site. But rather than bothering you with UI steps I will show you the configuration and then explain step by step what each piece is doing.
- Note that for this post I will only take care of Company1, but you can imagine the same steps apply for Company2, and to test this you can just save the configuration file below as web.config and save it in your inetpub\wwwroot\ or in any other site root and you can test it.
<configuration>
<system.webServer>
<rewrite>
<rules>
<rule name=”Route the requests for Company1″ stopProcessing=”true”>
<match url=”^company1/(.*)” />
<conditions>
<add input=”{CACHE_URL}” pattern=”^(https?)://” />
</conditions>
<action type=”Rewrite” url=”{C:1}://www.company1.com/{R:1}” />
<serverVariables>
<set name=”HTTP_ACCEPT_ENCODING” value=”” />
</serverVariables>
</rule>
</rules>
<outboundRules>
<rule name=”ReverseProxyOutboundRule1″ preCondition=”ResponseIsHtml1″>
<match filterByTags=”A, Area, Base, Form, Frame, Head, IFrame, Img, Input, Link, Script” pattern=”^http(s)?://www.company1.com/(.*)” />
<action type=”Rewrite” value=”/company1/{R:2}” />
</rule>
<rule name=”RewriteRelativePaths” preCondition=”ResponseIsHtml1″>
<match filterByTags=”A, Area, Base, Form, Frame, Head, IFrame, Img, Input, Link, Script” pattern=”^/(.*)” negate=”false” />
<action type=”Rewrite” value=”/company1/{R:1}” />
</rule>
<preConditions>
<preCondition name=”ResponseIsHtml1″>
<add input=”{RESPONSE_CONTENT_TYPE}” pattern=”^text/html” />
</preCondition>
</preConditions>
</outboundRules>
</rewrite>
</system.webServer>
</configuration>
Setup the Routing
<rule name=”Route the requests for Company1″ stopProcessing=”true”>
<match url=”^company1/(.*)” />
<conditions>
<add input=”{CACHE_URL}” pattern=”^(https?)://” />
</conditions>
<action type=”Rewrite” url=”{C:1}://www.company1.com/{R:1}” />
<serverVariables>
<set name=”HTTP_ACCEPT_ENCODING” value=”” />
</serverVariables>
</rule>
The first rule is an inbound rewrite rule that basically captures all the requests to the root folder /company1/*, so if using Default Web Site, anything going to http://localhost/company1/* will be matched by this rule and it will rewrite it to www.company1.com respecting the HTTP vs HTTPS traffic.
One thing to highlight which is what took me a bit of time is the “serverVariables” entry in that rule that basically is overwriting the Accept-Encoding header, the reason I do this is because if you do not remove that header then the response will likely be compressed (Gzip or deflate) and Output Rewriting is not supported on that case, and you will end up with an error message like:
HTTP Error 500.52 – URL Rewrite Module Error.
Outbound rewrite rules cannot be applied when the content of the HTTP response is encoded (“gzip”).
Also note that to be able to use this feature for security reasons you need to explicitly enable this by allowing the server variable. See enabling server variables here.
Outbound Rewriting to fix the Links
The last two rules just rewrite the links and scripts and other resources so that the URLs are translated to the right structure. The first one rewrites absolute paths, and the last one rewrites the relative paths. Note that if you use relative paths using “..” this will not work, but you can easily fix the rule above, I was too lazy to do that and since I never use those when I create a site it works for me 🙂
Setting up Caching for ARR
A huge added value of using ARR is that now we can with a couple of clicks enable disk caching so that the requests are cached locally in the www.site.com, so that not every single request ends up paying the price to go to the backend servers.
- To do that just launch IIS Manager and click the server node in the tree view.
- Double click the “Application Request Routing Cache” icon
- Select the “Add Drive…” task in the Actions panel.
- Specify a directory where you want to keep your cache. Note that this can be any subfolder in your system.
- Make sure that “Enable Disk Cache” checkbox is marked in the Server Proxy Settings mentioned above.
As easy as that now you will see caching working and your site will act as a container of other servers in the internet. Pretty cool hah! 🙂
So in this post we saw how with literally few lines of XML, URL Rewrite and ARR we were able to enable a proxy/routing scenario with the ability to rewrite links and furthermore with caching support.
Maximum request length exceeded
If you are using IIS for hosting your application, then the default upload file size if 4MB. To increase it, please use this below section in your web.config –
<configuration>
<system.web>
<httpRuntime maxRequestLength="1048576" />
</system.web>
</configuration>
Just to add – If you are using IIS7 then you need to use below lines instead of above –
<system.webServer>
<security>
<requestFiltering>
<requestLimits maxAllowedContentLength="1048576000" />
</requestFiltering>
</security>
</system.webServer>
Note: maxAllowedContentLength
is measured in bytes while maxRequestLength
is measured in kilobytes, which is why the values differ in this config example.
I don’t think it’s been mentioned here, but to get this working, I had to supply both of these values in the web.config:
In system.web
<httpRuntime maxRequestLength="1048576" executionTimeout="3600" />
And in system.webServer
<security>
<requestFiltering>
<requestLimits maxAllowedContentLength="1073741824" />
</requestFiltering>
</security>
IMPORTANT: Both of these values must match. In this case, my max upload is 1024 megabytes.
maxRequestLength has 1048576 KILOBYTES, and maxAllowedContentLength has 1073741824 BYTES.
I know it’s obvious, but it’s easy to overlook.
It may be worth noting that you may want to limit this change to the URL you expect to be used for the upload rather then your entire site.
<location path="Documents/Upload">
<system.web>
<!-- 50MB in kilobytes, default is 4096 or 4MB-->
<httpRuntime maxRequestLength="51200" />
</system.web>
<system.webServer>
<security>
<requestFiltering>
<!-- 50MB in bytes, default is 30000000 or approx. 28.6102 Mb-->
<requestLimits maxAllowedContentLength="52428800" />
</requestFiltering>
</security>
</system.webServer>
</location>