Varnish + logging + geoip howto

Varnish is the new buzzword making the rounds on the “speed up your website circuit. Does it work? Well suffice to say YMMV…. earlier there was Squid as well as Apache (in reverse proxy mode) so the difference will probably be in how it handles dynamic content and the resources it eats up.

So this one is about how to get geo targetting to work when you have a proxy software caching your static content (and also dynamic content if you set headers in your scripts properly). The way a proxy works is that it sits between your webserver and the visitor. It keeps a store of objects (images, static content etc.) and instead of over burdening your webserver with requests it frees up the task of serving content (or atleast reduces the frequency of requests). All good and happy happy specially for the server admin and moderately for the visitor as well coz response time is slightly reduced.

One of the problems you would face is when you look at your webserver access logs and try to figure out where your traffic is coming from. Since all your traffic is actualy now coming from the proxy you have just one IP in the logs, that of your varnish install. Here you would be advised to change the logformat of your access logs. The way to do that for apache is to set the log format like such:

LogFormat '%{X-Forwarded-For}i %l %u %t "%r" %>s %b "%{Referer}i" "%{User-Agent}i"' combined

Then make sure that the access log format is set to combined and restart apache. This will start logging the visitors IP instead of the proxy servers address in the access logs and then you can proceed to use your statistics programs as in the pre-proxy era.

For GeoIP also you need to make a few changes. What I use is the php geoip.inc method (keep switching between litespeed, lighttpd, apache depending on what I’m researching at the moment) so instead of mod_geoip the include method works best for me.
So a small change to the code:


include ("geoip.inc");
$headers = apache_request_headers();
$ip = $headers["X-Forwarded-For"];
$gi = geoip_open("/usr/local/share/GeoIP/GeoIP.dat",GEOIP_STANDARD);
$country_code=geoip_country_code_by_addr($gi,$ip);
echo $country_code;

This is using the geoip.inc and the php geoip extension from maxmind.

If you need help with varnish, website tuning, geo targetting or any thing related to linux webserver other than this please contact us

Some things to look at in geoip.inc (if you get the cannot redeclare errors)
replace these two function definitions.

if (!function_exists('geoip_country_code_by_name')){
function geoip_country_code_by_name($gi, $name) {
$country_id = geoip_country_id_by_name($gi,$name);
if ($country_id !== false) {
return $gi->GEOIP_COUNTRY_CODES[$country_id];
}
return false;
}
}
if (!function_exists('geoip_country_name_by_name')){
function geoip_country_name_by_name($gi, $name) {
$country_id = geoip_country_id_by_name($gi,$name);
if ($country_id !== false) {
return $gi->GEOIP_COUNTRY_NAMES[$country_id];
}
return false;
}
}

In the vcl for your varnish instance handling the website add this:

sub vcl_pass {

set bereq.http.connection = "close";

if (req.http.X-Forwarded-For) {

set bereq.http.X-Forwarded-For = req.http.X-Forwarded-For;

} else {

set bereq.http.X-Forwarded-For = regsub(client.ip, ":.*", "");

}

}

Leave a Reply

Your email address will not be published. Required fields are marked *