Introduction

OK, you now have varnish installed on your server and you started to catch the pages on it, so your webserver is now missing the hits to your pages.

Well, that is great for your webserver load, but not too good for your statistics analysis, because /var/log/apache/access_log will miss the hit.

In case you don’t know yet about Varnish, here is something about it from Wikipedia:

Varnish is an HTTP accelerator designed for content-heavy dynamic web sites. In contrast to other HTTP accelerators, such as Squid, which began life as a client-side cache, or Apache, which is primarily an origin server, Varnish was designed from the ground up as an HTTP accelerator.

Now, let’s focus on how to get real logs Apache style from Varnish Cache.

Configure varnishncsa

We’ll use varnishncsa to get the logs that awstats will be able to analyse.

Varnishncsa: Display Varnish logs in Apache / NCSA combined log format

The syntax is:

varnishncsa [?a] [?b] [?C] [?c] [?D] [?d] [?f] [?I regex] [?i tag] [?n varnish_name] [?P file] [?r file] [?V] [?w file] [?X regex] [?x tag]

What I did is to add this line in the /etc/rc.local file:

varnishncsa -a -w /var/log/varnish/access.log -D -P /var/run/varnishncsa.pid

That line tells varnishncsa

-a: To append the logs to an already existing file -w: To write the logs to the /var/log/varnish/access.log file -D: To run varnishncsa as a daemon -P: To write the PID file in the /var/run/ folder

We now have varnishncsa up and running, now configure logrotate to rotate the logs, everyday at midnight.

Create the following file /etc/logrotate.d/varnish and put this contents on it:

        /var/log/varnish/*log {
                create 640 http log
                compress
                postrotate
                        /bin/kill -USR1 `cat /var/run/varnishncsa.pid 2>/dev/null` 2> /dev/null || true
                endscript
        }

If you need more info or options run: man logrotate

Done, we now have varnish writing logs to a file, and logrotate will rotate them everyday, we only need now to analyse them.

Configure awstats with varnish

Download and install awstats

Get the lastest version from here, and install it basically all you need to do is to copy the contents of the .tar.gz file in /usr/local/awstats/ folder, and run the awstats_configure.pl tool, skip (write none) in the webserver config file question, then follow the questions and you will end up with a config file like: /etc/awstats/awstats.www.your.domanin.conf and should look more or less like this:

LogFile="/var/log/varnish/access.log"
LogType=W
LogFormat=1
LogSeparator=" "
SiteDomain="www.go2linux.org"
HostAliases="go2linux.org www.go2linux.org 127.0.0.1 localhost"
# Possible values:
# 0 - No DNS Lookup
# 1 - DNS Lookup is fully enabled
# 2 - DNS Lookup is made only from static DNS cache file (if it exists)
# Default: 2
DNSLookup=1
DirData="/var/lib/awstats"
DirIcons="/icon"
AllowToUpdateStatsFromBrowser=0
AllowFullYearView=2

Be sure the read the example located at /usr/local/awstats/wwwroot/cgi-bin/awstats.model.conf to fine tune your configuration.

Make awstats update hourly

Create the file /etc/cron.hourly/awstats and copy these lines inside the file:

#!/bin/sh
/usr/local/awstats/wwwroot/cgi-bin/awstats.pl -update -config=www.your.domain

Note: Change www.your.domain with your real domain name, the same one you entered when you configured awstats before.

That will update your awstats database hourly, let’s now make if visible, better only for you:

Copy these folders to your www root directory:

/usr/local/awstats/wwwroot/clases /usr/local/awstats/wwwroot/css /usr/local/awstats/wwwroot/icon

Do it with these commands.

cp -R /usr/local/awstats/wwwroot/clases /your-html-root-directory

The same way with the rest of them, be sure to replace your-html-root-directory with something like /html_public/root/ or anything you may have as your root directory, and be sure to use the -R options to copy recursively.

Now edit the /etc/httpd/conf/httpd.conf file and add this lines:

ScriptAlias /awstats/ "/usr/local/awstats/wwwroot/cgi-bin/"

<Directory "/usr/local/awstats/wwwroot">
    AllowOverride None
    Options None
    AuthType Basic
    AuthName 'Private scripts'
    AuthUserFile '/public_html/root/.htpasswd'
    Require valid-user

    Order allow,deny
    Allow from all

And now create a user that will be able to read the stats:

httpasswd /public_html/root/.htpasswd user

Replace /public_html/root/.htpasswd with the file and folder where you want the password file to be in, but be sure the file and upwards folders are owned by the user that runs your Apache server.

Final Steps

Now restart Apache, and start varnishnsca.

varnishncsa -a -w /var/log/varnish/access.log -D -P /var/run/varnishncsa.pid

And the command to restart Apache (It is not the same in all distributions).