Statistics Tutorials

Count search engine referrals using web server logs and shell commands

Suppose you want to know how much of your website traffic comes from Google or another search engine. It's very easy if you use Awstats or Google analytics, but what if you haven't configured these tools? Using only the web server log files and some shell commands will enable us to quickly parse log files and give a count of how many referrals we had from a search engine.

For the purposes of this tutorial, we will be assuming an Apache web server, and that the search engine that we are counting the referrals from is Google.

Ignore IP Address Awstats

In this tutorial, we will learn how to ignore or exclude specific IP addresses from our Awstats data. When set up, Awstats parses our server log files and generates detailed reports of user locations, IP addresses, referrers, and so on. Awstats can be used to give an excellent idea of where your visitors are coming from, how they got to your site, and what they are looking on once on your site.

Using this information will allow a webmaster to fine-tune aspects of his or her site for maximum user satisfaction.