CentOS7

バーチャルホスト用アクセスログ解析(AWStats)

AWStatsを利用してバーチャルホストのアクセスログを解析し、バーチャルホスト用へのアクセス統計をWebブラウザから参照できるようにする。

なお、ここでの環境条件は以下のとおりとする。

バーチャルホストAWStats導入済であること

AWStats設定

[root@localhost ~]# cp /etc/awstats/awstats.model.conf /etc/awstats/awstats.usage.com.conf ← 設定ファイルをサンプルよりコピー
[root@localhost ~]# vi /etc/awstats/awstats.usage.com.conf ← 設定ファイル編集
# "LogFile" contains the web, ftp or mail server log file to analyze.
# Possible values: A full path, or a relative path from awstats.pl directory.
# Example: "/var/log/apache/access.log"
# Example: "../logs/mycombinedlog.log"
# You can also use tags in this filename if you need a dynamic file name
# depending on date or time (Replacement is made by AWStats at the beginning
# of its execution). This is available tags :
# %YYYY-n is replaced with 4 digits year we were n hours ago
# %YY-n is replaced with 2 digits year we were n hours ago
# %MM-n is replaced with 2 digits month we were n hours ago
# %MO-n is replaced with 3 letters month we were n hours ago
# %DD-n is replaced with day we were n hours ago
# %HH-n is replaced with hour we were n hours ago
# %NS-n is replaced with number of seconds at 00:00 since 1970
# %WM-n is replaced with the week number in month (1-5)
# %Wm-n is replaced with the week number in month (0-4)
# %WY-n is replaced with the week number in year (01-52)
# %Wy-n is replaced with the week number in year (00-51)
# %DW-n is replaced with the day number in week (1-7, 1=sunday)
# use n=24 if you need (1-7, 1=monday)
# %Dw-n is replaced with the day number in week (0-6, 0=sunday)
# use n=24 if you need (0-6, 0=monday)
# Use 0 for n if you need current year, month, day, hour...
# Example: "/var/log/access_log.%YYYY-0%MM-0%DD-0.log"
# Example: "C:/WINNT/system32/LogFiles/W3SVC1/ex%YY-24%MM-24%DD-24.log"
# You can also use a pipe if log file come from a pipe :
# Example: "gzip -d </var/log/apache/access.log.gz
"
# If there is several log files from load balancing servers :
# Example: "/pathtotools/logresolvemerge.pl *.log
"
#
LogFile="/var/log/httpd/access_log"

LogFile="/var/log/httpd/usage.com-access_log" ← バーチャルホストアクセスログを指定

# "SiteDomain" must contain the main domain name, or the main intranet web
# server name, used to reach the web site.
# If you share the same log file for several virtual web servers, this
# parameter is used to tell AWStats to filter record that contains records for
# this virtual host name only (So check that this virtual hostname can be
# found in your log file and use a personalized log format that include the
# %virtualname tag).
# But for multi hosting a better solution is to have one log file for each
# virtual web server. In this case, this parameter is only used to generate
# full URL's links when ShowLinksOnUrl option is set to 1.
# If analysing mail log, enter here the domain name of mail server.
# Example: "myintranetserver"
# Example: "www.domain.com"
# Example: "ftp.domain.com"
# Example: "domain.com"
#
SiteDomain="localhost.localdomain"

SiteDomain="usage.com" ← Webサーバー名を指定

# If you want to have hosts reported by name instead of ip address, AWStats
# need to make reverse DNS lookups (if not already done in your log file).
# With DNSLookup to 0, all hosts will be reported by their IP addresses and
# not by the full hostname of visitors (except if names are already available
# in log file).
# If you want/need to set DNSLookup to 1, don't forget that this will reduce
# dramatically AWStats update process speed. Do not use on large web sites.
# Note: Reverse DNS lookup is done on IPv4 only (Enable ipv6 plugin for IPv6).
# Note: Result of DNS Lookup can be used to build the Country report. However
# it is highly recommanded to enable the plugin 'geoipfree' or 'geoip' to
# have an accurate Country report with no need of DNS Lookup.
# Possible values:
# 0 - No DNS Lookup
# 1 - DNS Lookup is fully enabled
# 2 - DNS Lookup is made only from static DNS cache file (if it exists)
# Default: 2
#
DNSLookup=2

DNSLookup=1 ← アクセス元の名前解決を行う(アクセス元をIPアドレスではなくホスト名で表示する)

# Plugin: GeoIPfree
# Perl modules required: Geo::IPfree version 0.2+ (from Graciliano M.P.)
# Country chart is built from an Internet IP-Country database.
# This plugin is useless for intranet only log files.
# Note: You must choose between using this plugin (need Perl Geo::IPfree
# module, database less up to date) or the GeoIP plugin (need Perl Geo::IP
# module from Maxmind, database more up to date).
# Note: Activestate provide a corrupted version of Geo::IPfree 0.2 Perl
# module, so install it from elsewhere (from www.cpan.org for example).
# This plugin reduces AWStats speed of 10% !
#
#LoadPlugin="geoipfree"

LoadPlugin="geoipfree" ← コメント解除(アクセス元国情報を詳細に取得)

AWStatsデータベース作成

AWStatsデータベース初期作成

Apacheのバーチャルホスト分のログファイルを過去分も含めて全てAWStatsのデータベースに取り込む

[root@localhost ~]# vi awstatsinit.sh ← AWStatsデータベース初期作成スクリプト作成
#!/bin/sh

# データベース作成先ディレクトリ作成
mkdir -p `grep ^DirData /etc/awstats/awstats.$1.conf | awk -F= '{print $2}'|tr -d \"`

# AWStats設定ファイルよりApacheログファイル名取得
logfile=`grep ^LogFile /etc/awstats/awstats.$1.conf|sed -e 's/LogFile="\([^ ]*\)"/\1/p' -e d`

# 過去ログを古い順に処理
grep dateext /etc/logrotate.conf > /dev/null 2>&1
if [ $? -eq 0 ]; then
# 過去ログファイル名に日付が設定されている場合
for log in `ls $logfile-*|sort`
do
`rpm -ql awstats|grep "awstats\.pl"` \
-config=$1 -update -logfile=$log
done
else
# 過去ログファイル名に通番が設定されている場合
for log in `ls $logfile.*|sort -r`
do
`rpm -ql awstats|grep "awstats\.pl"` \
-config=$1 -update -logfile=$log
done
fi

# 現在ログ処理
for log in `ls $logfile|sort`
do
`rpm -ql awstats|grep "awstats\.pl"` \
-config=$1 -update -logfile=$log
done
[root@localhost ~]# sh awstatsinit.sh usage.com ← AWStatsデータベース初期作成スクリプト実行
※ログ量により時間がかかる
Update for config "/etc/awstats/awstats.usage.com.conf"
With data in log file "/var/log/httpd/usage.com-access_log.4"...
Phase 1 : First bypass old records, searching new record...
Searching new records from beginning of log file...
Phase 2 : Now process new records (Flush history on disk after 20000 hosts)...
Jumped lines in file: 0
Parsed lines in file: 123047
Found 1821 dropped records,
Found 0 corrupted records,
Found 0 old records,
Found 121226 new qualified records.
Update for config "/etc/awstats/awstats.usage.com.conf"
With data in log file "/var/log/httpd/usage.com-access_log.3"...
Phase 1 : First bypass old records, searching new record...
Direct access to last remembered record has fallen on another record.
So searching new records from beginning of log file...
Phase 2 : Now process new records (Flush history on disk after 20000 hosts)...
Jumped lines in file: 0
Parsed lines in file: 130918
Found 1700 dropped records,
Found 0 corrupted records,
Found 0 old records,
Found 129218 new qualified records.
Update for config "/etc/awstats/awstats.usage.com.conf"
With data in log file "/var/log/httpd/usage.com-access_log.2"...
Phase 1 : First bypass old records, searching new record...
Direct access to last remembered record is out of file.
So searching it from beginning of log file...
Phase 2 : Now process new records (Flush history on disk after 20000 hosts)...
Jumped lines in file: 0
Parsed lines in file: 128865
Found 1680 dropped records,
Found 0 corrupted records,
Found 0 old records,
Found 127185 new qualified records.
Update for config "/etc/awstats/awstats.usage.com.conf"
With data in log file "/var/log/httpd/usage.com-access_log.1"...
Phase 1 : First bypass old records, searching new record...
Direct access to last remembered record is out of file.
So searching it from beginning of log file...
Phase 2 : Now process new records (Flush history on disk after 20000 hosts)...
Jumped lines in file: 0
Parsed lines in file: 115202
Found 1532 dropped records,
Found 0 corrupted records,
Found 0 old records,
Found 113670 new qualified records.
Update for config "/etc/awstats/awstats.usage.com.conf"
With data in log file "/var/log/httpd/usage.com-access_log"...
Phase 1 : First bypass old records, searching new record...
Direct access to last remembered record is out of file.
So searching it from beginning of log file...
Phase 2 : Now process new records (Flush history on disk after 20000 hosts)...
Jumped lines in file: 0
Parsed lines in file: 21746
Found 312 dropped records,
Found 0 corrupted records,
Found 0 old records,
Found 21434 new qualified records.
[root@localhost ~]# rm -f awstatsinit.sh ← AWStatsデータベース初期作成スクリプト削除

※以後のAWStatsデータベースの更新は/etc/cron.daily/awstatsにより毎日自動実行される

AWStats確認

http://usage.com/awstats/awstats.pl へアクセスしてアクセス統計ページが表示されることを確認

アクセス統計ページHTML版の作成

awstats.plは、実行するたびにAWStatsのデータベースからログ情報を入力してアクセス統計ページを作成するため、サーバーに負荷がかかることから、awstats.plは内部からのみ実行できるようにする。また、万が一、AWStatsのデータベースが失われた場合、アクセス統計が見れなくなってしまうため、アクセス統計ページをHTMLで作成するようにする。

アクセス統計ページHTML版格納ディレクトリ設定

[root@localhost ~]# mkdir /var/www/html/usage.com/awstatsreport ← HTML版格納ディレクトリ作成
[root@localhost ~]# vi /etc/httpd/conf.d/virtualhost-usage.com.conf ← バーチャルホスト設定ファイル編集
<VirtualHost *:80>
ServerName usage.com
DocumentRoot /var/www/html/usage.com
ErrorLog logs/virtual-error_log
CustomLog logs/usage.com-access_log combined env=!no_log
Alias /awstatsreport /var/www/html/usage.com/awstatsreport ← 追加
</VirtualHost>
[root@localhost ~]# systemctl reload httpd ← Webサーバー設定反映

awstats.plアクセス制限

Apacheアクセスログ解析(AWStats)で既に設定済

アクセス統計ページHTML版作成

※アクセス統計ページHTML版作成スクリプトはApacheアクセスログ解析(AWStats)で既に作成済

[root@localhost ~]# ./awstatsreport.sh usage.com /var/www/html/usage.com/awstatsreport ← アクセス統計ページHTML版作成スクリプト実行

アクセス統計ページHTML版確認

http://usage.com/awstatsreport/ へアクセスして今月分のアクセス統計ページが表示されることを確認

過去のアクセス統計ページは http://usage.com/awstatsreport/年月(YYYYMM).html (例:http://usage.com/awstatsreport/200601.html)へアクセスする

アクセス統計ページ'HTML版作成定期自動実行設定

[root@localhost ~]# echo "00 00 * * * root /root/awstatsreport.sh usage.com /var/www/html/usage.com/awstatsreport > /dev/null" >> /etc/cron.d/awstatsreport ← cron登録
毎日00:00にアクセス統計ページHTML版作成スクリプト実行

トップ   新規 一覧 検索 最終更新   ヘルプ   最終更新のRSS