Raw server logs

[Tipperton]

Besides my own site I help a friend manage and maintain his site here on TCH.

 

He sells a software protection and licensing system so we are constantly being watched by crackers/hackers and other people in the warez game.

 

As such it is sometimes important to be able to view the raw server logs on our site. There's the problem. According to TCH they process the logs daily, then delete them.

 

This is fine for most people but isn't at all good for us. Because unless you download the raw log imediately before they process and delete it you can never get the whole log for the day.

 

What would be far better is to copy the log to the user's account space before deleting it. That way the user can get the whole log. If they want to use the same file name for the copy process (over-writing the previous copy, that would be fine. I can handle a daily download.

 

This is definately needed. Webalyzer shows an access to our site by an unknown user. We would like to know at the least when and from where this access occured but with out the raw logs, we have nothing.

[Head Guru]

The logs are processed daily, they are not deleted daily. Your raw error log will show for 30 days and will roll over at the start of each calendar month.

 

With that said, we may have lost some stats when we completed the upgrade to Cpanel 6 - however the upgrade to six was a 100% must as a security flaw was found in cpanel5.

[erik98225]

I was fed up myself with the logs being deleted at 3:00 every day, so I made my own system that makes sure I don't miss anything.

 

Make all your pages .php instead of .html. Write the html code as usual, like a normal web page, but at the very top of the page, include this.

 

><?php

############# Change the following 2 lines as needed
$thispage = "yourpage.php";
$basedir = "/home/your_account/public_html/logs";

$ip = $_SERVER["REMOTE_ADDR"];
$referer = $_SERVER["HTTP_REFERER"];
$foo = $_SERVER["HTTP_X_FORWARDED_FOR"];
if ($foo) { $ip = $foo; }

############# Change the following line as needed
$bar = strpos("$referer", "yourdomainname.com");

if ($bar === false) { } else { $referer = ""; }

############# time()-3 will put the logs in west coast time.  Change as needed
$filename = date("m-d-y", time()-3*3600);
$localtime = date("g:i A", time()-3*3600);

$foobar = fopen("$basedir/$filename.txt", "a");
if ($foobar) { fwrite($foobar, "$ip\t$localtime\t$thispage\t$referer\n"); }
if ($foobar) { fclose($foobar); }

?>

 

Every time a page is viewed, it'll write logs into a regular directory on your web site. You can access the logs later anytime from an address like this:

 

www.yourdomainname.com/logs/03-04-03.txt

 

You need to create the directory first, so there is a place for the logs to get written into, and you should password-protect the directory so regular users can't see the logs.

 

Works like a charm.

[borfast]

Nice piece of code, erik. Thanks for sharing it

 

Just a little question: is the format you use compatible with Apache's format?

I'm asking this because if it is, it's great, since you can use the same log processing tools used to process Apache's logs

[KevinW]

If anyone is thinking about trying to implement Erik's code, understand:

 

1. The code goes at the start of each and every web page (or at least every page you wish to track!)

 

2. You will also need to rename each of those web pages, changing the suffix from .html/.html to .php

 

-kw

[Guest Guest_George]

The code seems very nice indeed!

Although this post is more or less "old" I have a question regarding its implementation: If my pages already are indexed, spidered, etc. in directories and engines... and I already have a more or less nice situation among them (if the searchers / surfers use some keywords that I had bet on them); what will hapen when I change the "family name" of my pages from htm / html to php? They will lost their actual position and I will need to start repositioning them from the beggining? It take me very hard work investigating keywords and costumes of the searchers for a long time (in different "lattitudes") until I get to focus on some keywords combinations that help positioning some of our pages at the top and bring us clients...

Any advise anyone?

 

 

Thank you all,

g.

[Lianna]

Oooh. Good question! What does happen?

[baseline]

I thought sites are always being re-ranked (well, as often as they have time for).. so if the filenames change but all the content and structure stays the same, wouldn't your new filenames end up ranking the same as the old ones (eventually)?

[adoute]

The logs are processed daily, they are not deleted daily. Your raw error log will show for 30 days and will roll over at the start of each calendar month.

 

With that said, we may have lost some stats when we completed the upgrade to Cpanel 6 - however the upgrade to six was a 100% must as a security flaw was found in cpanel5.

When are the access logs deleted? I would like to be able to schedule a cron job on a linux box at home to download my access logs once a month, ideally just before they are deleted. It is not critical that I get the log immediately before it is deleted, but as close to that time as possible would be nice. Considering that, is it possible to get the raw access log file via ftp? If so, what is the path to the logfiles & when are they deleted for the new month?

 

Thanks -

Allan Doute