Website Data Skimmers?

[runninghorse]

So how can you tell if your website has been "skimmed" by one of these programs? And what can you do about it? What do these people who run these programs on a website want? Just a bit of confusion.

[jandafields]

Are you talking about a program that completely downloads a website for offline viewing like BlackWidow?

 

There is nothing you can do that I know of to stop it or even detect it, but the program can only download the server output, not your php pages themselves.

 

A lot of people use this type of program to batch download pictures and such.

[runninghorse]

I'm thinking more for like emails and links and such...

 

Edited to say -- For instance, what would show up in your stats if a program was running on your site gathering emails or links or something?

[jandafields]

If the "skimming" program is coded properly itself, it will look just like a regular website visitor. There would be no way to tell the difference.

 

The best thing to do is not put any email addresses on your website. Use forms instead.

[Deverill]

You could look at the stats for someone looking at *every* page of your site. Most folks don't do that and it's a sure sign that someone's harvesting something.

[annie]

Also, people loading pages but not the images on them...

[snipe]

They are generally harvesting email addresses - usually to either 1) compile and sell "targeted" email lists to spammers or 2) spam you themselves

 

One solution mentioned is to ONLY use forms as a source of contact - although I admit I am not fond of this method. I for one hate filling in "contact forms", for a few reasons:

 

1) If the form is coded poorly, I may never know if my email bounced for some reason and they never actually got my email. It tends to give the website user the feeling that their message may or may not be received, and may or may not be addressed. Setting up an autoreponder once they submit the form makes this a little better.

 

2) No spellchecker. This is a big one for me, since my mind works faster than my fingers, and I often fatfinger when typing. If I am trying to contact someone professionally, its a pain for me to type the message in something else and then copy+paste, just so I can have my message spellchecked.

 

3) I have run into enough forms that are poorly written, so that if I accidentally don't have a required field filled out, it yells as me when I submit the form, and when I use the back button, my entire message is *gone*, and I have to start from scratch. While I realize that this doesn't ALWAYS happen, its happened enough times to make me dread forms on websites.

 

The only other trick that I have found that works to fool some spiders is to use javascript to create your links. (Naturally, there is a disadvantage there if you are concerned about browsers visiting your site that do not have javascript enabled), but here's a sample script:

 

<script language=javascript>

var linktext = "Send email";

var email1 = "name";

var email2 = "domain.com";

document.write("<a href=" + "mail" + "to:" + email1 + "@" + email2 + ">" + linktext + "</a>")

//-->

</script>

 

This script builds the link using javascript - which makes it invisible to spiders, since spiders can't grok javascript code.

[TCH-Bruce]

2) No spellchecker.  This is a big one for me, since my mind works faster than my fingers, and I often fatfinger when typing.  If I am trying to contact someone professionally, its a pain for me to type the message in something else and then copy+paste, just so I can have my message spellchecked.

For spelling, there is ieSpell for Internet Explorer and SpellBound for Firefox. Both work great for spell checking forms.