cmuskett Posted July 1, 2003 Posted July 1, 2003 I've noticed in my error logs that search engine spiders seem to be looking for pages and graphics that I removed long ago. Why do they keep coming back and looking for stuff that's long gone over and over again? Quote
SEO Posted July 1, 2003 Posted July 1, 2003 Multiple reasons... First, they are written to come back and respider. The duration depends on the page (several specific characteristics contribute to this). If the page shows multiple 404 errors, it will be flagged and removed from the index during the next index update. Until the next update, you could still see visits. In addition, it could occur if another page (maybe outside of your site) references the page or graphic. A spider will follow links from other pages, the so-called dead links. Actually a page can be 'penalized' if it contains too many dead links (makes sense to me, would not be a great reference). This help? Quote
boxturt Posted July 1, 2003 Posted July 1, 2003 Wondered about that myself. Thanks for the input. Quote
natimage Posted July 1, 2003 Posted July 1, 2003 I also had this happen. I caused my problem, though...I still had a link to one of my old pages on a working page and didn't know it. I also had problems at one point when Dreamweaver (or possibly me...) didn't completely delete all the code associated with some javascript that had been deleted! Worth a check to make sure you don't have dead links in your own page!! Tracy Quote
SEO Posted July 1, 2003 Posted July 1, 2003 Good point Tracy... If you can, do a 'html' search throughout your whole site for the suspected file names. A 'text' search can miss the vestigial JavaScript code that Tracy is referring to in the above reply. Quote
cmuskett Posted July 1, 2003 Author Posted July 1, 2003 When I clear old links off my site, I always make sure they are completely gone. I use FP 2002 for my html editor and I alway click the html tab to check to see that what I deleted is truely gone I think part of why I'm seeing this stuff, is there are people who have incorrect links to my site on theirs. When I have found a wrong link, I've asked them to change the link... but... they don't. They also link to specific pages on my site.. I use IFrames on my site and would rather they link to the main page only. Quote
SEO Posted July 1, 2003 Posted July 1, 2003 I should have added from the top that any time you remove a page from you site (and it is indexed) you should put a redirect in your .htaccess file. If you see in your logs that a particular link(s) causing a continuous problem, you could employ a redirect for that as well. Further reading on proper redirecting. Quote
cmuskett Posted July 1, 2003 Author Posted July 1, 2003 ahhh ok... will do that... there are about 10 in my error logs right now LOL Quote
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.