404 Errors

[jme574]

According to my awstats i have 132 404 errors for the month of oct. so far. When i clicked to see what the errors are i went and check the pages that are listed but most of them work - the biggest one was /robots.txt. (with a total of 63) Does anyone know why this might be happening?

[TCH-Don]

When search engines crawl your site,

they look for robots.txt to tell them what they can index and what not to index.

When they do not find it, you see the 404 (missing) error.

 

You can put a blank text filenamed robots.txt in your public_html folder to eliminate these errors.

Or you can create a robot file by using a Robots.txt file generator.

[jme574]

ok - i used the gen. you sent the link for and created a robot.txt file and saved it as such in microsoft word. Now if i understand right, i need to upload it via ftp to my root directory where my index file is located, right? I don't have a page named index, but i do have one named home, would that be the same? It is located in my public_html folder so is that where i should upload the robot.txt file to?

[TCH-Andy]

Yes, upload it to that directory. Make sure it's a text file though - not a word document (I'd use notepad rather than word)

[signatures]

Can this be done if i am using frontpage? can i just paste the robots.txt file on my index page, or somewhere else and then save it in my website from there?

hope this makes sense...thanks, dave

[TCH-Rob]

I wouldnt, FP and other editors put all the HTML code in there as well. This is nothing more than a simple text file. If you dont have an FTP program you can use the file manager in cPanel.

[signatures]

OK, so just use the ftp in CP and upload to my index.htm? haven't used the ftp yet, have just changed things from frontpage....guess i'll have to learn a few more things....dave

[TCH-Andy]

Hi Dave.

 

The robots.txt file is slightly different to all the stuff Frontpage does.

 

I would just loginto your cpanel, and go to the file manager.

Navigate to the public_html directory,

Click on "Create new file" and on the right you should see the space for a file name, and a selection box.

Type robots.txt in for the filename, and the file type should be text file

 

That's it. You do not need to edit it or anything, a blank file will be fine.

[signatures]

OK, i took care of that, maybe it will cut down on some of my 404 hits? anything else i could do? thanks for the help...dave

[signatures]

I still receive quite a few 404 hits, is there anything else to do to cut down or is it even necessary to? already have 120 first 5 days of the month. thanks, dave

[TCH-Bruce]

What is generating the 404 errors? Some 404 errors you cannot stop. Robots or other bots looking for scripts on your site that don't exist are going to give you a 404 error every time. You can't block them or stop them.

 

The only 404s you need to concern yourself with are for content (pages, images, scripts...) you actually created that may be named wrong in a link on one or several of your web pages.