Jump to content

What To Do With Robot Tags?


Recommended Posts

A Robot tag should only be used if you would like to prevent a spider from acting.

 

"noindex" tells the spider to continue onto linked pages but do not index the current page.

 

"nofollow" tells the spider to index the current page but do not follow the links on the page.

 

Thus...

 

 

><meta name="robot" content="noindex, nofollow">

 

 

would tell the spider not to index this page and not to follow any links.

 

One more option (there are others but really these three are the only ones to use) would be:

 

><meta name="robots" content="noarchive">

 

which tells the spider not to cache the page.

 

(Thanks to TCH-Scott for the answer to this question.)

Link to comment
Share on other sites

Guest
This topic is now closed to further replies.
×
×
  • Create New...