Robot Meta Tag

[SEO]

A family member writes:

 

I'm a bit confused on what I should be doing with the "robot" meta tags from page to page. I'm set up with xxx xxxx, and on one of their advanced meta tag articles, they stated that a "robot" tag should have "content"="index" if you wanted to allow access for the robots to scan all of the contents of the page. I thought that it should be "content"="All" in order for the robots to look at everything on the page's html and links etc..

 

Am I even barking up the right tree by being concerned with the "robot" tags?

[SEO]

A Robot tag should only be used if you would like to prevent a spider from acting.

 

noindex tells the spider to continue onto linked pages but do not index the current page.

 

nofollow tells the spider to index the current page but do not follow the links on the page.

 

Thus <meta name="robot" content="noindex, nofollow"> would tell the spider not to index this page and not to follow any links.

 

One more option (there are others but really these three are the only ones to use) would be:

 

<meta name="robots" content="noarchive">

 

which tells the spider not to cache the page.