Jump to content

Robot Meta Tag

Recommended Posts

A family member writes:


I'm a bit confused on what I should be doing with the "robot" meta tags from page to page. I'm set up with xxx xxxx, and on one of their advanced meta tag articles, they stated that a "robot" tag should have "content"="index" if you wanted to allow access for the robots to scan all of the contents of the page. I thought that it should be "content"="All" in order for the robots to look at everything on the page's html and links etc..


Am I even barking up the right tree by being concerned with the "robot" tags?

Link to post
Share on other sites

A Robot tag should only be used if you would like to prevent a spider from acting.


noindex tells the spider to continue onto linked pages but do not index the current page.


nofollow tells the spider to index the current page but do not follow the links on the page.


Thus <meta name="robot" content="noindex, nofollow"> would tell the spider not to index this page and not to follow any links.


One more option (there are others but really these three are the only ones to use) would be:


<meta name="robots" content="noarchive">


which tells the spider not to cache the page.

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

  • Create New...