Jump to content

cmuskett

Members
 View Profile  See their activity

  • Posts

    1,324

  • Joined

  • Last visited

 Content Type 

Posts posted by cmuskett

    Psbot

    in Web Site Promotion

    Posted · Edited by cmuskett

    I just found psbot at my site psbot. I have to add something else to my robots.txt :lol:

     

    I went to the URL and this is what the page says.

     

     

    What is Psbot?

    Picsearch is indexing pictures from the web. To do this we use a web-crawler which identifies itself as 'Psbot'.

     

    Load spreading technique

    Picsearch utilizes a load spreading technique which reduces the load on the indexed domains' web-servers. The effect is that then the web-servers will have several minutes between requests from Picsearch. Although Picsearch is up and running, there is still development work in progress, and it is possible that a webserver will occationally have shorter intervals between requests.

     

    How to avoid being indexed

    If you do not wish Picsearch to index your pictures please add 'psbot' to your robots.txt file according to the The Robot Exclusion Standard (example provided below). If you do not have access to the robots.txt file Psbot also obeys the robots meta tag (example provided below).

     

    robots.txt example:

    Include the following in your robots.txt file:

     

    User-agent: psbot

    Disallow: /

     

     

    META tag example:

    Include the following in the head section of your web-page:

     

    <meta name="robots" content="noindex,nofollow">

     

     

    Experiencing problems?

    Since Psbot is still very new we apologize for any inconvenience it may cause and would ask you to notify Picsearch about any problems at psbot@picsearch.com in order for Picsearch to rectify them as quickly as possible. Thank you.

×
×
  • Create New...