s4suser Posted June 27, 2012 Share Posted June 27, 2012 Hello all, I have created robot.txt for my site.earlier it was working properly.Robots was followed the rules written on robot.txt. But since last few days suddenly i faced issue regarding web robots which visits pages that i have disallowed earlier in robots.txt and send me spam mail. Please help me, why this happens? Quote Link to comment Share on other sites More sharing options...
TCH-Thomas Posted June 27, 2012 Share Posted June 27, 2012 Welcome to the forum, s4user. :9 Robots.txt have nothing to do with spam mail as far as I know. This file only controls which search robots are allwed to visit your site. To control email blacklisting/whitelisting you will have to use/modify spamassassin. Quote Link to comment Share on other sites More sharing options...
s4suser Posted June 27, 2012 Author Share Posted June 27, 2012 Thanks TCH-Thomas for ur reply. No actualy i have one form for sending mail whose action is .php file I have restricted direct access to this php file using Disallow: /sendmail.php Still some robots are accessing these file and that result in i got empty mail. This robots are ignoring my robots.txt and access the page. This was working earlier. But nowadays i m facing this issue. Quote Link to comment Share on other sites More sharing options...
TCH-Bruce Posted June 27, 2012 Share Posted June 27, 2012 First thing I would do is rename your sendmail.php file to something else, preferably without the word "mail" in it. The second thing I would do is make sure your script is secure. That should stop the mails from trying to be sent. Quote Link to comment Share on other sites More sharing options...
OJB Posted June 27, 2012 Share Posted June 27, 2012 Are you sure they are search engine spiders/robots and not just spam bots? Spam bots won't take notice of your robots.txt file and if your form allows anyone to submit an entry without some sort of anti-spam mechanism (reCaptcha for instance) then spambots will happily submit entries to your form. I'd suggest the following: 1) front-end (Javascript) validation to ensure valid emails, valid names and non-empty fields are prevented 2) back-end (PHP) validation to ensure valid emails, valid names and non-empty fields are prevented 3) reCaptcha or some other variety of anti-spam mechanism (it can be as simple as a challenge and response asking a question that only a human can answer) Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.