3 {{ upvoteCount | shortNum }}
3 {{ upvoteCount | shortNum }}

How to Stop Search Engine Crawling Database?

by JayGreen
2 replies
I've developed ajax driven survey questionnaire for my users. Possible outcomes for users are in the hundreds. Each outcome is slightly different. If I do a spider crawl check on the page, I noticed spiders dig through the survey and see all the survey content . Since a ton of this information is the same it's going to look like spam.
How can I block bots from crawling the survey?
I've blocked via robots.txt but it still doesn't matter. I thought since this is my database the spider needs password access.
#crawling #database #engine #search #stop
  • Profile picture of the author Karen Blundell
    that doesn't sound right!

    You can try changing permissions on the directory that houses the survey results.

    what kind of database is it using?
    Signature
    ---------------
    {{ DiscussionBoard.errors[3998022].message }}
  • Profile picture of the author JayGreen
    This is was error on my part. It' fine. I was up for 24 hours working hard, like most Warriors, and miss-read some data. Sorry to waste your time. Thanks for reply.
    {{ DiscussionBoard.errors[3999175].message }}

Trending Topics