6 {{ upvoteCount | shortNum }}
6 {{ upvoteCount | shortNum }}

Robots.txt File Destroys My Site Instantly

by Campbell24
5 replies
I need someone to help me understand this. I was looking for a way to host one of my info products on my site in a secretive way. So I used this method recommended on another thread:

Here's one solution for doing it with a download link.
create a folder in your ftp account.

Example, if your website is:
bathunter.com
then create the folder
bathunter.com/secretdownload/

then open a text editor, and save an empty file as "index.htm"

then you create a new file in the text editor and put this inside:
User-agent: *
Disallow: /secretdownload/
and you save that as robots.txt

then upload this into your main website folder.

then upload this into your secretdownload folder:
index.htm and your PDF.

If you do this, your PDF file won't get indexed by google, yahoo or bing.

This is exactly what I did. This created a page with my PDF on it, but my entire site with a first page ranking was immediately deindexed from Google.

I deleted the robots.txt file. Now what should I do?
#destroys #file #instantly #robotstxt #site
  • Profile picture of the author Campbell24
    My robots.txt file says:

    User-agent: *
    Disallow: /secretdownload/

    BUT YET MY ENTIRE SITE IS DEINDEXED. WTF?
    I installed the robots.txt file directly in my homepage.com folder.

    Please help!
    Signature
    FREE SEO CONSULTATION/ADVICE (from a 7-figure earner)

    I will answer your SEO questions 100% for free.

    Just ask me whatever you want!
    {{ DiscussionBoard.errors[6571838].message }}
    • Profile picture of the author Karen Blundell
      that makes no sense. There must be another reason your site got de-indexed because there is absolutely nothing wrong with your robots.txt file
      Signature
      ---------------
      {{ DiscussionBoard.errors[6572026].message }}
  • Profile picture of the author Raydal
    Maybe you need to take into consideration that
    simply because A follows B doesn't mean that
    A is the cause of B.

    That is a very common 'trick' marketers have
    been using for years now and I've never heard
    that their entire site got de-indexed.

    Maybe you need to look for another reason
    which with Google could sometimes be hard
    to find.

    -Ray Edwards
    Signature
    The most powerful and concentrated copywriting training online today bar none! Autoresponder Writing Email SECRETS
    {{ DiscussionBoard.errors[6572061].message }}
  • Profile picture of the author Yadira Barbosa
    There's something else that Google notice but your robots.txt it's correct.

    As a suggestion: Blackhatters always check the robots.txt to get your products, and you are telling them exactly where to search, I recommend you to use other domain, or Amazon S3, DLGuard or something else to protect your downloads.
    Signature
    FREE Amazing Sales Funnel Software

    List Building Made Easy
    {{ DiscussionBoard.errors[6572068].message }}
  • Profile picture of the author Campbell24
    How is Amazon S3 secure? From everything I've read it's not completely secure. But probably a lot more than this robots.txt way?

    How much does S3 cost generally?
    Per month?
    Per download?
    Signature
    FREE SEO CONSULTATION/ADVICE (from a 7-figure earner)

    I will answer your SEO questions 100% for free.

    Just ask me whatever you want!
    {{ DiscussionBoard.errors[6572749].message }}

Trending Topics