29 {{ upvoteCount | shortNum }}
29 {{ upvoteCount | shortNum }}

block wp-content/uploads

by monere
24 replies
Hi,

I need to block wp-content/uploads from appearing in google. I set up a new website on WP and when I checked google with site:sitename.com to see if it got indexed I noticed that it did indeed get indexed, but google also indexed wp-content/uploads and when I click on it I can see the whole content of my uploads folder, and this scares me cause of the possibility of my website being hacked/exploited through this very folder.

Anyway, long story short does anybody know how to block this folder (and any other important files/folders) from appearing in the search engines?

Thank you in advance!
John

PS: don't worry, I will also thank you after you advise hehe :p
#block #wpcontent or uploads
  • Profile picture of the author bapparabi
    you have to just mention this rule in robots.txt file which will in return block google and other search engine to index in search engine

    you can get info on how to do it in this page

    The Web Robots Pages
    Signature
    Earn $ 1000+ a Month Easily & Automatically - 100% Automated WordPress Hotel, Flight, Cruises, Rental Cars & Amazon Store Search Engine Script
    {{ DiscussionBoard.errors[8407645].message }}
    • Profile picture of the author monere
      Originally Posted by bapparabi View Post

      you have to just mention this rule in robots.txt file which will in return block google and other search engine to index in search engine

      you can get info on how to do it in this page

      The Web Robots Pages
      Ok, I will use that robots file, but will this get my already indexed page out of the google index as well?

      Also, is there any command line/code/whatever that automatically blocks all files/folders containing sensitive info in my website? Please just don't refer me to htaccess as I don't know how to use that file and I already ruined a website a few years ago because of misusing it.

      Thanks for the above link. Will go ahead right away and see how I can use it.
      Signature

      Try not to become a man of success but rather to become a man of value - Albert Einstein

      {{ DiscussionBoard.errors[8407906].message }}
  • Profile picture of the author SteveJohnson
    Install the Better WP Security plugin by Bit51. It will shut off directory listings in addition to the rest of its security features. I never run a WP site without it any more.
    Signature

    The 2nd Amendment, 1789 - The Original Homeland Security.

    Gun control means never having to say, "I missed you."

    {{ DiscussionBoard.errors[8409325].message }}
    • Profile picture of the author monere
      Originally Posted by SteveJohnson View Post

      Install the Better WP Security plugin by Bit51. It will shut off directory listings in addition to the rest of its security features. I never run a WP site without it any more.
      Installed, activated, set up and already loving its simplicity

      One question though: do you know if the plugin also removes (or at least hides/blocks) wp-content/uploads from google index?
      Signature

      Try not to become a man of success but rather to become a man of value - Albert Einstein

      {{ DiscussionBoard.errors[8410244].message }}
      • Profile picture of the author RobinInTexas
        You need to prevent future crawling of the directory by using the robots.txt file.

        If there is something indexed you don't want you can go to https://www.google.com/webmasters/tools/removals
        Signature

        Robin



        ...Even if you're on the right track, you'll get run over if you just set there.
        {{ DiscussionBoard.errors[8410449].message }}
    • Profile picture of the author kpmedia
      Originally Posted by SteveJohnson View Post

      Install the Better WP Security plugin by Bit51. It will shut off directory listings in addition to the rest of its security features. I never run a WP site without it any more.
      The server needs indexing disabled.
      Plugins won't really do anything about that.
      Signature
      FAQ: Need a Site5/Arvixe/Hostgator alternative? And who is EIG?
      Reviews:       Need a good VPS, dedicated server, or unlimited host? (This is NOT a fake "top 10" list!)
      {{ DiscussionBoard.errors[8420946].message }}
      • Profile picture of the author RobinInTexas
        Originally Posted by kpmedia View Post

        The server needs indexing disabled.
        Plugins won't really do anything about that.
        BetterWPSecurity is a plugin and it will prompt you to allow it to disable indexing and it does it correctly.
        Signature

        Robin



        ...Even if you're on the right track, you'll get run over if you just set there.
        {{ DiscussionBoard.errors[8423322].message }}
      • Profile picture of the author Karen Blundell
        Originally Posted by kpmedia View Post

        The server needs indexing disabled.
        Plugins won't really do anything about that.
        that plugin actually turns off indexing -

        people- I have spent the better part of the last year working on security of 2 websites of mine that have been heavily under attack -

        If you are not certain about something - please don't post mis-information.

        Security measures done wrong can really mess up your website - making a small error on an .htaccess file , for example, can render your website inaccessible - I made that mistake and I HAVE experience with all kinds of code. If you're not experienced with stuff in your server, it's best to hire a professional to do it for you. Asking in a forum such as this will result in possibly getting the wrong info and mess you up worse than before you posted your question.

        So beware.

        all you need to do to stop any search engine from indexing anything in your wp-content/uploads directory is enter this in your robots.txt file:

        Code:
        User-agent: *
Disallow: /wp-content/
        that way you disallow the whole wp-content directory - highly-advisable!

        now most of the good crawlers will respect your robots.txt - you can actually create a trap for those that don't - but that is more complicated and deserves another discussion.

        Many times, the more plugins you have on your WordPress-driven site, the more chance you have of using up all your cpu and bandwidth allotment if your site is getting hammered. So learning how to do some things on your server without having to use any more plugins than is really needed is advisable if your are using standard shared hosting.

        anyway - use the robots.txt and if you have to deny some crawlers because they are disobeying your rules, just use IP deny in your cPanel to block them individually.

        I wish you every success!

        peace
        Signature
        ---------------
        {{ DiscussionBoard.errors[8423563].message }}
  • Profile picture of the author aronprins
    [DELETED]
    {{ DiscussionBoard.errors[8413403].message }}
    • Profile picture of the author monere
      Originally Posted by aronprins View Post

      Hey monere,

      Simplest and fastest way: create a .htaccess file within the wp-content content folder and add "deny all" to it, that should work fine!

      Hope this helps.
      Cheers,
      Aron Prins
      Deny all?? Won't this stop my entire site from getting indexed by big G? Cause that's not really the purpose you know!
      Signature

      Try not to become a man of success but rather to become a man of value - Albert Einstein

      {{ DiscussionBoard.errors[8413720].message }}
      • Profile picture of the author RobinInTexas
        Originally Posted by monere View Post

        Deny all?? Won't this stop my entire site from getting indexed by big G? Cause that's not really the purpose you know!
        Essentially that will block all traffic and search engines don't even think about doing something

        so stupid
        Signature

        Robin



        ...Even if you're on the right track, you'll get run over if you just set there.
        {{ DiscussionBoard.errors[8414248].message }}
        • Profile picture of the author monere
          Originally Posted by RobinInTexas View Post

          Essentially that will block all traffic and search engines don't even think about doing something so stupid
          Yeah, something about this did smell fishy to me, that's why I asked. Don't worry I won't do it cause - like I said - I try to stay away from htaccess as much as possible after having had bad experience with this tricky file.

          But what I don't understand is why would someone recommend this, at least without putting it as a joke or something :confused:
          Signature

          Try not to become a man of success but rather to become a man of value - Albert Einstein

          {{ DiscussionBoard.errors[8414319].message }}
          • Profile picture of the author RobinInTexas
            Originally Posted by monere View Post


            But what I don't understand is why would someone recommend this, at least without putting it as a joke or something :confused:
            See my post above.

            Also my earlier post explains how to do what you need.
            Signature

            Robin



            ...Even if you're on the right track, you'll get run over if you just set there.
            {{ DiscussionBoard.errors[8414331].message }}
            • Profile picture of the author monere
              Originally Posted by RobinInTexas View Post

              See my post above.

              Also my earlier post explains how to do what you need.
              Yes, I did create robots.txt and added that code to disallow wp-content/uploads but I am not in a rush to add my website to GWT like someone else suggested on this thread, as I am trying for the first time in my life to use Google as little as possible. That's why I didn't install SEO plugins either.

              Well, if traffic comes from Google I won't say NO to it, but I am trying to work without Google from now on. At least for a while. I am checking to see if article marketing (EZA, goarticles, articlesbase, isnare etc) + targeted forum marketing + social media is a viable solution to SEO in terms of SE traffic....

              But anyway, I digress .... the point is, I already did the robots.txt trick, and I also installed "Better WP Security" (another recommendation on this thread) and all that's left now is getting rid of/hiding wp-content/uploads from Google. Oh well, I'll postpone this as I am desperate to start the marketing and traffic generation. But thanks for your tips. I already implemented them
              Signature

              Try not to become a man of success but rather to become a man of value - Albert Einstein

              {{ DiscussionBoard.errors[8414376].message }}
        • Profile picture of the author ok123
          You should not advise people if you don't know about something.

          .heaccess in the uploads folder will be applicable for only that folder and not the whole site. Google how .htaccess works


          Originally Posted by RobinInTexas View Post

          Essentially that will block all traffic and search engines don't even think about doing something

          so stupid
          Signature
          Action expresses priorities - Mahatma Gandhi
          {{ DiscussionBoard.errors[8414521].message }}
          • Profile picture of the author RobinInTexas
            Originally Posted by ok123 View Post

            You should not advise people if you don't know about something.

            .heaccess in the uploads folder will be applicable for only that folder and not the whole site. Google how .htaccess works
            In this thread I did not advise anyone anything that I didn't know about.

            There is no .heaccess, .htaccess applies to the folder where it resides AND to the directory tree below it, not only that folder.

            I don't need advice to Google how to do anything.
            Signature

            Robin



            ...Even if you're on the right track, you'll get run over if you just set there.
            {{ DiscussionBoard.errors[8414607].message }}
            • Profile picture of the author ok123
              Instead of quoting my spelling errors, if you could just explain why using .htaccess will stop all search engines from crawling( as you have clearly said in your post above), it will be beneficial for the forum members.

              Originally Posted by RobinInTexas View Post

              In this thread I did not advise anyone anything that I didn't know about.

              There is no .heaccess, .htaccess applies to the folder where it resides AND to the directory tree below it, not only that folder.

              I don't need advice to Google how to do anything.
              Signature
              Action expresses priorities - Mahatma Gandhi
              {{ DiscussionBoard.errors[8414913].message }}
              • Profile picture of the author RobinInTexas
                Originally Posted by ok123 View Post

                Instead of quoting my spelling errors, if you could just explain why using .htaccess will stop all search engines from crawling( as you have clearly said in your post above), it will be beneficial for the forum members.
                What I said was that someone else's suggestion to use "deny all" in .htaccess in the wp-content folder will block all traffic and search engines and that the op shouldn't to that.

                The OP was satisfied with my correct advice when you reopened the thread quoting me and delivering a wrongheaded criticism.
                Signature

                Robin



                ...Even if you're on the right track, you'll get run over if you just set there.
                {{ DiscussionBoard.errors[8415041].message }}
          • Profile picture of the author aronprins
            Originally Posted by ok123 View Post

            You should not advise people if you don't know about something.

            .heaccess in the uploads folder will be applicable for only that folder and not the whole site. Google how .htaccess works
            That is correct, but hey i removed my post
            Ill think twice before posting advice next time!

            Cheers,
            Aron Prins
            Signature
            Aron & Sharon
            {{ DiscussionBoard.errors[8417683].message }}
            • Profile picture of the author monere
              Originally Posted by aronprins View Post

              That is correct, but hey i removed my post
              Ill think twice before posting advice next time!

              Cheers,
              Aron Prins
              LOL. You scared me with that nasty advice, but it's all cool now
              Signature

              Try not to become a man of success but rather to become a man of value - Albert Einstein

              {{ DiscussionBoard.errors[8417843].message }}
  • Profile picture of the author munstersg01
    You can protect the folder or directory via through your Cpanel.

    There should be an section for you to password protect the folder.

    Look for it usually under security

    Alternatively you can also protect the folder via .htaccess and for the website not to be index you can do so via robots.txt

    Using Cpanel Index Manager you can also hide that particular file or folder.

    Search for it Index Manager is under the Advance section of the Cpanel.

    To prevent prying eyes upload a index.php or index.html to the folders within wordpress that does not have a index.php or index.html. Usually happens in the upload folders. Do note each month and year has their own folder.

    Danny
    Signature
    EZ Ecovers New WSO Live Now, Create Ecovers using GIMP or Sumopaint with Ease

    Unlimited Online Backup for less than $0.05 per day
    {{ DiscussionBoard.errors[8418124].message }}
    • Profile picture of the author monere
      Thank you. I found the index manager under Advanced section in my cpanel and set the entire 'wp-content" folder to not be indexed. Is it ok or should I do this just for the uploads folder? I don't know...

      Also, do you know if this will also take the folder out of google's index or do I still have to do it via GWT's removals?

      And also, there are 3 more folders appearing along with wp-content (cgi, wp-admin, and wp-includes). Do I have to protect these ones as well, or should I just let them be?

      Again, thanks for the awesome advice. I had no idea what that index manager was for. Actually, I have no idea what most of the options in my cpanel are for :p

      Originally Posted by munstersg01 View Post

      You can protect the folder or directory via through your Cpanel.

      There should be an section for you to password protect the folder.

      Look for it usually under security

      Alternatively you can also protect the folder via .htaccess and for the website not to be index you can do so via robots.txt

      Using Cpanel Index Manager you can also hide that particular file or folder.

      Search for it Index Manager is under the Advance section of the Cpanel.

      To prevent prying eyes upload a index.php or index.html to the folders within wordpress that does not have a index.php or index.html. Usually happens in the upload folders. Do note each month and year has their own folder.

      Danny
      Signature

      Try not to become a man of success but rather to become a man of value - Albert Einstein

      {{ DiscussionBoard.errors[8418510].message }}
  • Profile picture of the author YourOnlyWriter
    Banned
    Early last year, I just did this to my business' website to see how it works.
    If your domain is hosted by HostGator, log into to your host's CPanel and check out that root directory option. You may also password-protect directories if you wish.

    I just can't find that exact "kb article" from Hostgator's site. Anyway, that's already a year ago.
    {{ DiscussionBoard.errors[8418399].message }}
  • Profile picture of the author monere
    Thanks Karen. I know that htaccess is a nasty file, which is why I chose not to mess with it. That's one thing.

    Second thing, I know that the fewer plugins installed the better security, and speed-wise website I will have, so I keep these to the bare minimum as well.

    And third thing, I am only interested in hiding that directory from the search engines for now, which I already did (using the robots.txt). So, I don't care about creating those traps that you mentioned, at least not yet. Maybe later, when/if someone gets bored with their life and they will decide to destroy mine (talking about hackers, obviously). But for now I am good

    But, thanks for the nice explanations. I will really start at least to learn what those icons in my cpanel do, if not learn how to use them as well. I was always afraid of touching those icons (except for fantastico, stats, and email), but they seem to be all so useful especially for such cases like mine
    Signature

    Try not to become a man of success but rather to become a man of value - Albert Einstein

    {{ DiscussionBoard.errors[8423767].message }}
    • Profile picture of the author Karen Blundell
      Hi Monere,

      In cPanel, the icons you should become familiar with are in the email section "email authentication" and set up DKIM and an SPF record to protect yourself from email spoofers.

      Then you should set up Hot Link Protection, Under the "security" tab - That way people won't hotlink to any files you have on your server and use up your bandwidth

      And as mentioned earlier use IP Deny to block users by domain or by IP address or IP range

      If you ever need some help with .htaccess especially with regards to denying access to sensitive WordPress files, or blocking a rogue user agent or referer please let me know.
      The nice thing about making mistakes is that you eventually learn how to do things right -

      Unfortunately, WordPress sites continue to have a big target on their backs - WordPress users can never afford to leave their sites unattended for very long.
      Signature
      ---------------
      {{ DiscussionBoard.errors[8427275].message }}

Trending Topics