3 {{ upvoteCount | shortNum }}
3 {{ upvoteCount | shortNum }}

How to Get Disallow Page Indexed in Google?

by sggl
3 replies
  • SEO
    • |
Hello guys,
I created a diary on caloriecount.com. The diary has the following url
structure /users/myusername/blabla.html.

I wanted to post interesting content about weight loss there.

However, I found that /users directory is set as disallow in robots.txt

Is there any way to get my diary indexed? i've seen examples of disallow
pages that were succefully indexed by google in reddit (yes, certain categories are disallow there).

Thanks
#disallow #google #indexed #page
  • Profile picture of the author paulgl
    You cannot edit the robots.txt so you can't do much.

    You are probably mistaken about some pages appearing.

    And that leads to what you can do.

    You can create a link to the page somewhere that google will crawl.

    They sometimes have the url to the page in their search results.
    Not the content, but the url. That's why you are probably mistaken
    about pages appearing. Urls appear, not what's on them.
    A minor technicality. A searcher can still click on the link and get there.

    But that of course would mean a whole host of problems to anyone actually finding the page.

    Since the content is not indexed, google is (mostly) never going to include it in any real search.

    The last thing is that robots.txt is not fool proof. Crawlers are not perfect. Nothing online is perfect.

    The content may indeed get indexed, although rare and a moot point for you.

    Use a platform that you can control. Caloriecount is obviously made for people who are not spamming..er...sharing.

    Paul
    Signature

    If you were disappointed in your results today, lower your standards tomorrow.

    {{ DiscussionBoard.errors[10712313].message }}
  • Profile picture of the author DABK
    I find your actions confusing. What's the goal? (And the goal of the goal?)

    Originally Posted by sggl View Post

    Hello guys,
    I created a diary on caloriecount.com. The diary has the following url
    structure /users/myusername/blabla.html.

    I wanted to post interesting content about weight loss there.

    However, I found that /users directory is set as disallow in robots.txt

    Is there any way to get my diary indexed? i've seen examples of disallow
    pages that were succefully indexed by google in reddit (yes, certain categories are disallow there).

    Thanks
    {{ DiscussionBoard.errors[10712701].message }}
  • Profile picture of the author TomWest
    I want to recommend you a great article How to Tell Google Not to Index a Page in Search Results [Quick Tip]
    where you will find the information about how how to disallow Google to index your pages. In this article you will find two ways how to implement it via a robot.txt file and via "noindex" meta tag.
    {{ DiscussionBoard.errors[10713425].message }}

Trending Topics