A question about crawler

by seoboy
4 replies
There are few forums that require login to show links in the threads and posts. Plain text is visible without signing in. My question is, does crawler read these links or it only crawls plain text that is available without login.
#crawler #question
  • Profile picture of the author Matt Morgan
    Originally Posted by seoboy View Post

    There are few forums that require login to show links in the threads and posts. Plain text is visible without signing in. My question is, does crawler read these links or it only crawls plain text that is available without login.
    I think the viewable posts are only for view after log in but the SE's can view all the posts/ logged in or not, aslong as they forum owner hasn't put the robots settings to 'not index' certain forum categories.

    Perhaps some others can confirm this.
    Signature
    -> [FREE WSO 1] Discover 77 FREE Ways To Generate Traffic!
    -> [FREE WSO 2] Discover 67 Killer Traffic Methods Which Will Crush Your Competition!
    -> [FREE WSO 3] Discover 77 Amazing Blogging Tips To Explode Your Profits!

    ...…..Now LISTEN CAREFULLY! ===> [WSO REPORTS 4, 5, 6 are >> Found Here]
    {{ DiscussionBoard.errors[2412065].message }}
    • Profile picture of the author seoboy
      yeah m having the same point of view... thanks for the reply dude
      {{ DiscussionBoard.errors[2412740].message }}
  • Profile picture of the author addictiod
    How Morgan says, he's right.

    The SE's can view the posts logged in or not, as long as the forum owner hasn't put the robots settings to not index.
    {{ DiscussionBoard.errors[2413146].message }}
    • Profile picture of the author seoboy
      thanks addictiod for the confirmation
      {{ DiscussionBoard.errors[2415036].message }}

Trending Topics