Google doesn't crawl my website, need advice

by G0nzalez Banned

Posted: 11 years ago 13 replies

Hi WF members,

I have this problem with Google.

Googlebot cannot access my website, thus my ads won't display properly. My AdSense account is activ since 2008. The ads on the website some time show up and some time they don't (3 of my friends don't see ads even if they view multiple pages of my site). I think it's a problem caused by robots.txt file, as I saw in Webmaster Tools, it says: Googlebot cannot access: http://mysite.com/

My robot.txt file is this:

Code:

User-agent: *
Disallow: /wp-admin/
Disallow: /wp-includes/

This morning I've changed it to the above robots.txt file. But until now I didn't get any traffic from Google and the site is 90 days old.

What could be the solution for this problem?

#advice #crawl #google #website

yestyle Banned 11 years ago

Because your site is too new or you didn't still make any good backlinks on other good sites.
- Thanks
{{ DiscussionBoard.errors[8596852].message }}

probytes

11 years ago

Hi,
Google crawler always searches for new content, so you should need to add new content in your site, create blog’s and also increase off page activities like directory submission, bookmarking.

[ 1 ] Thanks
1 reply

{{ DiscussionBoard.errors[8596886].message }}

sarahjohs

11 years ago

Originally Posted by probytes

Hi,
Google crawler always searches for new content, so you should need to add new content in your site, create blog's and also increase off page activities like directory submission, bookmarking.

Directory, bookmarking...ohhh it's all that a remnant technique once you've got the Penguin update plunged into action. Just leave away these including the article submissions too, unless you make it worth by marketing in reputed sites.

Thanks

{{ DiscussionBoard.errors[8596899].message }}

G0nzalez Banned 11 years ago

Any good solutions for this problem?

My site had adsense on it before and it was working just fine, a week ago I've got rid of them and now I've put them back on my site. I'm losing money because my ads won't show.

I would really appreciate some help.

I don't need backlinks to get traffic, I know how to drive traffic to my website.
- Thanks
{{ DiscussionBoard.errors[8598329].message }}
DizenSounds 11 years ago

Could be for many reasons.

Are you building backlinks?
- Thanks
- 1 reply
{{ DiscussionBoard.errors[8599169].message }}
- G0nzalez Banned 11 years ago
  
  Originally Posted by DizenSounds
  
  Could be for many reasons.
  
  Are you building backlinks?
  
  I have like 100 dofollow backlinks. But they came automatically, I didn't build them.
  
  Thanks
  
  1 reply
  
  {{ DiscussionBoard.errors[8599424].message }}
  
  HeadStartSEO 11 years ago
  
  Originally Posted by G0nzalez
  
  I have like 100 dofollow backlinks. But they came automatically, I didn't build them.
  
  This could be a few things, make sure the site is indexable. Also, see if it was active before. I once bought an old domain that was spammed to death, I just liked them name, but didn't look at the bank links.
  
  Action Plan
  1. Check if you can find it within Google.
  2. Upload your sitemap to Google Web Master Tools
  3. Check to see if it was advice before.
  
  Thanks
  
  {{ DiscussionBoard.errors[8599439].message }}
G0nzalez Banned 11 years ago

It's not indexed in Google, only the URL: http://mysite.com without the pages/posts, like: http://mysite.com/my-post

It says: A description for this result is not available because of this site's robots.txt – learn more.

The domain is new, didn't have an owner before.
- Thanks
- 1 reply
{{ DiscussionBoard.errors[8599451].message }}
- InterNauert 11 years ago
  
  Delete the robots.txt. That's what I would do. You don't need it to get indexed.
  
  [ 1 ] Thanks
  
  {{ DiscussionBoard.errors[8599530].message }}

Simon Walker

11 years ago

Originally Posted by G0nzalez

Code:

User-agent: *
Disallow: /wp-admin/
Disallow: /wp-includes/

This morning I've changed it to the above robots.txt file. But until now I didn't get any traffic from Google and the site is 90 days old.

What could be the solution for this problem?

Generate some backlinks, share it on social networks i.e make it popular and it will be crawled. The other way is to use webmaster tool and submit your website to Google index manually.

Thanks
1 reply

{{ DiscussionBoard.errors[8599602].message }}

wpninja2013 11 years ago

Well, calm down ! First of all, write some fresh content and general a xml sitemap. Submit you sitemap to Google through GWT, make sure the robots.txt are properly set.

Ping the post you just wrote and submit the rss feed to rss directories to get your site notified by Google bot. Backlinks are welcomed, but it is not a must-have.

The problem should be solved in less two days. I just did that for my new blog.

BTW, check the site loading speed, it also effects the indexing.
- Thanks
{{ DiscussionBoard.errors[8599688].message }}

Monja 11 years ago

I also believe that the key to get Google on your site is to add content - the more the better. If people ask how quick they can make money I can just say: how quick can you add content?
e.g. if you add 5 posts per day it is much better than 1 per week because each and every post offers Google a chance to send you visitors.
- Thanks
Signature

Free Business Class on Skillshare:
http://skl.sh/2bvH8nS
{{ DiscussionBoard.errors[8599697].message }}
G0nzalez Banned 11 years ago

Thanks for you're help WF members, the problem is solved now.
- Thanks
{{ DiscussionBoard.errors[8599710].message }}

Google doesn't crawl my website, need advice

Trending Topics

Been a while! Any of you Old Timers here?

How much does the rebranding affect CR ?

Are billboards still effective in driving customers?

Boat and Jet Ski Rentals...... Door Hangers Effective?

A Very Strange Internet Problem