[Robots.txt] What is the correct User agent for Adsense Crawl

by bluemonster 1 replies
Happen to face Google adsense crawl error, found the fix.

It says, to make adsense to crawl your pages and show targetted ads, add these lines on top of robots.txt.

Code:
User-agent: Mediapartners-Google
Disallow:
But my robots.txt file has already had this lines

Code:
User-agent: Mediapartners-Google*
Allow: /

User-agent: Googlebot-Image
Allow: /wp-content/uploads/

User-agent: Adsbot-Google
Allow: /
 
User-agent: Googlebot-Mobile
Allow: /
Someone explain me what these lines refer to and what is the difference between both.

Which is the right one? Do I need to make any changes or just leave like that?
#search engine optimization #adsense #agent #correct #crawl #robotstxt #user #wordpress
Avatar of Unregistered
  • Profile picture of the author bluemonster
    Originally Posted by mfred90 View Post

    First is correct. In first- if you write disallow that means google crawler do not read those file which file you put there.
    ok. Why is second one wrong? Can you explain me whats wrong in the second thing?

    I need someone to explain me about each line. Why is that allow/disallow used. Why asterisk at the end of Google*.

    thanks.
    {{ DiscussionBoard.errors[7881294].message }}

Trending Topics

  • 56 {{ upvoteCount | shortNum }}

    Better/cheaper solution than Mailchimp?

    Hi all, I have an e-commerce on Squarespace and I make my own product (jewelry). I want to start building an email list in order to promote 1 great offer ... [read more]

  • 59 {{ upvoteCount | shortNum }}

    Are you the Warrior of the Month?

    so.....we are launching the Warrior of the Month mini competition... Yes... every Month we will announce who was the Warrior with the most valuable post. This is our own small ... [read more]

  • 8 {{ upvoteCount | shortNum }}

    My product idea exists nowhere, am I fooling myself here?

    A business partner and I have invested in clickbank university. In the vendor program they have you explore the clickbank marketplace and look to see if your idea/passion/etc is listed. ... [read more]

  • 3 {{ upvoteCount | shortNum }}

    Are solo ads worth all the hype?

    K40water in Internet Marketing

    How many of you use solo ads for driving traffic? Do you have any other sources of traffic that you prefer? What is the most newbie friendly method of traffic ... [read more]

  • 21 {{ upvoteCount | shortNum }}