2 {{ upvoteCount | shortNum }}
2 {{ upvoteCount | shortNum }}

[Robots.txt] What is the correct User agent for Adsense Crawl

by bluemonster
1 replies
  • SEO
    • |
Happen to face Google adsense crawl error, found the fix.

It says, to make adsense to crawl your pages and show targetted ads, add these lines on top of robots.txt.

Code:
User-agent: Mediapartners-Google
Disallow:
But my robots.txt file has already had this lines

Code:
User-agent: Mediapartners-Google*
Allow: /

User-agent: Googlebot-Image
Allow: /wp-content/uploads/

User-agent: Adsbot-Google
Allow: /
 
User-agent: Googlebot-Mobile
Allow: /
Someone explain me what these lines refer to and what is the difference between both.

Which is the right one? Do I need to make any changes or just leave like that?
#adsense #agent #correct #crawl #robotstxt #user #wordpress
  • Profile picture of the author bluemonster
    Originally Posted by mfred90 View Post

    First is correct. In first- if you write disallow that means google crawler do not read those file which file you put there.
    ok. Why is second one wrong? Can you explain me whats wrong in the second thing?

    I need someone to explain me about each line. Why is that allow/disallow used. Why asterisk at the end of Google*.

    thanks.
    {{ DiscussionBoard.errors[7881294].message }}

Trending Topics