Verify Google Web Cache in PHP

by 6 replies
9
Anyone have any ideas on how I would go about retrieving the Google Cache of a web page, in PHP?

I don't need the actual content. I am simply looking to verify if a particular page has been cached by Google.

A simple 1 or 0 is my desired output.




p.s. The last time I tried to Curl a Google web page a few years ago, I was blocked in my attempt. I assume this remains the case.

p.p.s. I do have a Google API Key, but cannot see any reference to cache in their documentation.
#programming #cache #google #php #web
  • Did you set a user agent string other than "Curl" when you downloaded from Google with Curl? If not, give it a shot. I'm pretty sure Google is going to flat out deny anything with a UA string like "Curl" or "Wget".

    I was using a google scraping script a couple of months ago just fine as long as I set a normal looking user agent. I also ran it through a proxy connected to TOR.
  • I have sent the precise user agent string that appears in my site logs for my computer.

    I have not run it through the proxy yet, because I have never looked at that before.


    p.s. You did get me thinking though... I am looking at Google Scraper codes now.


    I have been going through scrapers, and I have the code, but all require proxy urls and ports... I am at a loss to find valide proxies that I can use in my coding.
    • [DELETED]
      • [1] reply
  • Thank you for your help.

    I never was able to pull the cache page or get the cache date, but using the technique you described, I was able to verify that google had the URL, done with the scraping tool using Snoopy.

    Thank you again.
    • [1] reply
    • Hi Guys ... If I could jump in here....

      I am looking to hit the webcache directly as well and have used a few different strategies, many of them you have mentioned.

      Am I reading correctly that the only way to successfully do this is via proxy masking as the IP address will be blocked again and again?

      Is there a curl alternative that might be successful?

      Reply or PM

      Thank you
      -drMic
  • Banned
    [DELETED]
Join the discussion

Next Topics on Trending Feed

  • 9

    Verify Google Web Cache in PHP

    Anyone have any ideas on how I would go about retrieving the Google Cache of a web page, in PHP? I don't need the actual content. I am simply looking to verify if a particular page has been cached by Google.