I have a site that is exploding pretty fast, with traffic doubling from google search every few weeks. The problem is that the way I crawl the web for data to serve up pages. It seems that once I reach about 500,000 page views a day (with bots included), my site begins to crash as I can't crawl past that amount.
Google is now sending so much traffic to the site. It's growing US traffic as well, but a lot of it is coming from countries like India and Bangladesh. And that would normally be great, but since I'm limited on the number of page views I can serve up per day because of my crawler situation, I need to try to maximize my page RPM so that I can make a good living off of this site. Which means that I need to up my US/CA/UK/AU traffic and potentially reduce the traffic from other countries, unfortunately.
The traffic coming from the other countries have browsers that are generally set to us-en, so focusing on English only content doesn't seem like it can greatly improve the performance of this traffic geo ratio.
Do you have any ideas for how I can get Google to focus on sending me more traffic from US and similar countries overall, so that I can up my earnings while I'm still restricted by this maximum page view situation?