Live SEO case study - In 2 months, 56,900 pages indexed and reached $100+/month milestone

[dtang4]

EDIT -- Updated stats as of 6/17

So, I've been doing some experimenting in getting a new domain indexed and tracking everything on a daily basis.

For background, the site is a niche jobs search (focused on clerical jobs) based off an API. The API takes care of the majority of my initial content and ongoing fresh content creation. The rest of the content I pulled and modified from Wikipedia, e.g. facts about bookkeeping jobs, and is relatively static.

Upon launch, I estimated the site had ~300K pages. My goal was for the site was to operate on autopilot, relying solely on long tail search engine traffic. To do this, I wanted to get as many pages indexed as possible.

Here are the daily indexing results you can benchmark against.

• Day 1 - 2 pages indexed
• Day 2 - 31 pages indexed
• Day 3 - 92 pages indexed
• Day 4 - 129 pages indexed
• Day 5 - 361 pages indexed
• Day 6 - 3,340 pages indexed
• Day 7 - some fluctuation between 3,340 and ~3,800 pages indexed
• Day 8 - 3,960 pages indexed w/ some fluctuations during the day
• Day 9 - 6,510 pages indexed
• Day 10 - 8,080 pages indexed
• Day 11 - 10,100 pages indexed
• Day 12 - 11,000 pages indexed
• Day 13 - Forgot to check stats. I noticed that 80% of the traffic was from long tail search traffic, where my site is ranked on page 1 for all the the referring search terms. Here is a screenshot from Google Analytics for the week of 4/17 (http://themanwhosoldtheweb.com/blog/...analytics1.gif).
• Day 14 - 17,000 pages indexed
• Day 15 - Fluctuations between 16,000 - 17,000 pages indexed
• Day 16 - 19,900 pages indexed
• Day 17 - 19,800 pages indexed
• Day 18 - 21,300 pages indexed
• Day 19 - 21,300 pages indexed
• Day 20 - 22,100 pages indexed
• Day 21 - 22,700 pages indexed; for week of 2/24, received 150 visitors and 233 page views

By day 21, I did a fairly detailed analysis of all my daily stats and compiled the results into a guide called 21K in 21 Days. Here's a direct download to the PDF file (so no opt-in):
http://themanwhosoldtheweb.com/downl...in_21_Days.pdf

• Day 22 - 23,000 pages indexed
• Day 23 - 23,900 pages indexed
• Day 24 - 24,600 pages indexed; slapped on some Google Adsense ads, so will begin to monetize this site now
• Day 25 - 24,600 pages indexed
• Day 26 - 22,400 pages indexed
• Day 27 - 22,500 pages indexed
• Day 28 - 22,400 pages indexed; for week of 5/1, received 138 visitors and 394 page views; made $7.37 across Indeed and Adsense (almost recouped my initial $7.50 GoDaddy domain investment after only 1 week of monetization!)

Going forward, I started doing weekly updates tracking indexing results, web traffic, and earnings.

• Days 28-34
  • 21,900 pages indexed
  • 150 visits, 391 pageviews (78% US), 84 keyword searches
  • $8.40 earnings
• Days 35-41
  • 22,800 pages indexed
  • 194 visits, 480 pageviews (72% US), 79 keyword searches
  • $3.23 earnings
• Days 42-48
  • 47,500 pages indexed
  • 227 visits, 629 pageviews (63% US), 111 keyword searches
  • $11.79 earnings
• Days 49-55
  • 56,900 pages indexed
  • 394 visits, 1,045 pageviews (70% US), 240 keyword searches
  • $124.08 earnings (mostly from direct ad sales)

Feel free to reply with any questions. Don't PM.

In case you're wondering, this is the same case study as referenced in my sig.

Thanks.

Dave

[InternetMarketingIQ]

300 THOUSAND pages. And it all came online at once and Google did what now?

You can always watch the Bot in your log files to see how deep it is crawling.

The more interesting thing is how anyone can create 300,000 pages of content and throw it all online at once.

[dtang4]

Originally Posted by InternetMarketingIQ

300 THOUSAND pages. And it all came online at once and Google did what now?

You can always watch the Bot in your log files to see how deep it is crawling.

The more interesting thing is how anyone can create 300,000 pages of content and throw it all online at once.

Hmm, that's an interesting suggestion. I am using Google Analytics to track all my stats... didn't think of using log files. Is there a package I can use to easily peruse and visualize log files?

About your last comment, even though it's ~300K pages of unique content, the site is only comprised off a small handful of scripts (<10).

[guitarjosh]

Been there... done this. I disagree with getting 1000 pages published by putting out 300,000 pages of content. I was getting such random results on my different sites that I launched 100,000, 200,000, 300,000 and 400,000 pages at the same time. My 100,000 page site got the most pages indexed within a month at 8000. My 300,000 page site was the worst at almost 2000. 4 months in, it went down to 280 and stayed there for about a month. It's 6 months old now and has 8,000.

Launching larger sites always results in far less pages being indexed for me. Want to see 1000 get indexed even quicker? Launch a 700 page site. It will index at about 1000 almost every time.

Most of the sites I launch are 15,000 - 20,000 pages on day 1 and about 90% of them get 15% - 18% indexed in the first 2 weeks.. and don't budge from that number for months.. and then go up a few thousand.

I have exception sites too. My best performer has just over 22,000 pages total (15,000 on launch) and it now has over 80,000 pages indexed.

This is the beauty of auto-blogging. People can hate it all they want but you get tons of data to analyze like few others can and it provides a ton of insight that others simply don't get. Even if you don't throw adsense on these, you learn a ton.

Now, Dave.. if we could figure out the algorithm enough to understand why it chooses the number it does, we'd soon be wealthy, yes? I'm still working on that myself. I'm glad to see others are too!

[dtang4]

Originally Posted by guitarjosh

Been there... done this. I disagree with getting 1000 pages published by putting out 300,000 pages of content. I was getting such random results on my different sites that I launched 100,000, 200,000, 300,000 and 400,000 pages at the same time. My 100,000 page site got the most pages indexed within a month at 8000. My 300,000 page site was the worst at almost 2000. 4 months in, it went down to 280 and stayed there for about a month. It's 6 months old now and has 8,000.

Launching larger sites always results in far less pages being indexed for me. Want to see 1000 get indexed even quicker? Launch a 700 page site. It will index at about 1000 almost every time.

Most of the sites I launch are 15,000 - 20,000 pages on day 1 and about 90% of them get 15% - 18% indexed in the first 2 weeks.. and don't budge from that number for months.. and then go up a few thousand.

I have exception sites too. My best performer has just over 22,000 pages total (15,000 on launch) and it now has over 80,000 pages indexed.

This is the beauty of auto-blogging. People can hate it all they want but you get tons of data to analyze like few others can and it provides a ton of insight that others simply don't get. Even if you don't throw adsense on these, you learn a ton.

Now, Dave.. if we could figure out the algorithm enough to understand why it chooses the number it does, we'd soon be wealthy, yes? I'm still working on that myself. I'm glad to see others are too!

Thanks for the points of comparison. Very interesting!

Were your sites all architected similarly?

I guess I should clarify -- my 1%/1 week theory is based on a certain type of site architecture, where the majority of the pages follow a generic listings format.

I've had sites that have indexed as much as 500K pages (a blogging and social networking community) and 250K pages (a guitar tabs search engine). But, for both these cases, content was added in a more organic, drip way.

Care to share any of your site URLs? I'd be very interested in checking them out.

Thanks.

Dave

[dtang4]

Update -- it's now 1 week later and the site has 17K pages indexed and still growing daily. This is doing better than I originally thought.

I added the daily update to the original post.

[WealthWithin]

Great case study.

Do you mind sharing a bit about the site architecture? How the pages, keywords are generated, and how do you know that there are 300k pages?

Thanks.

[cheesee]

That is some great results, but your website started off withs LOTS of content. So its a little less impressive.

[dtang4]

Originally Posted by WealthWithin

Great case study.

Do you mind sharing a bit about the site architecture? How the pages, keywords are generated, and how do you know that there are 300k pages?

Thanks.

Sure thing.

Even though there are ~300K pages of unique content, there are only a small handful of actual scripts. There are 2 in particular that make up 99% of the pages.

• A listings page (e.g. search results page). There is a listings page per zip code, and per job category of bookkeeping, accounting, and auditing. Since there are ~40K zip codes in the US, this comes out to 40K * 3 = 120K listings pages.
• A job description page. The job description page is essentially a frame that displays the external Indeed.com job description page. On each listings page, I have 15 job listings. Each job listing points to a job description page. That brings me to 120K * 15 = 1.8MM jobs. However, I assume there is a lot of cross listing across zip codes and job categories and conservatively only 10% unique per listings page. Therefore, I have 1.8MM * 10% = 180K job description pages.

This then brings me to the estimated 300K pages. The number of pages will continue to grow organically as more clerical jobs are being added to Indeed.

All the pages have SEO-friendly URLs (courtesy of .htaccess/mod_rewrite).

Let me know if you have any other questions.

(I do go in a bit more detail in the case study article itself on my blog and screenshot the actual site architecture.)

[dtang4]

Originally Posted by cheesee

That is some great results, but your website started off withs LOTS of content. So its a little less impressive.

Well, the site has a lot of content, but it only has 7 actual files. Here is the screenshot of the site's physical file structure:
http://themanwhosoldtheweb.com/blog/...casestudy1.gif

All the dynamic content was pulled from an API. The remaining static content I have was pulled and modified from Wikipedia.

[dtang4]

Sorry for the lag guys -- just posted another big update to the original post.

As of last week, I started to monetize the site. So, going forward, I will be updating w/ 3 types of stats:
* pages indexed
* traffic stats
* $ revenue

[misterkailo]

That's a lot of pages indexed

[realworldincome]

I have just implemented Dave's method along with his excellent feedback and support. Will be tracking results over the next few weeks. Looks very promising.

Charles

[dtang4]

Originally Posted by realworldincome

I have just implemented Dave's method along with his excellent feedback and support. Will be tracking results over the next few weeks. Looks very promising.

Charles

Thanks for the testimonial, Charles!

[dtang4]

Just posted a huge update to the initial thread with updated stats.

[dtang4]

For those interested, this case study was based off a turnkey script I created, called 300K Page Job Search In-a-Box.

I have just release it as a discounted WSO (as of this morning) Here's the link:
http://www.warriorforum.com/warrior-...-megasite.html

[marco005]

Hy,

Please let me understand, a website with indexed 56.900 pages in google does only earn 124$ in month??? In my Eyes thats worst case or not?

Lets see the John xfactor authorithy health site with ~ 2000 ore more indexed pages, John makes more, many more than 124$ in month with adsense.

This is an inexplicable in my eyes.
A german website heimundgarten.net has over 30.000 indexed pages in google, and makes 300.000 $ or Euro in a year!

with best wishes
marco005

[dtang4]

Originally Posted by marco005

Hy,

Please let me understand, a website with indexed 56.900 pages in google does only earn 124$ in month??? In my Eyes thats worst case or not?

Lets see the John xfactor authorithy health site with ~ 2000 ore more indexed pages, John makes more, many more than 124$ in month with adsense.

This is an inexplicable in my eyes.
A german website heimundgarten.net has over 30.000 indexed pages in google, and makes 300.000 $ or Euro in a year!

with best wishes
marco005

No, I wouldn't call that the worst case.

There's not always a direct relationship between earnings and pages indexed. If you look only at the variable of pages indexed, you are most likely not comparing apples to apples. Other factors to take into account include traffic, visitor geography breakdown, monetization strategy (e.g. ads vs. ecommerce), site age, etc.

The fact the site is barely 2 months old and making $100+/mo is a decent situation, in my opinion.

[marco005]

Hy dtang4,

I still was in opinion, that a huge website earn more with adsense than a little number page website, all IM Gurus said, google adsense is a number game.

Look at this;https://www.google.com/adsense/static/de/Weblogs.html

Reading, at the beginning, they maked 1000 blogposts/articles every week! Google had not deindexed them, with this beginning they maked a few hundred dollars a day.

Later after six months more than 1000$ a day.

So you can see that this is a number game.
The more blogposts you have, the more articles/webpages are indexed in google, the more you earn.

Not forget, that you picked a niche who has a minimum cpc over 1-2$ per click for advertiser.
So that the number of indexed pages must been an indicator for adsense earnings.

Make a calculate;

You have time to write 500 articles every month, every article you can brng in in google news and you make article marketing for them, lets say every article brings you 30 visitors to your website/blog in month, thats are 15.000 visitors every month.
So when 10% of them click on your ads, thats are 1500, when you get 0,50$ from google, you made 750§ from this website alone.

When you write 1000 articles in month who brings you 30 visitors from every article to your website, you doubled your adsense income.

OK when you will make money as an affilia, for the huge affiliate income the number of articles/webpages is not an indicator, but too, then, more you writing, more you bring n google news or make article directorie marketing, than more visitors/traffic you become.
Later more traffic from the search engines.

With no webpages/ no articles you not become traffic, no traffic no sales, no traffic no adsense earnings.
Closing the circle. Correct me if I wrong there.

with best wishes
marco005

[WealthWithin]

Since most of the pages sort of share the similar content, was there a drop in traffic after recent Panda update?

[dtang4]

Originally Posted by marco005

Hy dtang4,

I still was in opinion, that a huge website earn more with adsense than a little number page website, all IM Gurus said, google adsense is a number game.

Look at this;https://www.google.com/adsense/static/de/Weblogs.html

Reading, at the beginning, they maked 1000 blogposts/articles every week! Google had not deindexed them, with this beginning they maked a few hundred dollars a day.

Later after six months more than 1000$ a day.

So you can see that this is a number game.
The more blogposts you have, the more articles/webpages are indexed in google, the more you earn.

Not forget, that you picked a niche who has a minimum cpc over 1-2$ per click for advertiser.
So that the number of indexed pages must been an indicator for adsense earnings.

Make a calculate;

You have time to write 500 articles every month, every article you can brng in in google news and you make article marketing for them, lets say every article brings you 30 visitors to your website/blog in month, thats are 15.000 visitors every month.
So when 10% of them click on your ads, thats are 1500, when you get 0,50$ from google, you made 750§ from this website alone.

When you write 1000 articles in month who brings you 30 visitors from every article to your website, you doubled your adsense income.

OK when you will make money as an affilia, for the huge affiliate income the number of articles/webpages is not an indicator, but too, then, more you writing, more you bring n google news or make article directorie marketing, than more visitors/traffic you become.
Later more traffic from the search engines.

With no webpages/ no articles you not become traffic, no traffic no sales, no traffic no adsense earnings.
Closing the circle. Correct me if I wrong there.

with best wishes
marco005

Yea, I think we're comparing different points.

On a high level, the more traffic a site earns, the more adsense $. I agree there. However, comparing 2 different sites, the site w/ more traffic doesn't mean it has more indexed pages; and vice versa.

[dtang4]

Originally Posted by WealthWithin

Since most of the pages sort of share the similar content, was there a drop in traffic after recent Panda update?

Nope, there's been no noticeable drop in traffic or pages indexed that coincided w/ the panda update.

[kdavis26]

Great work. I also had a job placement site for a while and I never saw any problems with any duplicate content. The information is close but not the exact same so you never have any problems. If you did, sites like monster and such would never rank well. As far as your results. Great work Keep us updated on how your site is doing.

[dtang4]

Originally Posted by kdavis26

Great work. I also had a job placement site for a while and I never saw any problems with any duplicate content. The information is close but not the exact same so you never have any problems. If you did, sites like monster and such would never rank well. As far as your results. Great work Keep us updated on how your site is doing.

Thanks a lot.

Yea, great point - the content in the results is never the same for a number of reasons. Moreover, I have customized text all around the page, which further differentiates the pages from Indeed and its other affiliates.

[kazba]

How are you building traffic to all these pages or are you only relying on organic search traffic?

I am very interested in creating these kind of sites but my down fall has always been in getting enough traffic to the site to make it profitable.

Thanks in advance

[dtang4]

Originally Posted by kazba

How are you building traffic to all these pages or are you only relying on organic search traffic?

I am very interested in creating these kind of sites but my down fall has always been in getting enough traffic to the site to make it profitable.

Thanks in advance

I'm just relying on organic traffic.

[Eric Lancheres]

Thanks for the case study, it's interesting to see that you have 300k pages... have so many indexed and yet, are receiving so little traffic.

I'm somewhat suspicious of the quality of your content though, especially since you're scrapping content from wikipedia.

Have you considered doing a backlink campaign for your site to increase the ranking of all your pages?

Since every index page carries "some" juice to it... one thing I could recommend would be to install some internal linking scripts to propagate your own juice. (Sounds dirty but it isn't)

[dtang4]

Originally Posted by Eric Lancheres

Thanks for the case study, it's interesting to see that you have 300k pages... have so many indexed and yet, are receiving so little traffic.

I'm somewhat suspicious of the quality of your content though, especially since you're scrapping content from wikipedia.

Have you considered doing a backlink campaign for your site to increase the ranking of all your pages?

Since every index page carries "some" juice to it... one thing I could recommend would be to install some internal linking scripts to propagate your own juice. (Sounds dirty but it isn't)

Thanks for the feedback.

Well, I'm not scraping wikipedia content. I rewrote/rephrased some of the text from wikipedia, which was a manual effort.

I did SEO initially to accelerate the indexing process. Stopped for a while, but have recently restarted my SEO/backlinking efforts.