14 {{ upvoteCount | shortNum }}
14 {{ upvoteCount | shortNum }}

Creating A Script That Creates Millions Of Pages

by PeterLarson
12 replies
I need to find a programmer who can create a script that creates millions of pages on a site it is installed on.

Where do I find such a programmer and how much should I pay a programmer in the Philippines or India for such a script?

Are they hard to create? I am looking for information without giving away too much information about what I am trying to accomplish.

What strategies can I use to find such a programmer at the lowest price?
#creates #creating #millions #pages #script
  • Profile picture of the author mojojuju
    Originally Posted by PeterLarson View Post

    Are they hard to create? I am looking for information without giving away too much information about what I am trying to accomplish.
    The following script will create a million pages...

    <?php
    $i=0;
    while ($i < 1000000) {

    exec('touch ' . $i . '.html');

    $i++;
    }
    ?>

    That's easy to do. So I guess the answer to "Are they hard to create?" depends on the complexity of the pages you want to create.
    Signature

    :)

    {{ DiscussionBoard.errors[8869921].message }}
    • Profile picture of the author Brandon Tanner
      Millions of pages? I don't know what your hosting setup is, but attempting to run a script like the one above on a shared server would get your hosting account shut down instantly. You'd probably need quite a few dedicated servers for that sort of volume (as a point of reference, Wikipedia has ~5 million articles, and they have hundreds of servers).

      You'd also need to set both 'set_time_limit' and 'max_execution_time' to "0" in your php.ini file, so that the script doesn't time out (creating millions of files via PHP will take a looooooooong time). And you'll need to make sure that all of the directories that will be written to have the proper 'write' permissions. Error handling / logging would be a good idea too.

      As per how difficult it would be to write the script? As mojojuju said... it all depends on what kind of content you want to put on each page. It could be anything from "super simple" ($) to "super complex" ($$$$). No way to give you a better answer than that without knowing more details. Either way, it sounds like your hosting costs will be a lot more expensive than the script development.
      Signature

      {{ DiscussionBoard.errors[8870383].message }}
      • Profile picture of the author HJdev
        Originally Posted by PeterLarson View Post

        I need to find a programmer who can create a script that creates millions of pages on a site it is installed on.
        Are you using a CMS? What CMS?

        Originally Posted by Brandon Tanner View Post

        Millions of pages? I don't know what your hosting setup is, but attempting to run a script like the one above on a shared server would get your hosting account shut down instantly. You'd probably need quite a few dedicated servers for that sort of volume (as a point of reference, Wikipedia has ~5 million articles, and they have hundreds of servers).
        It's not the amount of pages, but the amount of visitors (connections x bandwidth) that causes the problem for Wikipedia.
        {{ DiscussionBoard.errors[8870752].message }}
        • Profile picture of the author Brandon Tanner
          Originally Posted by HJdev View Post

          It's not the amount of pages, but the amount of visitors (connections x bandwidth) that causes the problem for Wikipedia.
          Of course Wikipedia has tons of traffic. But if you're serving up millions of pages and NOT expecting/planning for tons of traffic, then what's the point? Sounds like an ROI trainwreck, if that's the case. Hard to make a meaningful assessment though, without knowing more details about the OP's idea.
          Signature

          {{ DiscussionBoard.errors[8871003].message }}
          • Profile picture of the author PeterLarson
            Is it possible to reverse engineer a script on a website on the internet that has a script that creates millions of pages? Or, would you need to get inside of the control panel to reverse engineer such a script?

            Can it be done just by looking outwardly at the kinds of pages that are created and then do coding to recreate the kind of pages you see on the prototype site after looking at the pages outwardly and not going into the control panel? You would then add the script that creates millions of pages to the rest of the page coding?
            {{ DiscussionBoard.errors[8872308].message }}
            • Profile picture of the author Brandon Tanner
              Originally Posted by PeterLarson View Post

              Is it possible to reverse engineer a script on a website on the internet that has a script that creates millions of pages?
              You can only reverse engineer source code that gets output to the browser (ie HTML, CSS, Javascript). You can not reverse engineer source code that never gets output to the browser (PHP). Since PHP is the only one of those languages that can actually create files, then the answer to your question is... no.
              Signature

              {{ DiscussionBoard.errors[8873754].message }}
            • Profile picture of the author shahriyar
              Originally Posted by PeterLarson View Post

              Can it be done just by looking outwardly at the kinds of pages that are created and then do coding to recreate the kind of pages you see on the prototype site after looking at the pages outwardly and not going into the control panel? You would then add the script that creates millions of pages to the rest of the page coding?
              You cannot get PHP/ASP source of a page.

              You can get the HTML/Source of the page and create a page using that. But usually it may be broken if CSS/JS paths are relative, in that case you can programmatically fix the links or retrieve the CSS/JS etc. from the source site, save on your server and link to it. Its possible but page may not be perfect.
              Signature
              Super WSO => No hype, No Catch, 25+ SEO Tools


              **Keep Hackers Away** 176+ Rock Solid Security for Wordpress
              {{ DiscussionBoard.errors[8876475].message }}
  • Profile picture of the author fordix
    You can easily create millions of pages. You can do this even using descent CMS like wordpress. But the question is: what will the pages will contain? from SEO POV, they will all contain duplicate content probably, or will not have any value to the user.

    Most of the websites that contain millions of pages have some unique content in them. Also as someone has already told you, your hosting provider will not tolerate this (if it's shared hosting), you will probably need an expensive dedicated server.

    In short: I'm not sure what exactly you are trying to achieve in the big picture, but it's seems like a step in the wrong direction.
    {{ DiscussionBoard.errors[8872873].message }}
  • Profile picture of the author Danny Shaw
    These scripts are already out there and are very common in the BH community. I have tested them myself and are only any good for very targeted local websites.
    Signature
    **5 DAY FREE TRIAL** - The ultimate social media bot (FB, Instagram, Pinterest & G+).........
    Grab it >> HERE
    {{ DiscussionBoard.errors[8872885].message }}
  • Profile picture of the author michaelgoldstein
    Originally Posted by PeterLarson View Post

    I need to find a programmer who can create a script that creates millions of pages on a site it is installed on.

    Where do I find such a programmer and how much should I pay a programmer in the Philippines or India for such a script?

    Are they hard to create? I am looking for information without giving away too much information about what I am trying to accomplish.

    What strategies can I use to find such a programmer at the lowest price?
    That is easy. Don't physically create the pages. Let them be generated "on the fly".

    Create a .htaccess file and insert this
    Code:
    RewriteEngine on
RewriteBase /
RewriteCond %{http_host} !^www.yourdomain.com$ [nc]
RewriteRule ^(.*)$ http://www.yourdomain.com/$1 [r=301,nc,L] 
RewriteRule ^(.+).html$ /index.php?page=$1 [QSA]
    Next create your index.php

    Code:
     <?
   $page = $_GET['page'];
   if (!empty($page)) {
      echo $page;
   } else {
      echo "Empty page";
   }
 ?>
    Warning: Make sure to use something like htmlawed to remove any injected code to $_GET['page'];

    Now you just created unlimited pages:
    http://www.yourdomain.com/1.html
    http://www.yourdomain.com/2.html
    http://www.yourdomain.com/whatever.html

    Hope this helps.

    Michael
    {{ DiscussionBoard.errors[8875691].message }}
  • Profile picture of the author biggenius
    What's the point in doing that? There are already too much junk on internet. You won't rank anywhere with junk content. You won't make a cent doing that.
    {{ DiscussionBoard.errors[8875719].message }}
    • Profile picture of the author michaelgoldstein
      Originally Posted by biggenius View Post

      What's the point in doing that? There are already too much junk on internet. You won't rank anywhere with junk content. You won't make a cent doing that.
      If you are just creating millions of pages with crap content, then your statement is absolutely true.... but if you are generating "quality" pages (= what the visitor is searching for), then it can - in fact - be very effective.
      {{ DiscussionBoard.errors[8875731].message }}

Trending Topics