How much would it cost to have my own custom search engine like google built?

by Banned 33 replies
43
Hi, the question is the title. I have an idea, nobodies doing it, and I hope I can pay for it in the future unless someone wants to do it for free for me lol. But anyways if anyone knows like a price to develop a search engine like google or yahoo could you please give me a quote.
#main internet marketing discussion forum #built #cost #custom #engine #google #search
  • This has three major components. (1) the web crawler to collect the sites and index them (2) the database to store the data and organize it in the way you need it and (3) the GUI to display everything to the users.

    If you get someone who really knows what they're going (like yours truly) a good estimate, with a minimal amount of graphic design, would be upwards of $15k. If I bring my design people on board, it would be upwards of $20k
    • [1] reply
    • All I know is that you're going to need to rent a garage.
      • [ 3 ] Thanks
      • [1] reply
  • Now that is the post of the week. It seems all great online ventures started in Garages.

    Quentin
    • [1] reply
    • Yep, the moral of the story is Get a garage
      • [1] reply
  • Banned
    You can look for a Google directory clone which will have a template and script included.
    Look into DNK Google Clone - its free from CNET .com

    ~Vinchenz
  • here is a clearer outline of the process



    the problem you would face is the same as starting a competition company to Youtube - with that much data / indexed sites / uploaded videos it would be impossible to catch up / match or come close to google in search results and information

    you might look rather at using google's search API within your own custom site but this would still retrieve results from google and display them to the user

    i have seen some scripts before that search multiple search engines for results, but since they all use different algorithms so the results vary and therefore the quality of results would vary
  • This is not about massive servers. Servers are cheap. That's trivial.

    This is about speed and heat.

    Your single largest challenge is query latency, followed closely by the sheer quantity of heat your database servers generate.

    Cost of development and hardware is really pretty minimal, and getting smaller all the time. Remember, Google itself was started by a couple of grad students, and attained market leadership largely because of the specific ranking factors they pursued.

    The problem is getting your speed up, and keeping the temperature down. Clustered Linux will do what you need, if you can keep the heat under control.

    Which, if you ask me, means putting your data centres in Canada. You need a fat pipe to the US, and you need year-round cold.
  • I have seen search engine scripts on hotscripts and other places. There are tons of them and no you do not need a ton of servers.

    PHP Search Engine Scripts | Search & Download Software Programs Free
  • So, find yourself a garage in Canada

    Cheers,

    but seriously, using the api is probably the closest thing..Piggy backing off google itself.
  • You could create a niche Google Custom Search Engine. Look into it. You can use all of Google's databases and refine the results by hand. Then market it as an enhanced and refined niche search engine powered by Google. I have actually seen these work.

    Here is an example of one that I created for friends, just for fun. The customization possibilities are endless.

    Moderators: I am just showing this as a relevant example. I am not trying to promote it.

    The LSU Tiger Search Engine
    • [1] reply
    • Its a bit like the default search page when you install an application like FireFox - Its a firefox branded google search page showing google results

      this again as mentioned before uses googles search api

      Yes, however if you read the tasks that these search engines are designed for is

      eg 1.) Sphider is a web spider and search engine, suitable for adding search functionality to sites with up to 100,000 pages. It can index both static and dynamic pages, supports boolean and phrase searches and includes an option to build a searchable Yahoo-like directory system. Uses MySQL as its back end database....

      This script is designed to add search funtionality to a single site - as in upto 100k pages .. not harvest millions of sites like Google

      eg 2.) Create a search engine for your website in minutes! - again this will index your website not the "internet"

      eg 3.) With Google Clone Search Script You can run your own Google Like search engine.........(you need a valid Google API key)

      eg 4.) phpdig is a fully functional PHP/MYSQL search engine. it automatically spiders websites and stores the description in a MYSQL database. It's especially designed for site-internal search for sites with up to 10.000 pages


      So by search engine ... these scripts commonly index and search the folder that you upload the to - ie you have access to them ...

      again not harvesting millions of websites and links worldwide .....

      hope this helps with clarifications
  • I'm sure there are tons of scripts to do this already... Probably some free
    • [1] reply
    • Scripts to do what? to integrate API, or to build a Google-Like SE?
  • Just check out what happened to the last one:
    From Cuil to Frozen: The 'Google-Killer' Eats Its Own Medicine | News & Opinion | PCMag.com
    • [1] reply
    • It's way more than your allowance. Way more...

      ~Bill
  • I have a better idea for you, and the cost won't be much higher, why not to build a new "internet"? aim high-gain high...
  • Unfortunately, there are no "garages" in Canada.. Or at least not year-round, since I have to keep re building the 'car igloo' every year .

    Caleb

    Note:

    Caliban; To really put the facts down. Its not cold year round
  • WTF?

    All this talk about servers, garges and heat is complete and total nonsense!

    You guys really need to keep up on your tech trends and offerings. Any new venture would use the Amazon cloud. You can start on a micro-instance, about $22/month for Windows, and then scale up to just about anything you need including clustered instances for both your front-end and database.
  • The real question is: what advantage does doing your own search engine bring you? Do you also have a reason to start a search engine - a gap in the market, a better way to get results, present results, or a faster way to collate information... Seriously, What is your competitive edge?

    A niche... a technology... an idea... a market... a language/regional/grouping...

    If you have no real edge, then just use Google... it saves you time, money and effort to get the same results. Having worked in an industry where everyone seems intent on recreating the wheel, this philosophy works by simply leveraging other people's work.

    My first boss insisted that I rewrite and recreate our own materials, instead of just buying them off-the-shelf. While I enjoyed the work, the effort that we made on the materials simply did not materialize. Even putting 10% of this effort on building a better business would have paid off much more significantly. But we decided to do it the hard way and not leverage other people's work...

    In the end, the business failed, though we had a great set of materials. Those materials were never used again.

    Kenneth
    • [1] reply
    • A guy I know says he gets all of his great ideas sitting on the throne.

      Problem is, all he gets are crappy ideas... :p
  • Of course we all know that emulating Google is a ridiculous idea. We throw data center statistics and logistics at the OP, all in good fun.

    But maybe there is a kernel of an idea here. Rather than create a general search engine that competes with Google, why not create a niche-specific search engine? For example, maybe you'd crawl/search only new sites... or maybe sites about antiques. Yes, of course Google does this too and so do other sites... but by specializing in a certain area, your search results could be better tailored to your unique user base.
    • [2] replies
    • As was already mentioned that is what Google Custom Search does and you can customize the look. It works very well IF you know what you are doing.

      Many people use these for their own websites.
    • I believe Derek Franklin has put together a unique app that solves most search problems.

      'Searchautomator Force' will retrieve more info than can be reasonably used
      about any niche you may be interested in.

      It's Free and in my opinion it's the best search tool available.

      I am not affiliated in any way.

      Go to the site, watch the video, download and be prepared to be blown away.

      Ken

      Search Automator FORCE - Search The Web With FORCE!
  • A better business proposition would be to just write a better algorithm and sell it to them instead.

    A bunch of people are already working on this in a whole bunch of vertical markets.
    • [1] reply
    • I really doubt that would be possible. Google's whole business is search... you don't think they haven't worked out the best possible algorithms since they've been around?

      Yep, I know... but there are PLENTY of other untapped markets to be exploited that way.
      • [1] reply
  • Here's what it will cost to build "like" Google:

    1. Degree from Stanford.
    2. A rented garage
    3. A unique idea
    4. Mindset to simplify web (even more)
    5. Awesome board members

    I think only Google could have afford that.

    Karan
  • Hang on, doesn't Google run off a couple of old 286s, and a freeware PHP crawling script / mysql???
  • [DELETED]
Join the discussion

Next Topics on Trending Feed