How much would it cost to have my own custom search engine like google built?

14 years ago

This has three major components. (1) the web crawler to collect the sites and index them (2) the database to store the data and organize it in the way you need it and (3) the GUI to display everything to the users.

If you get someone who really knows what they're going (like yours truly) a good estimate, with a minimal amount of graphic design, would be upwards of $15k. If I bring my design people on board, it would be upwards of $20k

Thanks
1 reply

Signature

Looking for new affiliates for a penny auction web site -- please send me a PM

{{ DiscussionBoard.errors[2664963].message }}

mojojuju

14 years ago

All I know is that you're going to need to rent a garage.

[ 3 ] Thanks
1 reply

Signature

:)

{{ DiscussionBoard.errors[2665037].message }}

maesil

14 years ago

Originally Posted by mojojuju

All I know is that you're going to need to rent a garage.

You will absolutely need a large room to maintain more than massive servers to accommodate all of the files and information for when you list the results of each search query.

Google is worth billions for a reason. It's probably unlikely that someone who wants to know the cost of competing with some of the largest search engines of the world would be able to afford it...

Thanks
3 replies

{{ DiscussionBoard.errors[2665058].message }}

Dylan K

14 years ago

Originally Posted by maesil

You will absolutely need a large room to maintain more than massive servers to accommodate all of the files and information for when you list the results of each search query.

Google is worth billions for a reason. It's probably unlikely that someone who wants to know the cost of competing with some of the largest search engines of the world would be able to afford it...

I'm pretty sure he was hinting at the fact that google was founded in a garage.

Thanks

{{ DiscussionBoard.errors[2665080].message }}

Rob Thayer

14 years ago

Originally Posted by maesil

You will absolutely need a large room to maintain more than massive servers to accommodate all of the files and information for when you list the results of each search query.

Google is worth billions for a reason. It's probably unlikely that someone who wants to know the cost of competing with some of the largest search engines of the world would be able to afford it...

Yeah, it would have to be a pretty big room. It's estimated that Google owns more than 1 million servers... or about 2% of all the servers in the world. So start clearing out that spare bedroom!

A Comparison of Dedicated Servers By Company |

Thanks

{{ DiscussionBoard.errors[2665087].message }}

nailrogers

14 years ago

Originally Posted by maesil

You will absolutely need a large room to maintain more than massive servers to accommodate all of the files and information for when you list the results of each search query.

Google is worth billions for a reason. It's probably unlikely that someone who wants to know the cost of competing with some of the largest search engines of the world would be able to afford it...

I mean you can judge from the fact that Yahoo and Microsoft are trying (collectively) to beat Google. Y! and Microsoft no doubt has the best minds working for them, huge funds+ infrastructure availability which they would love to invest to beat google but still they haven't been able to even get a hint on how to beat Big G.

Boy, aren't we talking big here??

Thanks

{{ DiscussionBoard.errors[2665106].message }}

Quentin

14 years ago

Now that is the post of the week. It seems all great online ventures started in Garages.

Quentin

Thanks
1 reply

Signature

DIY Product Creation

Online Business Training on How To Create Products to sell online Plus Lots More.

{{ DiscussionBoard.errors[2665064].message }}

mojojuju 14 years ago

Yep, the moral of the story is Get a garage
- Thanks
- 1 reply
Signature

:)
{{ DiscussionBoard.errors[2665084].message }}
- Dylan K 14 years ago
  
  Originally Posted by mojojuju
  
  Yep, the moral of the story is Get a garage
  
  lol well put
  
  Thanks
  
  {{ DiscussionBoard.errors[2665111].message }}

Vinchenz

Banned 14 years ago

You can look for a Google directory clone which will have a template and script included.
Look into DNK Google Clone - its free from CNET .com

~Vinchenz

Thanks

{{ DiscussionBoard.errors[2665116].message }}

davewebsmith

14 years ago

here is a clearer outline of the process

http://www.googleguide.com/google_works.html

the problem you would face is the same as starting a competition company to Youtube - with that much data / indexed sites / uploaded videos it would be impossible to catch up / match or come close to google in search results and information

you might look rather at using google's search API within your own custom site but this would still retrieve results from google and display them to the user

i have seen some scripts before that search multiple search engines for results, but since they all use different algorithms so the results vary and therefore the quality of results would vary

Thanks

{{ DiscussionBoard.errors[2665652].message }}

CDarklock

14 years ago

Originally Posted by nelsencaleb

But anyways if anyone knows like a price to develop a search engine like google or yahoo could you please give me a quote.

This is not about massive servers. Servers are cheap. That's trivial.

This is about speed and heat.

Your single largest challenge is query latency, followed closely by the sheer quantity of heat your database servers generate.

Cost of development and hardware is really pretty minimal, and getting smaller all the time. Remember, Google itself was started by a couple of grad students, and attained market leadership largely because of the specific ranking factors they pursued.

The problem is getting your speed up, and keeping the temperature down. Clustered Linux will do what you need, if you can keep the heat under control.

Which, if you ask me, means putting your data centres in Canada. You need a fat pipe to the US, and you need year-round cold.

Thanks

Signature

"The Golden Town is the Golden Town no longer. They have sold their pillars for brass and their temples for money, they have made coins out of their golden doors. It is become a dark town full of trouble, there is no ease in its streets, beauty has left it and the old songs are gone." - Lord Dunsany, The Messengers

{{ DiscussionBoard.errors[2665686].message }}

jazbo

14 years ago

I have seen search engine scripts on hotscripts and other places. There are tons of them and no you do not need a ton of servers.

PHP Search Engine Scripts | Search & Download Software Programs Free

Thanks

Signature

CONTENT WRITER. Reliable, UK-Based, 6 Years Experience - ANY NICHE
Click Here For Writing Samples & Online Ordering

{{ DiscussionBoard.errors[2665786].message }}

mikkosant

14 years ago

So, find yourself a garage in Canada

Cheers,

but seriously, using the api is probably the closest thing..Piggy backing off google itself.

Thanks

Signature

{{ DiscussionBoard.errors[2665794].message }}

thebitbotdotcom

14 years ago

You could create a niche Google Custom Search Engine. Look into it. You can use all of Google's databases and refine the results by hand. Then market it as an enhanced and refined niche search engine powered by Google. I have actually seen these work.

Here is an example of one that I created for friends, just for fun. The customization possibilities are endless.

Moderators: I am just showing this as a relevant example. I am not trying to promote it.

The LSU Tiger Search Engine

Thanks
1 reply

Signature

Do Your Copywriting Skills Suck?

Let Us Help You Develop Your Writing Skills!

Submit Guest Posts With [ TheBitBot.Com ]

{{ DiscussionBoard.errors[2665839].message }}

davewebsmith

14 years ago

Originally Posted by thebitbotdotcom

You could create a niche Google Custom Search Engine. Look into it. You can use all of Google's databases and refine the results by hand. Then market it as an enhanced and refined niche search engine powered by Google. I have actually seen these work.

Here is an example of one that I created for friends, just for fun. The customization possibilities are endless.

Moderators: I am just showing this as a relevant example. I am not trying to promote it.

The LSU Tiger Search Engine

Its a bit like the default search page when you install an application like FireFox - Its a firefox branded google search page showing google results

this again as mentioned before uses googles search api

Originally Posted by jazbo

I have seen search engine scripts on hotscripts and other places. There are tons of them and no you do not need a ton of servers.

PHP Search Engine Scripts | Search & Download Software Programs Free

Yes, however if you read the tasks that these search engines are designed for is

eg 1.) Sphider is a web spider and search engine, suitable for adding search functionality to sites with up to 100,000 pages. It can index both static and dynamic pages, supports boolean and phrase searches and includes an option to build a searchable Yahoo-like directory system. Uses MySQL as its back end database....

This script is designed to add search funtionality to a single site - as in upto 100k pages .. not harvest millions of sites like Google

eg 2.) Create a search engine for your website in minutes! - again this will index your website not the "internet"

eg 3.) With Google Clone Search Script You can run your own Google Like search engine.........(you need a valid Google API key)

eg 4.) phpdig is a fully functional PHP/MYSQL search engine. it automatically spiders websites and stores the description in a MYSQL database. It's especially designed for site-internal search for sites with up to 10.000 pages

So by search engine ... these scripts commonly index and search the folder that you upload the to - ie you have access to them ...

again not harvesting millions of websites and links worldwide .....

hope this helps with clarifications

Thanks

{{ DiscussionBoard.errors[2665934].message }}

Patchworks

14 years ago

I'm sure there are tons of scripts to do this already... Probably some free

Thanks
1 reply

{{ DiscussionBoard.errors[2665962].message }}

A Bary 14 years ago

Originally Posted by Patchworks

I'm sure there are tons of scripts to do this already... Probably some free

Scripts to do what? to integrate API, or to build a Google-Like SE?
- Thanks
{{ DiscussionBoard.errors[2666166].message }}

theentry

14 years ago

Just check out what happened to the last one:
From Cuil to Frozen: The 'Google-Killer' Eats Its Own Medicine | News & Opinion | PCMag.com

Thanks
1 reply

Signature

Nifty Stats - the best free software to track your progress
at multiple affiliate programs within one interface. Give it a try!

{{ DiscussionBoard.errors[2666111].message }}

Bill Farnham 14 years ago

It's way more than your allowance. Way more...

~Bill
- Thanks
Signature
{{ DiscussionBoard.errors[2666140].message }}

A Bary

14 years ago

Originally Posted by nelsencaleb

Hi, the question is the title. I have an idea, nobodies doing it, and I hope I can pay for it in the future unless someone wants to do it for free for me lol. But anyways if anyone knows like a price to develop a search engine like google or yahoo could you please give me a quote.

I have a better idea for you, and the cost won't be much higher, why not to build a new "internet"? aim high-gain high...

Thanks

{{ DiscussionBoard.errors[2666172].message }}

Caleb Spilchen

14 years ago

So, find yourself a garage in Canada

Unfortunately, there are no "garages" in Canada.. Or at least not year-round, since I have to keep re building the 'car igloo' every year

.

Caleb

Note:

Which, if you ask me, means putting your data centres in Canada. You need a fat pipe to the US, and you need year-round cold.

Caliban; To really put the facts down. Its not cold year round

Thanks

Signature

Canadian Expat Living in Medellin, Colombia

{{ DiscussionBoard.errors[2666221].message }}

D_M_S

14 years ago

WTF?

All this talk about servers, garges and heat is complete and total nonsense!

You guys really need to keep up on your tech trends and offerings. Any new venture would use the Amazon cloud. You can start on a micro-instance, about $22/month for Windows, and then scale up to just about anything you need including clustered instances for both your front-end and database.

Thanks

Signature

Looking for new affiliates for a penny auction web site -- please send me a PM

{{ DiscussionBoard.errors[2666831].message }}

kentaiwan98

14 years ago

The real question is: what advantage does doing your own search engine bring you? Do you also have a reason to start a search engine - a gap in the market, a better way to get results, present results, or a faster way to collate information... Seriously, What is your competitive edge?

A niche... a technology... an idea... a market... a language/regional/grouping...

If you have no real edge, then just use Google... it saves you time, money and effort to get the same results. Having worked in an industry where everyone seems intent on recreating the wheel, this philosophy works by simply leveraging other people's work.

My first boss insisted that I rewrite and recreate our own materials, instead of just buying them off-the-shelf. While I enjoyed the work, the effort that we made on the materials simply did not materialize. Even putting 10% of this effort on building a better business would have paid off much more significantly. But we decided to do it the hard way and not leverage other people's work...

In the end, the business failed, though we had a great set of materials. Those materials were never used again.

Kenneth

Thanks
1 reply

{{ DiscussionBoard.errors[2666999].message }}

JohnMcCabe 14 years ago

Originally Posted by Quentin

Now that is the post of the week. It seems all great online ventures started in Garages.

Quentin

A guy I know says he gets all of his great ideas sitting on the throne.

Problem is, all he gets are crappy ideas... :p
- Thanks
{{ DiscussionBoard.errors[2667109].message }}

Rob Thayer

14 years ago

Of course we all know that emulating Google is a ridiculous idea. We throw data center statistics and logistics at the OP, all in good fun.

But maybe there is a kernel of an idea here. Rather than create a general search engine that competes with Google, why not create a niche-specific search engine? For example, maybe you'd crawl/search only new sites... or maybe sites about antiques. Yes, of course Google does this too and so do other sites... but by specializing in a certain area, your search results could be better tailored to your unique user base.

Thanks
2 replies

{{ DiscussionBoard.errors[2667106].message }}

thebitbotdotcom

14 years ago

Originally Posted by Rob Thayer

Of course we all know that emulating Google is a ridiculous idea. We throw data center statistics and logistics at the OP, all in good fun.

But maybe there is a kernel of an idea here. Rather than create a general search engine that competes with Google, why not create a niche-specific search engine? For example, maybe you'd crawl/search only new sites... or maybe sites about antiques. Yes, of course Google does this too and so do other sites... but by specializing in a certain area, your search results could be better tailored to your unique user base.

As was already mentioned that is what Google Custom Search does and you can customize the look. It works very well IF you know what you are doing.

Many people use these for their own websites.

Thanks

Signature

Do Your Copywriting Skills Suck?

Let Us Help You Develop Your Writing Skills!

Submit Guest Posts With [ TheBitBot.Com ]

{{ DiscussionBoard.errors[2667147].message }}

Ken

14 years ago

Originally Posted by Rob Thayer

Of course we all know that emulating Google is a ridiculous idea. We throw data center statistics and logistics at the OP, all in good fun.

But maybe there is a kernel of an idea here. Rather than create a general search engine that competes with Google, why not create a niche-specific search engine? For example, maybe you'd crawl/search only new sites... or maybe sites about antiques. Yes, of course Google does this too and so do other sites... but by specializing in a certain area, your search results could be better tailored to your unique user base.

I believe Derek Franklin has put together a unique app that solves most search problems.

'Searchautomator Force' will retrieve more info than can be reasonably used
about any niche you may be interested in.

It's Free and in my opinion it's the best search tool available.

I am not affiliated in any way.

Go to the site, watch the video, download and be prepared to be blown away.

Ken

Search Automator FORCE - Search The Web With FORCE!

Thanks

{{ DiscussionBoard.errors[2667345].message }}

MichaelHiles

14 years ago

A better business proposition would be to just write a better algorithm and sell it to them instead.

Originally Posted by Rob Thayer

Of course we all know that emulating Google is a ridiculous idea. We throw data center statistics and logistics at the OP, all in good fun.

But maybe there is a kernel of an idea here. Rather than create a general search engine that competes with Google, why not create a niche-specific search engine? For example, maybe you'd crawl/search only new sites... or maybe sites about antiques. Yes, of course Google does this too and so do other sites... but by specializing in a certain area, your search results could be better tailored to your unique user base.

A bunch of people are already working on this in a whole bunch of vertical markets.

Thanks
1 reply

Signature

10XTS... I got your digital strategy right here...

{{ DiscussionBoard.errors[2667116].message }}

Rob Thayer

14 years ago

Originally Posted by MichaelHiles

A better business proposition would be to just write a better algorithm and sell it to them instead.

I really doubt that would be possible. Google's whole business is search... you don't think they haven't worked out the best possible algorithms since they've been around?

A bunch of people are already working on this in a whole bunch of vertical markets.

Yep, I know... but there are PLENTY of other untapped markets to be exploited that way.

Thanks
1 reply

{{ DiscussionBoard.errors[2667148].message }}

MichaelHiles

14 years ago

Originally Posted by Rob Thayer

I really doubt that would be possible. Google's whole business is search... you don't think they haven't worked out the best possible algorithms since they've been around?

That's the funny thing about theoretical mathematics... just when we think that it's absolute, someone comes along with a different abstraction and supporting metaphors and WHAM! Game theory shifts all over the place.

But yeah, Google does have a massive lead and a sizeable patent arsenal.

Thanks

Signature

10XTS... I got your digital strategy right here...

{{ DiscussionBoard.errors[2667211].message }}

Karan Goel

14 years ago

Here's what it will cost to build "like" Google:

1. Degree from Stanford.
2. A rented garage
3. A unique idea
4. Mindset to simplify web (even more)
5. Awesome board members

I think only Google could have afford that.

Karan

Thanks

Signature

SafeSpokes
Penalty Safe, Long Term, 100% Whitehat Backlinks
Love your site? Then check out SafeSpokes!

~_~_~_~_~_~_~_~_~_~_~_~_~_~_~_~_~_~_~_

karan996@irchiver.com karan997@irchiver.com

{{ DiscussionBoard.errors[2667132].message }}

snapcontent

14 years ago

Hang on, doesn't Google run off a couple of old 286s, and a freeware PHP crawling script / mysql???

Thanks

{{ DiscussionBoard.errors[2667143].message }}

How much would it cost to have my own custom search engine like google built?

Trending Topics

Best AI for spell checking & grammar (2024)?

Google Ads, landing pages, and Clickbank links

A Very Strange Internet Problem

Carry out better target audience research

Melb, Interat, Flower Show 2024