SEOClerks

Building a search engine to compete with Google, Yahoo!, and Bing



Write the reason you're deleting this FAQ

Building a search engine to compete with Google, Yahoo!, and Bing

We all know what the major search engines are, and how much traffic they're getting, but we never really know what goes into planning them and how much work it actually is.  If you don't know how to do the programming yourself, you will have to bring in people who can get the job done for you, which could cost a lot of money and time because of all the intricate things that need to go into just the planning of the engine itself.

Sure, you can go purchase a script that someone else already coded up, but it won't have all the customization you want and it will actually be riddled with holes that someone else could hack into.  Also, you never know if the person who coded it up has a backdoor in the script so they can access your search engine whenever they want, which is a critical security break that would need to be identified before it even happens.  There are many things you'll need to focus on, mainly the coding and security of the engine itself, but you also need to work on the marketing in order to bring people in from all over the world. 

You need to plan out everything in order to know what you need to be done and when it should be done.  You need to bring on plenty of programmers if you can't get it done yourself, who can get the job done for you.  You'll need to do plenty of marketing in order to bring people in, and keep them coming back.  Bug fixes will need attention, which your programmers should be able to do quickly, so be sure to have at least one programmer on call at all times.  You'll need to profit from the engine if it gets too successful because just your server costs could cripple the engine if too many people are using it at once or they come back all the time to conduct searches.  One of the final things you'll need to focus on is the hackers, and how you can keep them out of the backend of your engine, so they can't get in and wreak havoc on your scripts or inject anything they want.

In this discussion, I'll be going over everything you'll need to do in order to put up your own search engine and compete with the top dogs in the industry such as Google, Yahoo! and Bing. 

The Planning
When it comes to any website, not just an engine, you'll need to do a lot of planning.  A search engine isn't a service or a blog, but it needs a lot of attention even on the smallest tasks.  If you're slacking just a little on things, you could have vulnerabilities that a hacker would love to take advantage of lol.

You'll need to plan everything out, from the domain name to hosting and even how you're going to market the engine to bring people in multiple times and make it their primary search engine of choice.  There are so many things that are involved in the planning that you could even miss some things, but as long as you address them before you launch, you should be fine.

With a search engine, you're probably not going to be able to use something like a shared hosting plan to run it efficiently.  You'll likely want to put it on its own dedicated server, which costs more than a common server you're likely used to, but it's worth it. 

Set up a monthly schedule for what needs to be done, and set up milestones for your programmers to hit, so everyone is on track for the launch date that you want to stick to. 

Bringing on programmers
Not just any programmer can come in and program what a search engine does for its visitors.  You will need to find someone that knows the specific coding language that can design crawlers, a system to index everything, remove duplicates from the index, and implement a ranking algorithm that will only push the best websites to the top of the rankings.  You wouldn't want someone to build 1,000,000 backlinks and get ranked #1 for a specific keyword, so your ranking algorithm needs to be coded extremely well and can't be shared anywhere online.

You can find programmers here on SEOclerks or on a website called UpWork.  I've used UpWork plenty of times to find credible programmers, and I've usually had a great experience with them.  Just be sure to hire the people who know what they're doing and have a high success rate with their jobs.

Bringing in recurring visitors
When you run any type of website, you'll need a lot of recurring visitors to become as popular as possible.  You'll need to prove why you're the search engine people should use over Google, Yahoo!, and Bing and that comes in the form of amazing features that everyone gets to use for free without registration.  Every search engine has key tools for you to use, but you have to register to use them, so you'll need to figure out a way you can have multiple tools in order to keep people coming in all the time.

If you get someone to come back a second and third time, it's likely they loved their experience and you have created a loyal searcher Building a search engine to compete with Google, Yahoo!, and Bing  Now you just have to replicate this a few hundred million times and you'll be on everyone's radar lol.

Making money from ads
A good search engine will have to make money somehow, and that means you'll need to set up an advertising platform for website owners to get their links in front of searchers of specific keywords.  If you've ever been on Adwords, it's extremely advanced and seems like an entirely separate company from Google, but it's their main source of revenue and is pretty competitive.

When you're in the early stages of your search engine, you won't need an advertising platform in place for website owners to use, mainly because you will be able to front the money for the server.  When you start to get a massive amount of traffic, you'll need to think about an advertising platform so you can get website owners to pay you for clicks or per impression.  You won't make much per click, but it will usually be enough to profit off of in the long run Building a search engine to compete with Google, Yahoo!, and Bing

Bug fixes
A programmer should always be on call to handle bug fixes.  You will be actively using, and trying to find bugs, in order to fix them and make a users experience top notch.  If you ignore even the smallest bug, you could get a lot of people talking about it, and that's never a good thing.  Think about it, if someone is talking about all the bugs in your search engine and not the benefits of using your engine, you will notice some negative PR spreading like cancer that is difficult to remove.

Fixing bugs on a website like this is crucial, mainly because it could result in a hacker finding them first and breaking your engine completely, which leads to my next point.

Blocking the hackers
Hackers want to find vulnerabilities within a big website and take advantage of it for their own benefits.  Some hackers will just inject their own codes to siphon money from an advertising platform, or they will inject their own scripts to completely break your engine and basically hold it for ransom.  This is actually pretty common, and people send money to the hackers all the time to get their sites back, mainly because they don't have backups or can't access their codes.  For this reason, you'll want to have a backup on an external hard drive just in case you can't access your codes and scripts. 

Your programmers should understand how hacking works and they will be able to block anyone from getting in.  You can also hire a 3rd party security service to handle all of the work, which will cost some cash up front, but it could be extremely worth it.


In conclusion
Building your own search engine, and giving the major engines a run for their money, will take some time to do.  Google didn't start making millions of dollars right out the gates, it took a long time and they eventually got investors who helped push them into the stratosphere.  You'll need to have a bunch of features that everyone wants, but don't have to pay or register for, so think of what you can give out for free and bring in massive waves of people from all over the internet Building a search engine to compete with Google, Yahoo!, and Bing  You need a great team of programmers who know what they're doing, and they will need to know how to block hackers from accessing your codes and scripts, so nothing gets broken or held for ransom.  If you can do all of this, then acquire funding from investors, you might be able to give a major search engine a run for their money but it won't be anytime soon.


Remember to follow me!
https://www.seoclerks.com/user/TommyCarey


Thanks!

Tommy Carey

Comments

Please login or sign up to leave a comment

Join
stephanopoly
Getting traffic to your search engine seems like the hardest part. "Just Google it" has become a common phrase that everyone uses. Instead of saying look it up on the web everyone says "google it". That's pretty stiff competion. Plus Google, Yahoo and Bing has millions to spend on advertising in order to get people to their sites. Seems like you would need to pair up with Firefox or Safarri in order to get enough traffic to make it worth it.



Are you sure you want to delete this post?

jaybee19
That's very true. I can't actually hear anyone anymore who says 'just search it on the web' nowadays, everyone will always say 'just google it' and so I am. Google has definitely set the standards in search engines. I still remember how popular Yahoo was 10 years ago and how they had a very close competition with Google. Now, Google is much more popular and based from my observations, almost everyone if not all uses Google search engine instead of other search engines. And so, creating your own search engine that would compete against the giant Google is definitely not a joke but a very challenging and a totally risky venture.



Are you sure you want to delete this post?

jayjaydimson
Yeah, that's really true. I used that phrase as well instead of saying that try to search it on the internet, just by saying google it, they will simply know what they need to do.



Are you sure you want to delete this post?

Corzhens
I have to agree with the expression “google it” which means people have the habit of using Google for their searching. That expression is also common here especially for the seniors who seem to be the loyal customers of Google because they always do a search whenever they would hit on something in the internet that is new to them. The younger people in their home say that Google has given them respite from the incessant questions of their grandparents.



Are you sure you want to delete this post?

theresajane
Creating a search engine out of a small amount of budget is a complete joke. These major search engines, though didn't hit the jackpot overnight, they sure have spent a lot of money and resources just to take the spot where they are now. So, thinking that you can create a search engine that would compete against Bing, Yahoo and Google, is such a very challenging thing to do and yes, you're right, you need thorough planning, programmers, other resources and of course, money.



Are you sure you want to delete this post?

vhinz
Search Engines like Google, Yahoo, and Bing have servers and data centers that costs billions of money so that they can process and store huge amount of information about the website that they've indexed. Competing with the big Search Engine companies requires really big funding. It's difficult to prove that your search engine is better than the existing search engines if you don't have fast servers and data centers capable of storing huge information.



Are you sure you want to delete this post?

Martinsx1
I used to make use of Ask.com but looking at how Google, Bing and Yahoo took the market with avalanche force, the search engine Ask.com looked obsolete. I'm not even sure whether people still remember the search engine still even exist.

I actually checked out the search engine last month, and it's still functional but it doesn't look good compared to Google, Bing or yahoo.



Are you sure you want to delete this post?

focusedwriter10
Competing with Google, Yahoo and Bing require billions of money. Apart from the development fee, one needs to market it. Nonetheless, I doubt if Google, Yahoo, and Bing can just sit and wait to see any new search engine taking other.
Honestly, this is a race that might cost billions and you will eventually lose. To me, managing fast servers and data centers is no joke, because that builds the credibility of your site.



Are you sure you want to delete this post?

vhinz
It also need great analyst, researchers, programmers, and engineers. If there are people that has the needed skills, possibly they will be aiming to work at the largest search engine company.



Are you sure you want to delete this post?

focusedwriter10
Very true, which calls for extra funding, even before you get the results. This kind of investment needs a well-thought-of strategy, otherwise, it might fail within a short time.



Are you sure you want to delete this post?

NerdIndeed
Creating a robust search engine website that can match Google would take TONS and TONS of resources. If I am not mistaken, I believe Google downloaded ALL the pages on the internet to come up with the best algorithm for their services. It requires a lot of data centers to store all that data.



Are you sure you want to delete this post?

Martinsx1
Seriously, doing it almost impossible task to accomplish. Even it's going to be possible, it's going to require billions to dollars just like you mentioned before on it needing lots of resources to be able to pull it off.

It's just like trying to create a new social media that can kick off Facebook and Instagram from business, it's almost impossible to achieve that.



Are you sure you want to delete this post?

NerdIndeed
Also, let's say you have a vision. You need to have the proper funding for it. Where will you ever find an angel investor that will solely bet on your idea? Unless it's "the cure for cancer," or your family is filthy rich, it will be tough.



Are you sure you want to delete this post?

Martinsx1
In as much as it's interesting to conceive the thoughts of building a new search engine capable of competing with Google, it's going to be very difficult getting it done because of the level of supremacy Google have toady in the world. Bing and yahoo have been there for so long but they still haven't offered serious competition to Google. The only to being able to achieve this would be to steal Google's core foundation staff and programmers to sell you the secrets of Google.



Are you sure you want to delete this post?

jayjaydimson
Well, I think having your own search engines is not an easy task, you really need to invest a lot of time, money and efforts for it to be done successfully.



Are you sure you want to delete this post?

Barida
Employing people that are going to make this work is going to cost a whole lot of money no matter how we look at it unless we are not looking to challenge the top people in the industry. We should map out plans to ensure that everything works pretty fine when we are looking at having search engine built from scratch.



Are you sure you want to delete this post?

jaymish2
I think this is a really fantastic idea. I hate google and I think its a monopoly. We all know what monopolies do, they take advantage of their position. And they think they can do anything they want.Look how hard it is to get accepted to Google Adsense. This is because you need them more than they need you. I would really love someone to do this and give Google a real run for their money and a good kick up their...



Are you sure you want to delete this post?

HappyLady
I really like this idea and could see it working in with other Open Source projects with a great deal of advantage, I find it amazing that for most things you pay for you can find a perfectly viable Open Source alternative. But then I am a fan of Ubuntu, LibreOffice, and all the others. It can certainly be done if people worked to create those. Maybe it could be worked on by a collective as happened with Open and then Libre Office? I'm not clever enough to do it, but would love to test it out. Maybe survey companies would be a good route to pay for the working of it as many people do earn from that route. (I certainly do.) The use of Google Adsense is not easy for amateurs but a simple way of adding adverts would be great. The main thing would be to get it as good or even better than the Google search engine. It would perhaps need something Google has not got. Perhaps it could incorporate some of the things we add on from grammar checkers and as I had a spell unable to type an integrated voice to text that really worked would be fantastic. Side chat bar to create your own community. Integrated easy resizing of pages. Maybe also some way of raising money for a charity.



Are you sure you want to delete this post?

anyone01
Ubuntu is really great for operating with Android Studio. I’ve been using Android Studio with Ubuntu for yearss and as of that I'm honestly happy with it. The hassle you might face might be adaptability. Not necessarily though. The alternate from windows to Ubuntu certain leaves some matters behind. If your primary concern is Android improvement, there’s infrequently any trouble you’ll face. Just make yourself at ease with terminal commands. That would assist you a lot common.



Are you sure you want to delete this post?

overcast
I think real time search with the updated keywords and the search pattern is lot harder. That is one reason there are only two search engines today. And even the duckduck go is making use of the Bing result. And Yahoo also makes use of the Bing result. So that goes to show you that there seems to be a lot of improvement with Google and other similar search engines. But real time search makes it harder to compete.



Are you sure you want to delete this post?

Barida
The summary is that we should always bring on board those programmers that really understand what they are doing in such niche. Building a search engine is something that will also require a whole lot of work to achieve and being able to bring it into limelight is something that is very important as well. You can't just have it all unless you want to lose out at the end.



Are you sure you want to delete this post?

Everett
Creating a search engine is a huge task this is why there are only a few search engines. For one, the major search engines such as Google, Bing, and Yahoo are basically the main go to's. I remember when people wanted to compete with the main search engines and there was a lot of scripts out there. I do not believe any of those websites ever made it.

I would never want to compete with Google. They have so much money and resources that it may be impossible to compete with them. The only search engine that comes close is Bing search engine by Microsoft. Microsoft has basically unlimited money to work with.

Also on many devices search engines like Google are already embedded into the device. If I say "Hey Siri look up SEO techniques", Siri would automatically use Google for that search query. Based on that your search engine would have to build trust for these companies. They won't just add your search engine because it's a search engine you would need to prove why you are better, or how you are different.

It has been years and the search engines operate the same way. Type in a query and you're flooded with results. I sometimes wonder what the exact future of searches will be like.



Are you sure you want to delete this post?

kaka135
This is a genius idea! I have never thought of building a search engine, but I think it's not impossible at all. Even though Google seems like dominating the search engine world, there are still new Search Engine coming up and there are still users for those new Search Engine, so I believe if one can plan and manage the project well, it's still possible to build a search engine that can compete with Google, especially if there are good features that Google doesn't provide.

I understand all the steps mentioned above, but I have never thought about the hacking part. It's really important, and if we are building the search engine, getting programmers who can handle the security well and block the hackers is important.



Are you sure you want to delete this post?

anyone01
To be honest it is miles very hard for the new company to compete with search engine like google etc. Google has made a strong basis and had a protracted journey for achieving their current position. They started out as simple website, now it has prompted the people to the center. It had made majority of the internet users to assume, that they can't live on without google or miss google throughout their surfing activities. As I've said earlier It's been a long adventure for Google to attain this position and as of that for me it was impossible to compete big company like that.



Are you sure you want to delete this post?

esteban123
I agree that having a good search engine will give us more benefits especially into earning possibilities because that is what some people or business minded people are looking for. It would give us some huge amount of money. But I know making a search engine is really tough especially if we don't have much knowledge and skills to do that, it would really cost a lot of time and money in order to build a good and reputable search engine but I know that it is not impossible as long as we are determined to do it.



Are you sure you want to delete this post?

cmoneyspinner
Build your own search engine? Why not? Isn't that what China is doing? This is a major initiative and if a person wants to take it on I admire them. I'm content being a Googler.



Are you sure you want to delete this post?

Corzhens
Have you seen the list of new search engines last year that was in this forum? If my memory serves me right I think there are more than 10 but I don’t remember any of those which means they did not prosper much. One freelancer said that creating your own search engine is easy and the hardest part is in getting users to like it. Take note that until now Bing is still struggling with the rankings that it couldn’t even get in stride with Yahoo Search in spite of Bing Rewards.



Are you sure you want to delete this post?

koki12346
This is very ambitious! I hope that you make it my friend. This is very complex also and it surely isn't easy to do. Wish you all the best!



Are you sure you want to delete this post?

mhingnhormz
Building a search engine to compete with Google, Yahoo, and Bing: It was a very very big plan an it cost billions to make your start since your competing to the biggest search engine company as of todays. To be honest in my opinion it was vey hard to achieve especially Google is the term of mouth of allmost people now in terms of searching.



Are you sure you want to delete this post?