How to check if your site is present in web directories

Posted on July 9th, 2012. Written by Andrei.

People making link building (and their clients too) have a very big problem when it comes to submitting a site to directories: they know the number of directories in which their site is submitted, they know the number (partially) of directories which accepted the link but they don’t know entirely the number of directories which approved the site but didn’t send an approval.

Or, in time, they don’t know the number of directories which still give a link to a domain. After some time, links are lost. It’s important to know which links you’ve lost in order to be able to explain some decreases in SERPs.

These days I’ve remembered an explanation given by a SEO expert referring to the fact that Google Custom Search Engine could help inferior pages of a site to have a better crawling and a better indexing due to the searches made through this personalized search engine. Plus, you can earn something if you have Adsense on them.

I agree with this part, because the arguments are rational and you can test them. The problem I was seeing and still see it is the fact that Google Custom Search Engine acts link a typical Google search engine and so, at some certain results there is a chance for it to not return relevant results. Plus, the freeware version can include only 100 on-demand pages.

Back to our subject, I’ve said that I could make a Google Custom Search Engine in which I could introduce a number of web directories which I will put at crawling and indexing, and after this do some searching and see if the sites are submitted to those directories.

So I’ve picked a list of directories, took the list.txt and put it in Google Custom Search Engine. I’ve set it to return results only from those links and I’ve waited for the indexing to be complete. They say that it takes about 24 hours to return the results, but depending on the number of links, the amount of time can decrease.

I’ve uploaded 606 URLs to return results. That’s the number of URLs contained by the list I’ve used.

Take the specific code for this personalized search engine, put it in a simple HTML and upload it. By this, your page has its own search engine.

 The searches are made after the domain and not after the keywords, meaning that you will search “domain.com” or “www.domain.com”, using “ “ in order to return the results containing the name of the domain. The big problem is that at some results which will have 10 pages you will see only some of them. Google doesn’t show all the pages, because the free version of the engine is limited to 100 pages indexed on-demand.

With other words, if a domain appears on many pages of an included web directory, those pages will remove from viewing other pages of other included web directories.

 You can still somehow estimate the number of directories, because a page has 10 sites, so multiplied by 100 gives you an approximate number. The number of results is not the same with the number of domains. It represents the number of pages from web directories containing that domain.

This is not an exact tool. It’s useful for quick estimations so that you can figure out if a site is submitted in directories or not.

Andrei