Community Discussion

Is it true Robot.txt are no longer used with google

June 27, 2012
seohelper2012 Author Level 1

Write the reason you're deleting this FAQ

I wanted to know if its true robot.txt are no longer needed for google seo purposes
5

Comments 5

June 27, 2012
Bert
They are still valid if you don't want the search spiders to crawl certain pages. This is what the robot.txt file is for. They didn't really have any seo purpose anyway and most people don't bother with them.

Enter a reason for deleting this comment

September 2, 2012
riacom
me same used it to block certain pages

Enter a reason for deleting this comment

December 22, 2018
Corzhens
To be honest I do not know about robot.txt because I am not really well versed with SEO yet. I am glad to encounter this discussion to broaden my knowledge with the search engine crawlers. My question now is the robot.txt is a necessity of the site otherwise the site will not be inspected by the crawler as if it is not updated with the contents because dormant sites are not checked by the crawlers. Correct me if I’m wrong.

Enter a reason for deleting this comment

Please login or sign up to leave a commentJoin