Comments on: How To Use Robots.Txt File? http://www.webresourcesdepot.com/how-to-use-robotstxt-file/ Free Web Resources Sun, 12 Feb 2012 08:29:00 +0000 hourly 1 http://wordpress.org/?v=3.0.4 By: Umut M. http://www.webresourcesdepot.com/how-to-use-robotstxt-file/comment-page-1/#comment-312736 Umut M. Fri, 01 Apr 2011 15:10:49 +0000 http://www.webresourcesdepot.com/how-to-use-robotstxt-file/#comment-312736 @John, It is just a text file that you can create and name robots.txt + place into the website's root folder. @John,
It is just a text file that you can create and name robots.txt + place into the website’s root folder.

]]> By: John Chris http://www.webresourcesdepot.com/how-to-use-robotstxt-file/comment-page-1/#comment-311963 John Chris Thu, 31 Mar 2011 08:50:59 +0000 http://www.webresourcesdepot.com/how-to-use-robotstxt-file/#comment-311963 How to generate robots.txt and where to place this file in site coding..? How to generate robots.txt and where to place this file in site coding..?

]]> By: Ganeshbabu http://www.webresourcesdepot.com/how-to-use-robotstxt-file/comment-page-1/#comment-309908 Ganeshbabu Sat, 26 Mar 2011 16:31:54 +0000 http://www.webresourcesdepot.com/how-to-use-robotstxt-file/#comment-309908 can you explain some more examples to allowed contents and disallowed contents. that must very use ful to beginner(Robots.txt) can you explain some more examples to allowed contents and disallowed contents. that must very use ful to beginner(Robots.txt)

]]> By: Franksite1 http://www.webresourcesdepot.com/how-to-use-robotstxt-file/comment-page-1/#comment-65774 Franksite1 Sat, 01 Aug 2009 20:42:03 +0000 http://www.webresourcesdepot.com/how-to-use-robotstxt-file/#comment-65774 Hey thanks for this information. It has really helped me out in figuring out the first steps in removing my old website. Thanks again! :) Hey thanks for this information. It has really helped me out in figuring out the first steps in removing my old website. Thanks again! :)

]]> By: creativebrain http://www.webresourcesdepot.com/how-to-use-robotstxt-file/comment-page-1/#comment-54495 creativebrain Mon, 08 Jun 2009 14:26:49 +0000 http://www.webresourcesdepot.com/how-to-use-robotstxt-file/#comment-54495 hehe, I only know disallow and allow parameter.. but there are other parameters.. thanks for your post :D btw, how to use Request-rate, Crawl-delay, Visit-time, and Sitemap parameter? can u give me some example? hehe, I only know disallow and allow parameter.. but there are other parameters.. thanks for your post :D

btw, how to use Request-rate, Crawl-delay, Visit-time, and Sitemap parameter? can u give me some example?

]]>