Showing posts with label robots.txt. Show all posts
Showing posts with label robots.txt. Show all posts

Wednesday 8 January 2014

Hi dear friends

Hi my dear friends (the friends of thomas yuvan). I just want to teach Google not to crawl this particular blog. I have recently learnt about the search engine bots. We can do that with robots.txt files. Blogger allows us to enter the robots.txt file. I entered that. I have installed the stat counter codes in this blog. When I publish this post, I can easily track whether it is crawled or not. I have written the below lines:

User - agents : Googlebot
Disallow: /