Wednesday 8 January 2014

WOW. Everything is blocked


Finally, good learning, everything is blocked. However, stat counter can track this site.


Googlebot's name is changed

I think, Googlebot's name is changed. I'm blocking all bots.

User-agent: *
Disallow:

See more: https://support.google.com/webmasters/answer/1061943?hl=en

Disallow is enough

I think / is not required. If this doesnt work, I will change the name of the Googlebot and will put * in the place to block all the bots.

This time:

User-agent: Googlebot
Disallow: 

My last post was crawled by google

Ouch. My last post was crawled by google. So my coding must be wrong. Let me try this time with this new code:

User-agent: Googlebot
Disallow: /search


Hi dear friends

Hi my dear friends (the friends of thomas yuvan). I just want to teach Google not to crawl this particular blog. I have recently learnt about the search engine bots. We can do that with robots.txt files. Blogger allows us to enter the robots.txt file. I entered that. I have installed the stat counter codes in this blog. When I publish this post, I can easily track whether it is crawled or not. I have written the below lines:

User - agents : Googlebot
Disallow: /