Comments on: Search Robots: The Good, The Bad, and The Googlebot https://www.searchenginepeople.com/blog/search-robots-the-good-the-bad-and-the-googlebot.html?utm_source=feed&utm_medium=feed&utm_campaign=feed Canada's Search and Social Media Authority Mon, 10 Nov 2008 07:18:49 +0000 hourly 1 By: Singapore SEO https://www.searchenginepeople.com/blog/search-robots-the-good-the-bad-and-the-googlebot.html/comment-page-1#comment-5214 Mon, 10 Nov 2008 07:18:49 +0000 https://www.searchenginepeople.com/blog/search-robots-the-good-the-bad-and-the-googlebot.html#comment-5214 There are certain bots that we can usually block using the robots.txt and .htaccess file. Very clear example listed.
Rif Chia

]]> By: SoLinkable https://www.searchenginepeople.com/blog/search-robots-the-good-the-bad-and-the-googlebot.html/comment-page-1#comment-5047 Sun, 02 Nov 2008 20:17:14 +0000 https://www.searchenginepeople.com/blog/search-robots-the-good-the-bad-and-the-googlebot.html#comment-5047 Don’t mean to nitpick or anything but I believe you meant to write: DO index this page but DO not read or list links

]]> By: YourDownline.co.uk https://www.searchenginepeople.com/blog/search-robots-the-good-the-bad-and-the-googlebot.html/comment-page-1#comment-5045 Sun, 02 Nov 2008 15:12:29 +0000 https://www.searchenginepeople.com/blog/search-robots-the-good-the-bad-and-the-googlebot.html#comment-5045 Great post, I love the pics lol, they are really good.

]]> By: comunactivo https://www.searchenginepeople.com/blog/search-robots-the-good-the-bad-and-the-googlebot.html/comment-page-1#comment-5042 Sun, 02 Nov 2008 13:28:09 +0000 https://www.searchenginepeople.com/blog/search-robots-the-good-the-bad-and-the-googlebot.html#comment-5042 Very useful post, thanks for sharing! What I can’t figure out is the Yahoo Bot – anyone know if they work much differently to the Googlebots? Can’t seem to even get indexed in Yahoo! Could it be that they index pages less often?

]]> By: mobile wallpapers https://www.searchenginepeople.com/blog/search-robots-the-good-the-bad-and-the-googlebot.html/comment-page-1#comment-5041 Sun, 02 Nov 2008 11:16:04 +0000 https://www.searchenginepeople.com/blog/search-robots-the-good-the-bad-and-the-googlebot.html#comment-5041 Shannon, thank you for so detailed information about bots.

]]> By: Craig S. Kiessling https://www.searchenginepeople.com/blog/search-robots-the-good-the-bad-and-the-googlebot.html/comment-page-1#comment-4875 Thu, 30 Oct 2008 04:15:12 +0000 https://www.searchenginepeople.com/blog/search-robots-the-good-the-bad-and-the-googlebot.html#comment-4875 It’s nice to see such a well-detailed breakdown of all the bots 🙂

]]> By: Comparison Shopping https://www.searchenginepeople.com/blog/search-robots-the-good-the-bad-and-the-googlebot.html/comment-page-1#comment-4736 Wed, 29 Oct 2008 10:17:04 +0000 https://www.searchenginepeople.com/blog/search-robots-the-good-the-bad-and-the-googlebot.html#comment-4736 Thank you for a very interesting and instructive post. I am copy pasting this for reference and further learning. Thank you again.

]]> By: dymphna boholt https://www.searchenginepeople.com/blog/search-robots-the-good-the-bad-and-the-googlebot.html/comment-page-1#comment-4730 Wed, 29 Oct 2008 02:00:47 +0000 https://www.searchenginepeople.com/blog/search-robots-the-good-the-bad-and-the-googlebot.html#comment-4730 Solid Piece!

I have checked the robots.txt file of public black hat bloggers, and it kinda is an interesting learning experience. You get to know the names of all the bad bots, via the instructions they place in there. Go do it, its fascinating.

]]> By: Roger Hamilton https://www.searchenginepeople.com/blog/search-robots-the-good-the-bad-and-the-googlebot.html/comment-page-1#comment-4643 Tue, 28 Oct 2008 15:22:24 +0000 https://www.searchenginepeople.com/blog/search-robots-the-good-the-bad-and-the-googlebot.html#comment-4643 Hey, thanks for the information. I’m still a little unsure of this and your post has allowed me to understand more on it! Especially those nasty bad bots. Thanks again!

]]>