Flashes of Panic

After a few weeks of cursory glances at (OK, obsessive combing of) the site traffic report, you get used to some standard spiders, like the googlebot. When you’re a low-traffic site like this one, there’s a little anticipatory glee: hey, more traffic from random Google searches for “drive not ready system halted” (try switching the ribbon cable plugs, dude) or “morning africa antelope” (just get the relative speeds right) or “panic girls” (sorry, can’t help you there.)

On the other hand, for the last day or so there’s been Tutorial Crawler 1.4.

Now, the discussion I found on Webmaster World was pretty mixed. The spider follows the robot exclusion protocol. It doesn’t hit your server hard. But, damn if it doesn’t index everything three or four times. For the last few days, it’s been the number one browser type on this site, outstripping both IE/Win (the most common web browser in the world—I’d say “most popular,” but not everyone who has to use it, likes it) and Safari, which is used by this site’s biggest visitor, me.

And there’s nothing even remotely resembling a tutorial on this site, which is what it claims to be indexing. Nor am I seeing return traffic from their search engine.

Friend or parasite? I can’t tell. At least I know if I close the door, they’ll stay out, unlike the spambots.