Apologies for the following post. It’s really quite dull.
Internet Seer, on the face of it, offer a pretty useful service. What they do, according to the sales pitch, is monitor the availability of your website. They’ll ‘ping’ your domain on a regular basis to check whether it’s online or not, and if they don’t get a response, you’ll receive an e-mail letting you know. Very useful.
In reality, however, I suspect that Internet Seer are not as sweet and lovely as they might appear at first glance, a theory backed up by little bit of googling – there’s this example, for instance. Or this sorry tale. Or this little nugget. Or this healthy dose of vitriol. Or this… you get the picture. And now I’m joining in.
Let’s start here. On this page Internet Seer claim that they adhere to the robots exclusion standard, which is a way for webmasters to declare that they don’t want particular parts of their sites to be indexed, or they don’t want to allow certain robots access at all. A random sample of five recent day’s log files reveals that the Internet Seer robot requested my robots.txt file exactly ZERO times, completely contradicting their own claims.
But does this matter? If they’re only monitoring my site to check it’s availability, then it’s no big deal, right? Well, let’s choose a day at random, say November 19th. On this day the Internet Seer robot visited on a number different occasions, examining 67 different pages as it passed. Now wait a minute… if all Internet Seer are doing is checking to see if my site is online, why are they visting 67 different pages? What use could they possibly have?
I dunno, let’s play devil’s advocate here. What’s the worst they could be doing? Harvesting e-mail addresses, perhaps? Surely not. But then again… two people have e-mailed me recently after receiving an e-mail from the company, notifying them of the following or similar:
On Fri Dec 05, 2003 at 10:55:01 PM EST we were unable to reach your website: http://www.blogjam.com/old/000760.html due to the following reason: Host Not Found. As of Mon Dec 08, 2003 at 11:11:59 AM EST we were able to access your website again.
How strange. Why are other people being notified when my website is supposedly down and, more pertinantly, where did Internet Seer obtain their addresses from? The first part I can’t answer, but the second seems obvious – from my website, where both people in question had left comments on the pages Internet Seer had claimed were down in their respective e-mails. I guess the obvious question at this point is this: how could Internet Seer possibly grab a couple of e-mail addresses if my site was actually down? Playing Devil’s advocate once more, I rather suspect that the company aren’t monitoring my site at all in the way thet they claim. I’m sure Internet Seer will deny all this, claiming that the two people in question are subscribers that signed up for their service (unlikely, since neither has their own website), or that they were picked up during one of Internet Seer’s extraordinarily vague “studies on the connectivity of the internet
and the related web sites that were involved in such studies.”
Suffice to say I’ll be preparing a friendly e-mail to the company to question their actions and, in the meantime, if anyone else reading this has had mails from Internet Seer regarding my site, please let me know. If you’ve still got the original mails, even better – please forward, with full headers. Thanks.
Update: I heard back from Internet Seer. They’ve promised to remove me from their database, and offered the following explanation of how they came to be be e-mailng people who left comments on my site:
I can only assume that we must have noted the emails on those pages during a prior visit, stored them as a contact in the event we found the site down.
Which appears to confirm in no uncertain terms my theory that they’re scraping sites for e-mail addresses. If they’re doing anything further with these addresses other than annoying the owners with irrelevent monitoring mail is a different matter altogether – I’ve no reason to believe that they are, but as their current business model seems at best chaotic and at worst unethical, nothing would surprise me.
How bizzarre – Mr merde receives one today too. Mail sent,
Thanks for all this useful info. I do have a quick question. Would the spidertrap you mentioned on 16 November, 2003 be sufficient to block these clowns or would i have to modify my .htaccess in addition to installing the spidertrap?
Some interesting ideas. Is it possible to get notified of future updates from your blog?
Hi i would like to know how i can get these internetseer clowns from accessing my web site.
Regards Jonny Orange
I run a virtual server with around 25 websites hosted. Two of these sites have been monitored by Internet Seer for some time. When informed that either of these sites are ‘down’ I try to access them and get messages like “HTTP/1.1 New Session Failed”. However, all the other sites on the server and email services appear fine. Last night I added a third website to Internet Seer and was informed that it was ‘down’ and got the same server error. Again, apart from the three monitored sites, all seemed OK on the server. I have now cancelled my account for the third site and await my next alert with interest. If either of the two original sites is ‘down’ but the third is working then I will be sure that it is Internet Seer which is causing the problem. If I cancel all accounts with Internet Seer how do I stop them from accessing the sites in future?
How’z about a new thread for your “Vent – the get it off your chest portal”.
How’z about to the #8#@&*#@* who invented sub-routines which infect computers?
They’re all around us! Frankly, if I had my ‘druthers’ I would relish the opportunity to personally diminish the circumference of their neck using my own bare hands!
When I was a teenager, in High School, they only allowed me to learn how to type on old, poorly maintained, manual typewriters. That was when I was 15. I’m 55 now.
There’s never been a twist-off top I can’t dislodge using these digits of mine!
Just imagine the vents on that thread! It could contain vents about brand names, technicians (or the lack thereof), or the spammers who have yet to find something CONTSTRUCTIVE to do?
So, how’z about it?
Pls. keep me advised. Your site is a godsend!
Yeah….add a new thread to the webpage, “Vent – the get it off your chest portal”. You may have to increase your server’s memory, mostly because I know it’ll be THE most popular on your site!
And websurfers WILL hear about it, read about it, and maybe by then, they’ll wise up, possibly for the first time in their stupid, ignorant, overpaid lives!
Let me know if it’s a “go”, ok?
Hello! Good Site! Thanks you! hukwquwiis
A client of mine received one such email and when I checked his site it was down, and the control panel had it listed as “turned off”. InternetSeer.com is indeed an evil entity and must be destroyed!
One more thing, do you think a meta tag exclude would be helpful?
Since I installed a mail server on my NAS, I can see in my SMTP log that Internet Seer tries to send e-mails to my domain users.
I suspect them to be spammers.
Do they send their ad this way?
I think so and it’s a very bad way.
Today they tried to send 84 e-mails through my SMTP gateway: a try all ten minutes during 14 hours !!!
Internet Seer = Spammers! definitely.