December 06, 2003
Address harvesting in comments

One of my readers just sent me a note saying he's been spammed by some outfit that referenced a blog post of mine in which he'd left a comment, of which his email address was a part. I hate spam, and I hate being even tangentially responsible for someone getting spammed, so I want to do what I can to prevent this from happening. My questions are:

1. Is there a way to prevent webcrawlers from scanning comments? I don't want to keep search engines out, but if blocking them from comments is the only way to achieve this, then that's what I'll do.

2. In the case of this particular spammer, is there anything I can do to prevent them from accessing my site?

3. Any other obvious things I should be doing but probably am not?

Thanks!

Posted by Charles Kuffner on December 06, 2003 to Administrivia | TrackBack
Comments

1:
By default, Movable Type will only show a mail address if no web address is given. In those circumstances, MT encodes the address with entitites. This should stop a great deal of the automated harvesting activity. If your complainant is correct, then either the harvesting software is getting more sophisticated, or a real-live person followed a link and added him to a spam list.

There are two ways to block search engines
A: robots.txt This only blocks polite crawlers.
B: an .htaccess or firewall based block.

2:
a:You may want to post a "no-spamming" policy, and then complain to their isp/upstream, but that may not be effective.
b: tell your reader to put a web page in all his MT blog comments.

3: Change your template not to link comment author's names.

Posted by: Michael on December 6, 2003 2:52 PM

I haven't yet seen this addressed over at Jay Allen's Spam Clearinghouse, but if it continues, I'd post something over there in the comments.

Posted by: Linkmeister on December 6, 2003 6:46 PM

Check out the MT Plugins website (http://mt-plugins.org/), they have one called MT-Obfuscate that purports to hide email addresses from harvestor bots.

I cannot testify to its effectiveness, but at least you can say your trying...

Posted by: Retrogrouch on December 8, 2003 12:19 PM

Check out the MT Plugins website (http://mt-plugins.org/), they have one called MT-Obfuscate that purports to hide email addresses from harvestor bots.

I cannot testify to its effectiveness, but at least you can say you're trying...

Posted by: Retrogrouch on December 8, 2003 12:20 PM