Trouble with blacklists
Posted by Maxim Weinstein
Over at the Washington Post, columnist James McGrath Morris vented this weekend about his trouble with spam blacklists:
I recently wrote e-mail messages to two people at Columbia University. My e-mail was blocked because my Internet protocol, or IP, address was, at the time I pushed "send," listed at www.spamhaus.org. That company’s Web site explains that the firm maintains a database of "IP addresses of verified spam sources and spam operations (including spammers, spam gangs and spam support services)." Spamhaus supplies its list free of charge "to help email administrators better manage incoming email streams."
The list is dynamic, changing all the time. When I checked again later, my IP address was no longer on it. In fact, when I ran my IP address through 125 of the most commonly used blacklists, it was not on any of them. But how many e-mail senders know whether they are on these blacklists or even know these types of lists exist? Worse, the makers of these lists do not contact those whom they damn, so senders are convicted without any chance of offering a defense.
In other words, the 1950s anti-communist blacklists, assembled without due process, have essentially returned in a new form on the Internet.
I’m not sure about the McCarthyism comparison, but Mr. Morris otherwise raises an important point about the power of blacklists and the need for transparency and due process. Back in July, I guest blogged over at the ZDNet Zero Day blog and laid out five principles for a fair website blacklist:
- A low false-positive rate
- Clear, publicly-available criteria for determining which sites are listed
- Information about why a particular site is listed
- A transparent, responsive process for requesting removal of incorrect or outdated listings
- Support and education for owners of compromised sites
The same principles, of course, could and should be applied to spam and other blacklists.
StopBadware.org, together with its data partners, strives to be a model for a system that adheres to these principles. There may be other models that work, too, so one of our goals is to keep experimenting and learning about how to address this challenge effectively, even as we continue to evangelize the importance of the principles.
Mr. Morris also raises an interesting point about blacklists notifying someone when their site/IP gets listed, to which I offer no answers, but only more questions:
- Who should be notified in the event that a computer, an IP address, an IP range, a website, etc., is found to contain malware or send spam? The end user? The site owner? The ISP? The hosting company? The hosting reseller? The domain registrar?
- How can blacklist providers cost-effectively identify and notify these parties in a timely way?
- What is a reasonable expected response from each of the notified parties? (e.g., should an ISP suspend the account of a spamming computer? notify the user? put the computer into a "walled garden" until the problem is fixed? provide support to help the user get the spambot off his/her computer?)
- What is an appropriate response from the rest of the industry and/or the Internet community when a party does not respond "appropriately" to the notification?
These are not easy questions, but they are ones that really need to be answered. Please join us as we strive to lead an effort to answer them by contributing your thoughts in the comments and/or in our BadwareBusters.org community.
"Who should be notified that a computer, an IP range is found to contain malware ..." I do not believe it is incumbent on any blacklist operator to notify anyone that he has added to his list. What I do believe is that it is the responsibility of webmasters and postmasters to periodically check blacklist query engines with their own set of IP addresses to see if they become listed. Checking external sources should be the last line of defense in their vigilance to keep malware off their websites and to assure they have secured the email infrastructure. That's just my opinion. I know others disagree. But, let's talk about who owns this. I like your site! I read it daily via RSS.
Rex, you raise an interesting point, though not all blacklists are publicly searchable. For example, many AV products block known or suspected malware/phishing sites based on lists that are proprietary. I was more interested, though, in researchers and blacklist providers who are willingly trying to notify affected parties, but don't always know who the right party/parties are and what to expect from them once they've been notified.