rblefftest: simple RBL false-positive test
Posted September 27, 1999.
This document describes a simple test that was done to determine if the RBL
spam-prevention utilities returned too many false positives or not to be
deployed into production on my server. I present the data. You may make your
Available for consulting work
I am currently available for consulting work. For consulting,
I am happy to telecommute or come to your site for a period of time.
For more information about my skills and experience,
view my statement of consulting capabilities
or contact me for more information.
Back to davideous.com home.
Purpose of test
The purpose of the test was to determine how many false positives were
identified by the various RBL spam-prevention filters out there. I
simply captured copies of e-mail that would have been filtered by the
RBL filtering services and examined them to see if they were really SPAM
No attempt was made to measure how many SPAM messages slipped through
the filters, which would be a useful datum in determining how effective
the filters are at catching SPAM.
Here is the data I collected over seven days and 5,000 e-mail messages
accepted through SMTP by my machine:
- filtered 3 borderline spam messages. By borderline SPAM,
I mean that they claimed to be corporate opt-in announcements,
but the user many not have opted-in. One was from RealAudio and the other
two were from PowerQuest Corporation.
- filtered 19 spam messages
- filtered 26 legitimate non-spam messages, which break down as follows:
- 8 typical legitimate non-spam messages
- 11 messages from a list-serv using an open relay (the user did subscribe)
- 7 messages from an about.com opt-in list-serv (this is not borderline spam because I know that user opted in)
I admit that this test does not have a lot of datapoints, but I
was able to look at the data and decide if I wanted to deploy
an RBL system or not.
- filtered 10 spam messages
- filtered 12 legitimate non-spam messages
Other people's data
If you have some data that you think would be useful here, please send it to me.
- Qmail list discussion
Some people on the qmail list volunteered their data about how many messages
RBL was rejecting. This data does not show anything about the number of false
positives received, but I've included it here because it is interesting.
Conduct your own test
If you'd like to conduct your own test, here is the software that I
created to run this test.
If you run your own test, I'll include your results here if you like.