The first mail delivery attempt (by a server that has never attempted delivery before) connects to port 25 is redirected to port 8025 on localhost. This is done with pf redirect rules. If the delivering server is not currently in the whitelist pf table it goes directly to spamd. The delivering mail server is denied delivery by spamd and told to come back at a later time. The mail servers visit is logged in spamd database and a timer starts. During this time the marked server is not allowed to deliver it's mail. It can come back as many times as it wants in this time period but will always be denied delivery. The connecting server is in what is called a "grey state" and is put on a greylist. This means it is not blacklisted (never allowed) nor whitelisted (always allowed).
Once our specified deny time period ends the delivering mail server will hopefully attempt another delivery (most legit servers will). When it does it will be denied delivery once again. The difference this time is it's status in the spamd database will be changed from grey to white. It's ip address is put in the pf whitelist table and any delivery from this server is redirected straight to the mail server (port 25) on any future delivery attempts. The future delivery attempts for a whitelisted server is done via the whitelist table in pf. Once spamd puts the server's ip in this table the best match in the firewall rules is the whitelist table match. Any servers in this table are redirected to the mail server.
The pf whitelist table is mantained by the spamd deamon. It will insert and remove entries based on it's database. Spamd removes whitelist entries from the spamd database if no mail delivery activity has been seen from the whitelisted address by spamlogd within our specified number of hours from the initial time an address is whitelisted.
After being white listed the delivering server will try another delivery attempt. This time it will be redirected right to mail server.
Spamd is installed on OpenBSD by default. The following are configuration options we are going to use to run spamd. The "-5" option is for black-listed entries, it returns an error code 550 to the spammer. The "-S12" option will stutter greylisted connections for the specified amount of seconds, after which the connection is not stuttered. The next option is "-G25:4:864". The first 25 stands for pass time in mins. Spamd will not accept mail from the initial connection attempt time up to this set time (25). At this point the server is grey listed. The second number 4 is called grey expiration time (hrs). This is the maximum amount of time the server to try to make another delivery after the 25 min pass time period. So, the server can attempt a delivery between 25 mins and 4 hrs after the initial connection attempt and it will be whitelisted. But remember when it connects during this time period it will be denied and whitelisted at the same time. It will have to come back one last time to delivery it's mail. It can come back any time after it is whitelisted. The last number is called the whitelist experation time. It is the time you have in hours to send a message to the server after you have been whitelisted. If you don't send a message to the server from the time your whitelisted up to this time then you will be removed from the whitelist and have to go through the whole filtering process over again. "-b" tells spamd what ip to bind to. "-v" turns on verbose logging. "-nOpenMailServer" is the SMTP version banner that is reported upon initial connection. Change that to whatever you want. The spamd line is put in the /etc/rc.conf file. An example of the revelant lines from this file are below.
The /etc/rc.conf below has pf being turned on and points to it's config file. It will also turn on spamd (in greylisting mode) and sendmail (or if you installed and configured postfix then that). The rest of the lines are not needed to get spamd working but are there for reference.
# These are the settings needed for spamd and postfix and or sendmail to work. pf=YES pf_rules=/etc/pf.conf sendmail_flags="-bd -q30m" spamd_grey=YES spamd_flags="-5 -S20 -G25:4:864 -b127.0.0.1 -v -nOpenMailServer" syslogd_flags="-a /var/spool/postfix/dev/log" # These settings are not needed to get spamd and postfix working they are just what I use. # The rest of the options in the rc.conf are either left blank or turned off. dhcpd_flags="fxp1" ntpd_flags="-s" sshd_flags="-4"
These are the lines you will need in your pf.conf file to get spamd working. Only the lines that involve spamd are below. You will have to fit these into your ruleset in the correct order (Macros, tables, options, traffic normalization, queuing, traslation, and packet filtering). The comments explain what the rules do. I have left in the rules for my MTA Postfix as an example of what you would need to do when using spamd and Postfix together. Postfix and spamd listen on localhost and connections for both are redirected to localhost from the outside. After putting the rules in restart PF (pfctl -f /etc/pf.conf).
# use a macro for the interface name, so it can be changed easily ext_if = "fxp0" # Spamd whitelist table table <spamd-white> persist # Outside servers whitelist. Uncomment if you use a generated whitelist like in script at bottom of page. # table <whitelist> persist file "/etc/whitelist.txt" # Outside servers whitelist redirect to postfix. Uncomment if used/needed. Line above goes with this. # rdr on $ext_if proto tcp from <whitelist> to ($ext_if) port smtp -> lo0 port smtp # Spamd and Postfix redirect on external interface. # If in whitelist table go to Postfix. If not in whitelist table go to spamd. rdr on $ext_if proto tcp from <spamd-white> to ($ext_if) port smtp -> lo0 port smtp rdr on $ext_if proto tcp from !<spamd-white> to ($ext_if) port smtp -> lo0 port spamd # Postfix Mail Server (internal interface) redirect for internal hosts. rdr on $int_if proto tcp from any to $int_if port smtp -> lo0 port smtp # Allow traffic for spamd from outside. Synproxy the connection. pass in quick on $ext_if inet proto tcp from any to lo0 port spamd flags S/SAFPRU synproxy state # Allow traffic for postfix mail server from anyone on the outside and the inside. pass in log on $ext_if inet proto tcp from any to lo0 port smtp flags S/SAFPRU synproxy state pass in on $int_if inet proto tcp from any to lo0 port smtp flags S/SAFPRU keep state
Setup and save a very simple /etc/spamd.conf file for now. We will do blacklist checks with the MTA.
all:\ ::
You should be able to just reboot your machine to have all of these services start. I would suggest doing it to be sure they all start when the machine is rebooted. After rebooting you should see spamd (spamd,tcpdump) running. Hopefully you will see your MTA running also (postfix or sendmail or whatever). Tcpdump is running for spamd to watch incoming connections from mail servers. It uses it to keep the spam database (spamdb) up to date.
After being setup you can try to send an e-mail to your machine. Here's to hoping you setup all your dns records correctly. If you did and the e-mail gets to your machine spamd should respond with the a general message of "Come back later". Now you have an entry in your spam database (spamdb). To check this just execute the program: spamdb. You will see output like the following below.
GREY|10.20.30.40|| |1152284462|1152298862|1152298862|1|0
Then after comming back after our pass time but less then our greyexp time you will see a whitelist entry like the following
WHITE|10.20.30.40|||1152284462|1152286088|1155396513|2|0
For GREY or WHITE entries, the format is: type|source ip|from|to|first|pass|expire|block|pass The fields are as follows: type WHITE if whitelisted or GREY if greylisted. source ip IP address the connection originated from. from envelope-from address for GREY (empty for WHITE entries). to envelope-to address for GREY (empty for WHITE entries). first time the entry was first seen. pass time the entry passed from being GREY to being WHITE. expire time the entry will expire and be removed from the database. block number of times a corresponding connection received a temporary failure from spamd. Usually the blocked delivery attempts up to the passtime mins time. pass number of times a corresponding connection has been seen to pass to the real MTA by spamlogd. Successful e-mails passed to postfix.
To see the time entries in a human readable format just use the date command with "-r" on the system. Times are in seconds since the Unix Epoch. For example:
date -r 1152286088
Will give the output:
Fri Jul 7 11:28:08 EDT 2006
Once the delivering server is whitelisted it can deliver the mail unhindered. The server will make the delivery and MTA will handle the mail transfer. That should be it. Spamd should be up and rolling.
One thing about Spamd. It can be a problem with webmail services like gmail and hotmail. The problem goes like this. When someone sends and e-mail from this service the first delivery is sent out by lets say server A. It will be greylisted like it is supposed to. Then the webmail system waits and tries delivery again but this time from server B. Sever B is then greylisted. It the webmail system waits for some time again like it is supposed to and tries to deliver the mail again this time from server C. Server C is then greylisted. It can go on and on like this never having the same server get back to our server in the right amount of time before the greylisting is removed. Then the process would have to start over again. This happens because large e-mail services use pools of servers to deliver mail. You can never expect to get the mail delivered from the same server. People who run greylisting servers try to get around this by whitelisting as many webmail servers then can find. You can comb through past e-mails from people that sent mail from these services and try to guess the naming scheme. There is a nice script that checks the SPF records and a mantained whilelist file. This will help in trying to whitelist valid servers. The results of this file need to be put in a whitelist table for PF. For anyone intrested the script looks like:
!/bin/sh # Script from http://home.xnet.com/~ansible/openbsd_spamd_conf.html FILE=/etc/whitelist.txt rm -f $FILE touch $FILE for domain in \ aol.com \ apple.com \ amazon.com \ gmx.net \ _spf.google.com \ spf-a.hotmail.com \ spf-b.hotmail.com \ spf-c.hotmail.com \ spf-d.hotmail.com \ _spf-a.microsoft.com \ _spf-b.microsoft.com \ _spf-c.microsoft.com \ mynethost.com do echo \#$domain >> $FILE; dig $domain TXT +short | tr "\ " "\n" | grep ^ip4: | cut -d: -f2 >> $FILE; done echo '# http://cvs.puremagic.com/viewcvs/*checkout*/greylisting/schema/whitelist_ip.txt' >> $FILE; ftp -o - http://cvs.puremagic.com/viewcvs/*checkout*/greylisting/schema/whitelist_ip.txt \ | awk '/^[^#]/ {print $0}' >> $FILE;
Whatever you do it's a pain but it may be a small price to pay to for the reduction in spam.