Defense against SPAMs

We describe here one possible counter measure against the increasing flow of SPAM mails.

Content:

1. Introduction
2. What to do   <== NECESSARY !!
   1. Activate spamassassin
   2. Teaching the program to recognise SPAMs
3. Configuring spamassassin (you can skip this)
   1. Change the weight of the scores
   2. Where is what: Customising SpamAssassin
   3. File Locations
   4. Automatic Whitelist System

BUT YOU WILL HAVE TO START IT AND TEACH IT !!
 

2- What to do

Activate spamassassin

1. log in into one of our UNIX (SUN) machines
   (from a PC for instance you could use the TELNET facility)
2. edit your ~/.procmailrc file and add the lines given below.
   Please use an UNIX editor (xemacs, emacs, vim, gvim, vi, etc) because this utility need the LineFeed character! Otherwise you could edit the file on your PC and then on the UNIX side convert it:
   your_UNIX_machine% dos2unix your_PC_file ~/.procmailrc

   If you do not have this file, then this is a minimum one that you can create with the following commands:

   your_UNIX_machine% cd  
   your_UNIX_machine% cat >.procmailrc
   SHELL=/bin/sh
   MAILDIR=$HOME/mail_folders
   
   INCLUDERC=/usr/pack/spamassassin-2.60-ds/procmail.inc
   
   :0:
   * ^X-Spam-Status: Yes
   spams
   ^D     <-- this means Control + D keys at the same time 
   

   Notice that the line above consists of four strings:
   "star" "blank" "^" "X-Spam-Status: Yes"

   Now all mails which are detected as being a SPAM will be written into your subdirectory
   ~/mail_folders/ with the name spams

   PLEASE, from time to time go and clear up this file !!!!!!!

   If you do NOT wish to store (and thus be able to check) those mails, then substitute /dev/null for spam in the last line of ~/.procmail above. (This is NOT recommended.....)

   Please, feel free to look at my /home/ballisti/.procmailrc file

For those of you who want to know more:

• Instruction from David Schweikert at the Dept. Web Site:
  http://computing.ee.ethz.ch/email/spamassassin.en.html
• http://www.spamassassin.org
• A list of scores is in http://www.spamassassin.org/tests.html

Teaching the program to recognise SPAMs (Learning)

Since SpamAssassin now includes a Bayesian learning filter (in version 2.50 on), it is worthwhile training SpamAssassin with your collection of non-spam and spam, if possible. This will make it more accurate for your incoming mail.
Do this using the "sa-learn" tools.

AS YOU SHOULD DO IT IN OUR SYSTEM:

Teach to recognise spam mails:
bigbang:ballisti:[ballisti]:93 % cd mail_folders
New directory is : /home/ballisti/mail_folders
bigbang:ballisti:[mail_folders]:95 % sa-learn --spam --mbox SPAM
Learned from 44 message(s) (49 message(s) examined).
bigbang:ballisti:[mail_folders]:96 % sa-learn --spam --mbox spams
Learned from 238 message(s) (250 message(s) examined).

Teach to recognise good mails:
bigbang:ballisti:[mail_folders]:97 % sa-learn --ham --mbox SYSTEM/betriebsinfo
Learned from 42 message(s) (42 message(s) examined).
bigbang:ballisti:[mail_folders]:98 % sa-learn --ham --mbox SYSTEM/erledigt
Learned from 449 message(s) (451 message(s) examined).
bigbang:ballisti:[mail_folders]:99 % sa-learn --ham --mbox SYSTEM/complaints
Learned from 59 message(s) (62 message(s) examined).

Use as many mailboxes as you like. Note that SpamAssassin will remember
what mails it's learnt from, so you can re-run this as often as you like.

As given by the instructions in the distribution package:
(do NOT do it like this, this is kept here for information only)
sa-learn --spam ~/Mail/saved-spam-folder
sa-learn --nonspam ~/Mail/inbox
sa-learn --nonspam ~/Mail/other-nonspam-folder

Configuring spamassassin

How to change the weight of the scores

You may want for instance to change the weight of the scores:
then create a new file ~/.spamassasin.cf as follows:

cat >.spamassasin.cf
score  JAVASCRIPT  1.0
^D

(this is just an example)
otherwise edit the file ~/.spamassassin/user_prefs as explained below.

Where is what: Customising SpamAssassin

These are the configuration files installed by SpamAssassin.
Note that in our system we use a special method for packages called SEPP. Thus the file position is quite different that the one you find in the distribution documentations.
The commands that can be used therein are listed in the POD documentation for the Mail::SpamAssassin::Conf class.
Run the following command to read it:
perldoc /usr/pack/spamassassin-2.60-ds/lib/perl/5.8.0/Mail/SpamAssassin/Conf.pm .
Note: The following directories are the standard defaults that people use. There is an explanation of all the default locations that SpamAssassin will look at the end.
Note that the location in our (IFH) system is always under
/usr/pack/spamassassin-2.60-ds/
but the following description is more general and applies to the distribution package:
• /usr/share/spamassassin/*.cf:
  Distributed configuration files, with all defaults. Do not modify these, as they are overwritten when you upgrade.
  At IFH: /usr/pack/spamassassin-2.60-ds/share/spamassassin/*.cf
  
• /etc/mail/spamassassin/*.cf:
  Site config files, for system admins to create, modify, and add local rules and scores to. Modifications here will be appended to the config loaded from the above directory.
  We do not have it at IFH
  
• /usr/share/spamassassin/user_prefs.template:
  Distributed default user preferences. Do not modify this, as it is overwritten when you upgrade.
  At IFH it is in /usr/pack/spamassassin-2.60-ds/share/spamassassin
  You can copy it into your directory and edit it:
  cd ; cp /usr/pack/spamassassin-2.60-ds/share/spamassassin/user_prefs.template ~/.spamassassin/user_prefs
  
• /etc/mail/spamassassin/user_prefs.template:
  Default user preferences, for system admins to create, modify, and set defaults for users' preferences files. Takes precedence over the above prefs file, if it exists.
  At IFH we do not have it.
  Do not put system-wide settings in here; put them in the /etc/mail/spamassassin directory. This file is just a template, which will be copied to a user's home directory for them to change.
  
• $USER_HOME/.spamassassin:
  User state directory. Used to hold spamassassin state, such as a per-user automatic whitelist, and the user's preferences file.
  
• $USER_HOME/.spamassassin/user_prefs:
  User preferences file. If it does not exist, one of the default prefs file from above will be copied here for the user to edit later, if they wish.

  Unless you're using spamd, there is no difference in interpretation between the rules file and the preferences file, so users can add new rules for their own use in the "~/.spamassassin/user_prefs" file, if they like. (spamd disables this for security and increased speed.)

• $USER_HOME/.spamassassin/bayes*
  Statistics databases used for Bayesian filtering. If they do not exist, they will be created by SpamAssassin.
  Spamd users may wish to create a shared set of bayes databases; the "bayes_path" and "bayes_file_mode" configuration settings can be used to do this.
  See "perldoc sa-learn" for more documentation on how to train this.
  At IFH use: man sa-learn

File Locations:
SpamAssassin will look in a number of areas to find the default configuration files that are used. The "__*__" text are variables which get set during installation. You can see their values by viewing the first several lines of the "spamassassin" or "spamd" scripts.

Distributed Configuration Files	IFH Configuration files
'__def_rules_dir__' '__prefix__/share/spamassassin' '/usr/local/share/spamassassin' '/usr/share/spamassassin'	/usr/pack/spamassassin-2.60-ds/share/spamassassin /usr/pack/spamassassin-2.60-ds/share/spamassassin /usr/pack/spamassassin-2.60-ds/etc/mail/spamassassin /usr/pack/spamassassin-2.60-ds/etc/mail/spamassassin
Site Configuration Files	IFH Configuration files
'__local_rules_dir__' '__prefix__/etc/mail/spamassassin' '__prefix__/etc/spamassassin' '/usr/local/etc/spamassassin' '/usr/pkg/etc/spamassassin' '/usr/etc/spamassassin' '/etc/mail/spamassassin' '/etc/spamassassin'
Default User Preferences File	IFH Default user Preferences
'__local_rules_dir__/user_prefs.template' '__prefix__/etc/mail/spamassassin/user_prefs.template' '__prefix__/share/spamassassin/user_prefs.template' '/etc/spamassassin/user_prefs.template' '/etc/mail/spamassassin/user_prefs.template' '/usr/local/share/spamassassin/user_prefs.template' '/usr/share/spamassassin/user_prefs.template'

After installation, try "perldoc Mail::SpamAssassin::Conf" to see what can be set.
Common first-time tweaks include:

• required_hits
   
  Set this higher to make SpamAssassin less sensitive.
  If you are installing SpamAssassin system-wide, this is **strongly** recommended!

  Statistics on how many false positives to expect at various different thresholds are available in the "STATISTICS.txt" file in the "rules" directory.
  

• subject_tag
   
  When rewrite_subject is on, the subject stamp is *****SPAM*****.
  This can be used to change it.
   
• ok_locales
   
  If you expect to receive mail in non-ISO-8859 character sets (ie. Chinese, Cyrillic, Japanese, Korean, or Thai) then set this.

Automatic Whitelist System

SpamAssassin includes automatic whitelisting; The current iteration is considerably more complex than the original version. The way it works is by tracking for each sender address the average score of messages so far seen from there. Then, it combines this long-term average score for the sender with the score for the particular message being evaluated, after all other rules have been applied.
This functionality is off by default, and is enabled with the "-a" flag to either spamassassin or spamd.
A system-wide auto-whitelist can be used, by setting the auto_whitelist_path and auto_whitelist_file_mode configuration commands appropriately, e.g.


    auto_whitelist_path        /var/spool/spamassassin/auto-whitelist
    auto_whitelist_file_mode   0666


The spamassassin -W and -R command line flags provide an API to add and remove entries 'manually', if you so desire. They operate based on an input mail message, to allow them to be set up as aliases which users can simply forward their mails to. See the spamassassin manual page for more details.