Spamassassin Creating Bayes_toks.expire ... Files

[likoma]

I was cleaning up a domain (TCH: if I should start a ticket for this and tell you which domain, I'd be happy to.) and found that the Spamassassin folder had 13 MB in it. I found some "bayes_toks.expire$$$$" files and had a search around and found this post on another host's boards about it.

 

My question: are these expire files necessary? Can I delete them?

 

Thanks,

 

- Bradley

 

A few days ago I reported a problem I was having with my bayes

database to the SATalk mailing list along with the observation that I

was pretty sure it was a bug in the bayes expiry software.

 

The problem was that under heavy load "bayes_toks.expire[pid]" files

were piling up in the bayes database area as seen in the following

partial "ls -l" listing

 

  32 Jan 26 12:19 bayes.lock

  2750039 Jan 26 12:27 bayes_journal

  20897792 Jan 26 12:19 bayes_seen

  21733376 Jan 26 12:22 bayes_toks

  9437184 Jan 26 12:22 bayes_toks.expire16781

  11173888 Jan 26 11:35 bayes_toks.expire23012

  5341184 Jan 26 10:54 bayes_toks.expire27549

  11182080 Jan 26 11:59 bayes_toks.expire27570

  11403264 Jan 26 10:44 bayes_toks.expire4752

 

A few days later I got a message from another person, David Lee, who

had run into the same problem and who thought it might be due to the

controlling agent, in his case a program called Mailscanner, timing

out the expiry process before it could complete.

 

 

 

It turns out that is exactly what was happening (I think). Bayes

expiry can often take 3 or 4 minutes to complete, and if the system

load happens to be really high when a mimedefang/spamassassin process

decides its time to do an expiry, the process can easily take much

longer, and if it takes longer than 5 minutes your're in trouble,

since AFAIK the sendmail default timeout on a milter operation

is 5 minutes. At least that's how I interpret the following info

in the Mimedefang HowTO document

 

  E      Overall timeout between sending end-of-message to filter

        and waiting for the final acknowledgment

 

  Note the separator between each is a ';' as a ',' already separates

  equates and therefore can't separate timeouts.  The default values (if

  not set in the config) are:

 

  T=C:5m;S:10s;R:10s;E:5m

 

  where 's' is seconds and 'm' is minutes.

 

Have I got that right? If a bayes expiry takes longer than 5 minutes

it will be abruptly terminated? Sure fits the observed phenomenon.

 

I'm also pretty sure this must be the case because I copied the files

to another location for testing and ran an expire via sa-learn and it

finished successfully in about 8 minutes, so it wasn't a matter of a

corrupted database causing the problem.

 

The point is that allowing bayes expiry to take place opportunisticaly

on a heavily loaded high volume site is a recipe for disaster, and what

you need to do is set "bayes_auto_expire 0" in your sa-mimedefang.cf or

local.cf file and use sa-learn to force an expire on a regular basis

via cron.