Setup of the mailing lists
postfix
main.cf
# for administration of mlmmj recipient_delimiter = + # to talk to rspamd for dmarc munging smtpd_milters=inet:127.0.0.1:11332 non_smtpd_milters=inet:127.0.0.1:11332 # to map the ml with the proper transport virtual_alias_maps = hash:/usr/local/mlmmj/virtual transport_maps = hash:/usr/local/mlmmj/transport propagate_unmatched_extensions = virtual
master.cf
mlmmj unix - n n - - pipe flags=ORhu user=mailman argv=/usr/local/bin/mlmmj-receive -F -L /usr/local/mlmmj/lists/$nexthop
rspamd
/usr/local/etc/rspamd/local.d/dmarc.conf
munging { list_map = "/usr/local/mlmmj/rspamd-munging"; mitigate_strict_only = true; }
mlmmj
Layout
/usr/local/mlmmj/bin |
Home made scripts |
/usr/local/mlmmj/lists |
Directory containing all the mailing lists |
/usr/local/mlmmj/www |
static html assets |
/usr/local/mlmmj/mboxes |
list of mboxes corresponding to archive backups, if archives should be rebuilt they will be rebuilt from there |
/usr/local/mlmmj/archived/mboxes |
mboxes for archived mailing lists |
/usr/local/mlmmj/archived/archives |
html generated mailing archive for archived mailing lists |
/usr/local/mlmmj/tmp |
directory where the subscriptions will be handled |
/usr/local/mlmmj/virtual* |
virtual mapping of mailing lists email and internal email for postfix (run postmap on it once modified) should contain 1 entry per valid email for a given mailing list |
/usr/local/mlmmj/transport* |
mapping between internal email and transport for postfix) run porstmap on it once modified should contain 1 entry per mailing list |
/usr/local/mlmmj/webarchive |
Directory where the html archives will be generated |
/usr/local/mlmmj/rspamd-munging |
list of mailing list to activate munging on, one per line |
maintenance (bounce etc)
the mlmmj-maintd daemon is run every 5 minutes to handle all the maintenance via crontab
crontabs
$ cat /usr/local/etc/cron.d/mlmmj SHELL=/bin/sh PATH=/bin:/usr/bin:/usr/local/bin */5 * * * * mailman /usr/local/mlmmj/bin/webarchives.sh */2 * * * * mailman /usr/local/mlmmj/bin/mlsubscriptions.sh */5 * * * * mailman /usr/local/bin/mlmmj-maintd -F -d /usr/local/mlmmj/lists
archives
Archives are run via hypermail if a mailing lists have at its root directory a file named noarchive, then the incremental upgrade will stop running The configuration is done in hmrc
/usr/local/etc/hmrc
folder_by_date = "%Y-%B" usegdbm = "1" indextable = "0" reverse = "1" reverse_folders = 1 describe_folder = "%B %Y" require_msgids = 0 discard_dup_msgids = 0 gmtime = 1 monthly_index = 1 message_pattern = %.6d file_by_thread = true href_detection = true thrdlevels = 2 monthly_index = 1 yearly_index = 1 thread_file_depth = 1 append=1 ihtmlheaderfile= /usr/local/etc/hypermail/index-header.hyp mhtmlheaderfile= /usr/local/etc/hypermail/msg-header.hyp ihtmlfooterfile= /usr/local/etc/hypermail/index-footer.hyp mhtmlfooterfile= /usr/local/etc/hypermail/msg-footer.hyp
Regenerating from scratch the archives
Do this as user mailman
$ hypermail -c /usr/local/etc/hmrc -o append=0 -a "/index" -m /usr/local/mlmmj/mboxes/freebsd-test.mbox -l "freebsd-test: Test posting area" -u -x -d /usr/local/mlmmj/webarchive/freebsd-test
Incremental updates
The incremental updates are performed by crontab via the webarchives.sh script
It will take all untreated files from /usr/local/mlmmj/lists/ml/archive and pass them through mexport (from the mblaze package) to generate a mbox formatted output which will be piped to hypermail
The untreated files are the files which filename is a number higer then the last number recorded in the lastindex file
As a result the mbox of the mailing will be prepended with the new mails and the html archives updated
The script runs every 5 minutes
mlmmj-webview
Small CGI to handle subscription and links to archive It should run as user ***nobody***
It takes /usr/local/mlmmj/list as an entry file to determine what it should show to the user
The subscription will be written in files in /var/spool/mlmmj-webview/subscribe The unsubscription will be written in files in /var/spool/mlmmj-webview/unscubsribe
a crontab will every 2 minutes look at new (un)subscriptions here and dedup them and proceed The script being /usr/local/mlmmj/bin/mlsubscriptions.sh
Tools
list_members [mailinglist] |
list the members of a given mailing list |
list_lists |
list the public mailing lists |
mkml.sh -n name -d description |
create a new mailing list |
webarchives.sh |
incrementally update the web archives and mbox |
mlsubscriptions.sh |
deal with pending (un)subscriptions |
msgconv.sh |
lookup all html files in webarchive and try to convert to unicode what can be converted to it |