Setup of the mailing lists

postfix

main.cf

# for administration of mlmmj
recipient_delimiter = +

# to talk to rspamd for dmarc munging

smtpd_milters=inet:127.0.0.1:11332
non_smtpd_milters=inet:127.0.0.1:11332

# to map the ml with the proper transport
virtual_alias_maps = hash:/usr/local/mlmmj/virtual
transport_maps = hash:/usr/local/mlmmj/transport
propagate_unmatched_extensions = virtual

master.cf

mlmmj    unix  -       n       n       -       -       pipe
    flags=ORhu user=mailman argv=/usr/local/bin/mlmmj-receive -F -L /usr/local/mlmmj/lists/$nexthop

rspamd

/usr/local/etc/rspamd/local.d/dmarc.conf

munging {
        list_map =  "/usr/local/mlmmj/rspamd-munging";
        mitigate_strict_only = true;
}

mlmmj

Layout

/usr/local/mlmmj/bin

Home made scripts

/usr/local/mlmmj/lists

Directory containing all the mailing lists

/usr/local/mlmmj/www

static html assets

/usr/local/mlmmj/mboxes

list of mboxes corresponding to archive backups, if archives should be rebuilt they will be rebuilt from there

/usr/local/mlmmj/archived/mboxes

mboxes for archived mailing lists

/usr/local/mlmmj/archived/archives

html generated mailing archive for archived mailing lists

/usr/local/mlmmj/tmp

directory where the subscriptions will be handled

/usr/local/mlmmj/virtual*

virtual mapping of mailing lists email and internal email for postfix (run postmap on it once modified) should contain 1 entry per valid email for a given mailing list

/usr/local/mlmmj/transport*

mapping between internal email and transport for postfix) run porstmap on it once modified should contain 1 entry per mailing list

/usr/local/mlmmj/webarchive

Directory where the html archives will be generated

/usr/local/mlmmj/rspamd-munging

list of mailing list to activate munging on, one per line

maintenance (bounce etc)

the mlmmj-maintd daemon is run every 5 minutes to handle all the maintenance via crontab

crontabs

$ cat /usr/local/etc/cron.d/mlmmj
SHELL=/bin/sh
PATH=/bin:/usr/bin:/usr/local/bin
*/5     *       *       *       *       mailman /usr/local/mlmmj/bin/webarchives.sh
*/2     *       *       *       *       mailman /usr/local/mlmmj/bin/mlsubscriptions.sh
*/5     *       *       *       *       mailman /usr/local/bin/mlmmj-maintd -F -d /usr/local/mlmmj/lists

archives

Archives are run via hypermail if a mailing lists have at its root directory a file named noarchive, then the incremental upgrade will stop running The configuration is done in hmrc

/usr/local/etc/hmrc

folder_by_date = "%Y-%B"
usegdbm = "1"
indextable = "0"
reverse = "1"
reverse_folders = 1
describe_folder = "%B %Y"
require_msgids = 0
discard_dup_msgids = 0
gmtime = 1
monthly_index = 1
message_pattern = %.6d
file_by_thread = true
href_detection = true
thrdlevels = 2
monthly_index = 1
yearly_index = 1
thread_file_depth = 1
append=1

ihtmlheaderfile=        /usr/local/etc/hypermail/index-header.hyp
mhtmlheaderfile=        /usr/local/etc/hypermail/msg-header.hyp
ihtmlfooterfile=        /usr/local/etc/hypermail/index-footer.hyp
mhtmlfooterfile=        /usr/local/etc/hypermail/msg-footer.hyp

Regenerating from scratch the archives

/!\ Do this as user mailman /!\

$ hypermail -c /usr/local/etc/hmrc -o append=0 -a "/index" -m /usr/local/mlmmj/mboxes/freebsd-test.mbox -l "freebsd-test: Test posting area" -u -x -d /usr/local/mlmmj/webarchive/freebsd-test

Incremental updates

The incremental updates are performed by crontab via the webarchives.sh script

It will take all untreated files from /usr/local/mlmmj/lists/ml/archive and pass them through mexport (from the mblaze package) to generate a mbox formatted output which will be piped to hypermail

The untreated files are the files which filename is a number higer then the last number recorded in the lastindex file

As a result the mbox of the mailing will be prepended with the new mails and the html archives updated

The script runs every 5 minutes

mlmmj-webview

Small CGI to handle subscription and links to archive It should run as user ***nobody***

It takes /usr/local/mlmmj/list as an entry file to determine what it should show to the user

The subscription will be written in files in /var/spool/mlmmj-webview/subscribe The unsubscription will be written in files in /var/spool/mlmmj-webview/unscubsribe

a crontab will every 2 minutes look at new (un)subscriptions here and dedup them and proceed The script being /usr/local/mlmmj/bin/mlsubscriptions.sh

Tools

list_members [mailinglist]

list the members of a given mailing list

list_lists

list the public mailing lists

mkml.sh -n name -d description

create a new mailing list

webarchives.sh

incrementally update the web archives and mbox

mlsubscriptions.sh

deal with pending (un)subscriptions

msgconv.sh

lookup all html files in webarchive and try to convert to unicode what can be converted to it

Teams/clusteradm/MailingLists (last edited 2021-05-12T00:40:33+0000 by DanLangille)