Blame


1 800cf002 2022-08-29 op GOTMARC -- A web archive for the Game of Trees mailing list
2 800cf002 2022-08-29 op ===========================================================
3 b2508be5 2022-08-24 op
4 b2508be5 2022-08-24 op A collection of scripts to generate a static mail archive from a
5 b2508be5 2022-08-24 op Maildir; intended to provide a public web interface for the Game of
6 b2508be5 2022-08-24 op Trees mailing list.
7 b2508be5 2022-08-24 op
8 1efe003f 2022-08-29 op The only dependency outside base is mblaze.
9 b2508be5 2022-08-24 op
10 b2508be5 2022-08-24 op
11 b2508be5 2022-08-24 op How it works
12 b2508be5 2022-08-24 op ------------
13 b2508be5 2022-08-24 op
14 7067473a 2022-08-24 op The makefile uses mlist(1) to gather all the mail of the mailing list,
15 7067473a 2022-08-24 op mthread(1) for threading and mscan(1) for parsing.
16 b2508be5 2022-08-24 op
17 1c159d06 2022-08-26 op Then, three custom scripts are called:
18 b2508be5 2022-08-24 op
19 b2508be5 2022-08-24 op - mexp: creates a separate HTML page for each mail, as well as the
20 7067473a 2022-08-24 op raw message/body parts and the attachments. It's the heaviest part
21 7067473a 2022-08-24 op of the pipeline. If a message appears to have already been exported,
22 7067473a 2022-08-24 op it will be skipped to save some time.
23 b2508be5 2022-08-24 op
24 57784939 2022-08-25 op - pe: (parallelized export) since mexp is slow, attemps to run N mexp
25 57784939 2022-08-25 op processes in parallel and dispatch threads to them.
26 57784939 2022-08-25 op
27 b2508be5 2022-08-24 op - mkindex: generates the index and threads page.
28 b2508be5 2022-08-24 op
29 b2508be5 2022-08-24 op
30 b2508be5 2022-08-24 op Usage
31 b2508be5 2022-08-24 op -----
32 b2508be5 2022-08-24 op
33 b2508be5 2022-08-24 op $ make MDIR=/path/to/got/maildir
34 b2508be5 2022-08-24 op
35 19a4c72c 2022-08-26 op will create and populate a `/var/www/marc' directory with the HTML
36 19a4c72c 2022-08-26 op files. Pass OUTDIR to override the destination directory. MDIR
37 19a4c72c 2022-08-26 op defaults to ~/Mail/gameoftrees.
38 b2508be5 2022-08-24 op
39 408edb25 2022-08-26 op There's some basic support for parallelizing the export, although it's
40 1c159d06 2022-08-26 op disabled by default. Tweak MAKE_JOBS for `pe' to use multiple jobs.
41 1c159d06 2022-08-26 op
42 5f54ec1a 2022-08-24 op Every once in a while, the maildir should be synced (with mbsync for
43 5f54ec1a 2022-08-24 op example) and `make' re-run to update the archive. gotmarc does
44 5f54ec1a 2022-08-24 op incremental updates to some extent so except for the first run, which
45 5f54ec1a 2022-08-24 op will take a while, updating the archive should be fast.
46 5f54ec1a 2022-08-24 op
47 b2508be5 2022-08-24 op For use with httpd' `gzip-static' (or similar)
48 b2508be5 2022-08-24 op
49 b2508be5 2022-08-24 op $ make gzip
50 b2508be5 2022-08-24 op
51 b2508be5 2022-08-24 op will recursively compress every file in OUTDIR, leaving the original
52 b2508be5 2022-08-24 op file intact.
53 2215cb0b 2022-08-24 op
54 2215cb0b 2022-08-24 op
55 2215cb0b 2022-08-24 op License
56 2215cb0b 2022-08-24 op -------
57 2215cb0b 2022-08-24 op
58 2215cb0b 2022-08-24 op As far as I'm concerned, gotmarc (i.e. all the code in this
59 2215cb0b 2022-08-24 op repository) is in the public domain.