Blob
- Date:
- Message:
- parallelize exporting mexp is the slowest part of the pipeline. Try to speed it up by paralellizing it. this adds a `pe' scripts that sits in front on N mexp childs and dispatch threads to them, while still outputting the unchanged lines for mkindex. Unscientific testing has shown that this effectively reduces the time for a full export, even if not linearly. (4 jobs cut the time in half, 8 jobs was just barely faster)
- Actions:
- History | Blame | Raw File
1 ENV = MBLAZE=.mblaze OUTDIR='${OUTDIR}'2 MBLAZE_PAGER = cat3 MDIR = ${HOME}/Maildir/op/GoT4 OUTDIR = www6 .PHONY: all gzip dirs assets clean8 all: .mblaze dirs assets9 mlist '${MDIR}' | mthread -r | \10 ${ENV} mscan -f '%R %I %i %16D <%64f> %128S' | \11 ${ENV} ./pe | ${ENV} ./mkindex13 gzip:14 gzip -fkr ${OUTDIR}/16 dirs:17 mkdir -p ${OUTDIR}/mail/18 mkdir -p ${OUTDIR}/mbox/19 mkdir -p ${OUTDIR}/parts/20 mkdir -p ${OUTDIR}/text/21 mkdir -p ${OUTDIR}/thread/23 assets: dirs got.png style.css24 cp got.png ${OUTDIR}/got@2x.png25 convert got.png -resize 200x200 ${OUTDIR}/got.png26 convert got.png -resize 128x128 ${OUTDIR}/got-tiny@2x.png27 convert got.png -resize 64x64 ${OUTDIR}/got-tiny.png28 cp style.css ${OUTDIR}30 ${OUTDIR}:31 mkdir -p '${OUTDIR}'33 .mblaze:34 mkdir -p .mblaze35 touch .mblaze/seq37 clean:38 rm -rf ${OUTDIR}