Tree
Tree:
338373325d487acfd724acee6217b3ce0a007a71
Date:
Thu Aug 25 08:58:12 2022
UTC
Message:
parallelize exporting
mexp is the slowest part of the pipeline. Try to speed it up by
paralellizing it.
this adds a `pe' scripts that sits in front on N mexp childs and
dispatch threads to them, while still outputting the unchanged lines
for mkindex.
Unscientific testing has shown that this effectively reduces the time
for a full export, even if not linearly. (4 jobs cut the time in half,
8 jobs was just barely faster)
Omar Polo