1 f3a795ae 2021-08-03 op I just switched my mailserver from a setup with a single UNIX user to a slightly more complex one with virtual users. I don’t know how other admins manages their virtual users, but in this entry I’m going to discuss the method I’m using.
3 f3a795ae 2021-08-03 op This is *not* a tutorial on how to install and configure OpenSMTPD or Dovecot or anything else, as I don’t feel like I’m the most qualified to do so. Instead, if you’re looking on how to deploy your own mail server, I’m going to recommend the tutorial from Gilles Chehade:
5 f3a795ae 2021-08-03 op => https://poolp.org/posts/2019-09-14/setting-up-a-mail-server-with-opensmtpd-dovecot-and-rspamd/index.html Setting up a mail server with OpenSMTPD, Dovecot and Rspamd
7 f3a795ae 2021-08-03 op In the past I’ve used a shared SQLite database to store users authentication data, but this time I wanted to manage the data differently. I don’t need to handle hundreds of users, and every user needs to be manually added by me, so a database is overkill.
9 f3a795ae 2021-08-03 op A more text-centric approach requires five configuration files:
10 f3a795ae 2021-08-03 op * a passwd-like file
11 f3a795ae 2021-08-03 op * an aliases table
12 f3a795ae 2021-08-03 op * a domains table
13 f3a795ae 2021-08-03 op * an authentication table
14 f3a795ae 2021-08-03 op * a virtuals table
16 f3a795ae 2021-08-03 op The tables are needed to load data into OpenSMTPD, while for Dovecot a single ‘/etc/passwd’-like file is enough.
18 f3a795ae 2021-08-03 op Keeping the information in sync between these five files definitely not hard, but I’m particularly lazy, so I’ve wrote a simple AWK script to parse a custom ‘userdb’ file and populate all those files. But before going into that, let’s see an excerpt from my OpenSMTPD configuration:
21 f3a795ae 2021-08-03 op # these are the paths on a FreeBSD host, on OpenBSD they’re
22 f3a795ae 2021-08-03 op # just /etc/mail.
23 f3a795ae 2021-08-03 op table aliases file:/usr/local/etc/mail/aliases
24 f3a795ae 2021-08-03 op table domains file:/usr/local/etc/mail/domains
25 f3a795ae 2021-08-03 op table passwd file:/usr/local/etc/mail/passwd
26 f3a795ae 2021-08-03 op table virtuals file:/usr/local/etc/mail/virtuals
28 f3a795ae 2021-08-03 op # pki, filters and listen directives omitted
30 f3a795ae 2021-08-03 op action "remote_mail" lmtp "/var/run/dovecot/lmtp" rcpt-to virtual <virtuals>
31 f3a795ae 2021-08-03 op action "local_mail" lmtp "/var/run/dovecot/lmtp" rcpt-to alias <aliases>
32 f3a795ae 2021-08-03 op action "outbound" relay helo example.com
34 f3a795ae 2021-08-03 op match from any for domain <domains> action "remote_mail"
35 f3a795ae 2021-08-03 op match from local for local action "local_mail"
36 f3a795ae 2021-08-03 op match from any auth for any action "outbound"
37 f3a795ae 2021-08-03 op match for any action "outbound"
40 f3a795ae 2021-08-03 op The four ‘match’ rules matches in order
41 f3a795ae 2021-08-03 op * incoming emails for the domains we’re serving
42 f3a795ae 2021-08-03 op * local emails from one UNIX user to another
43 f3a795ae 2021-08-03 op * outgoing emails from authenticated users
44 f3a795ae 2021-08-03 op * outgoing emails from local UNIX users (there’s an implicit ‘from local’, to turn your server into an open relay you need to be really, really explicit!)
46 f3a795ae 2021-08-03 op Two of the three actions deliver the mail over LMTP to Dovecot. An important bit there that I was missing on my first try was the ‘rcpt-to’ keyword: as we’ll see in a moment, all the mail are handled by a local user, but we need to use the recipient email address instead of the local user in the LMTP session, so Dovecot can save the email in the correct maildir.
50 f3a795ae 2021-08-03 op Dovecot needs only a single file for the authentication. One of the supported format, and the one I’m using, is a ‘passwd’-like format, like the following:
53 f3a795ae 2021-08-03 op op@example.com:<hash>::::::
56 f3a795ae 2021-08-03 op On the Dovecot site, things are a bit easier because there is no aliasing, resolving or expansions to do on the received emails.
58 f3a795ae 2021-08-03 op ### alias table
60 f3a795ae 2021-08-03 op An alias table looks like this:
62 f3a795ae 2021-08-03 op ```example of an alias table file
64 f3a795ae 2021-08-03 op op: op@example.com
67 f3a795ae 2021-08-03 op It maps *local* users to other local or remote users. In the example above, mail for the UNIX root user are forwarded to the user op, that in turns redirects his mail to op@example.com.
69 f3a795ae 2021-08-03 op ### domains table
71 f3a795ae 2021-08-03 op Holds all the domains we’re accepting mails from. It can be specified in-line in the configuration file:
73 f3a795ae 2021-08-03 op ```example of a domains table in smtpd.conf
74 f3a795ae 2021-08-03 op table domains { "example.com", "foo.bar.net", … }
77 f3a795ae 2021-08-03 op or in a file with one domain name per line
79 f3a795ae 2021-08-03 op ```example of a domains table as plain file
84 f3a795ae 2021-08-03 op ### Credentials table
86 f3a795ae 2021-08-03 op A credentials table file looks like this:
88 f3a795ae 2021-08-03 op ```example of a credentials table
89 f3a795ae 2021-08-03 op user@doma.in password-hash
90 f3a795ae 2021-08-03 op user2@example.com password-hash
93 f3a795ae 2021-08-03 op just a simple user ↔ hash mapping. Hashes can be computed with the encrypt subcommand of smtpctl
95 f3a795ae 2021-08-03 op ```example on how to hash a password
96 f3a795ae 2021-08-03 op $ smtpctl encrypt
98 f3a795ae 2021-08-03 op $2b$10$jpdOj8WPIMABsMs.LzFbiuSpgZ1TlGUj2ztBxEimoaQylQD/jhelS
102 f3a795ae 2021-08-03 op NB: on OpenBSD-CURRENT (and as of a couple of releases already at least) the ‘smtpctl encrypt’ computes the BLF-CRYPT hash of the password, but for some reason on FreeBSD it uses SHA512-CRYPT. Dovecot needs to be told the default hashing scheme in ‘conf.d/auth-passwdfile.conf.ext’. Here’s mine
104 f3a795ae 2021-08-03 op ``` example of auth-passwdfile.conf.ext file for Dovecot
106 f3a795ae 2021-08-03 op driver = passwd-file
107 f3a795ae 2021-08-03 op # adjust SHA512-CRYPT eventually!
108 f3a795ae 2021-08-03 op args = scheme=SHA512-CRYPT username_format=%u /usr/local/etc/dovecot/users
112 f3a795ae 2021-08-03 op driver = passwd-file
113 f3a795ae 2021-08-03 op args = username_format=%u /usr/local/etc/dovecot/users
114 f3a795ae 2021-08-03 op override_fields = home=/var/vmail/%d/%n
118 f3a795ae 2021-08-03 op Refer to the Dovecot documentation:
119 f3a795ae 2021-08-03 op => https://doc.dovecot.org/configuration_manual/authentication/password_schemes/ “Password Schemes” in the Dovecot documentation.
123 f3a795ae 2021-08-03 op The virtual table is used to map address to other addresses (i.e. alias) or addresses to local users (to allow the delivery.) It looks like this
125 f3a795ae 2021-08-03 op ```example of a virtuals table
126 f3a795ae 2021-08-03 op postmaster@example.com: op@example.com
127 f3a795ae 2021-08-03 op aaa@example.com: op@example.com
128 f3a795ae 2021-08-03 op op@example.com: vmail
129 f3a795ae 2021-08-03 op otheruser@example.com: vmail
132 f3a795ae 2021-08-03 op ## Making it painless
134 f3a795ae 2021-08-03 op Since maintaining this whole bunch of files may not be the easiest thing ever. To be a bit more declarative, I’ve come up with the following ‘userdb’ file. It’s an invented syntax that gets parsed by a super-simple AWK script and generates all the other files. Here’s an example:
136 f3a795ae 2021-08-03 op ```example of syntax of userdb
137 f3a795ae 2021-08-03 op # local alias
138 f3a795ae 2021-08-03 op alias root op
139 f3a795ae 2021-08-03 op alias op op@example.com
141 f3a795ae 2021-08-03 op # per virtual-domain config
143 f3a795ae 2021-08-03 op # Indentation is optional, but improves legibility.
144 f3a795ae 2021-08-03 op # The following defines the user op@example.com;
145 f3a795ae 2021-08-03 op # <hash> is the hash of the password computed
146 f3a795ae 2021-08-03 op # with `smtpctl encrypt`
147 f3a795ae 2021-08-03 op user op <hash>
148 f3a795ae 2021-08-03 op # and define an arbitrary number of aliases
149 f3a795ae 2021-08-03 op alias service1
150 f3a795ae 2021-08-03 op alias other-alias
152 f3a795ae 2021-08-03 op user otheruser <hash>
154 f3a795ae 2021-08-03 op # aliases can be to virtual users on other hosts
155 f3a795ae 2021-08-03 op alias abuse someone@example2.com
157 f3a795ae 2021-08-03 op example2.com:
158 f3a795ae 2021-08-03 op user someone <hash>
162 f3a795ae 2021-08-03 op The syntax is as simple as possible, to make the parsing easier. It’s also open for additions: for instance, adding a ‘quota’ keyword to define custom quotas shouldn’t be too hard.
164 f3a795ae 2021-08-03 op => //git.omarpolo.com/vuserctl/ All the code examples are available in a git repository.
166 f3a795ae 2021-08-03 op The AWK implementation that parses the file is also pretty simple:
168 f3a795ae 2021-08-03 op ``` userctl.awk
169 f3a795ae 2021-08-03 op #!/usr/bin/env awk
171 f3a795ae 2021-08-03 op # expects action to be defined, like -v action=aliases
173 f3a795ae 2021-08-03 op /^[[:space:]]*$/ { next }
174 f3a795ae 2021-08-03 op /^[[:space:]]*#/ { next }
178 f3a795ae 2021-08-03 op gsub(":", "", $1);
180 f3a795ae 2021-08-03 op domains[domainslen++] = domain;
184 f3a795ae 2021-08-03 op $1 == "user" {
185 f3a795ae 2021-08-03 op user = sprintf("%s@%s", $2, domain);
186 f3a795ae 2021-08-03 op users[user] = $3
188 f3a795ae 2021-08-03 op # change “vmail” to match the local user that
189 f3a795ae 2021-08-03 op # delivers the mail
190 f3a795ae 2021-08-03 op aliases[user] = "vmail";
194 f3a795ae 2021-08-03 op $1 == "alias" {
195 f3a795ae 2021-08-03 op if ($3 != "") {
198 f3a795ae 2021-08-03 op target = user;
201 f3a795ae 2021-08-03 op if (domain != "") {
202 f3a795ae 2021-08-03 op alias = sprintf("%s@%s", $2, domain);
206 f3a795ae 2021-08-03 op aliases[alias] = target;
209 f3a795ae 2021-08-03 op # output in the correct format
211 f3a795ae 2021-08-03 op if (action == "aliases") {
212 f3a795ae 2021-08-03 op for (alias in aliases) {
213 f3a795ae 2021-08-03 op if (match(alias, "@"))
215 f3a795ae 2021-08-03 op printf("%s: %s\n", alias, aliases[alias]);
217 f3a795ae 2021-08-03 op } else if (action == "virtuals") {
218 f3a795ae 2021-08-03 op for (alias in aliases) {
219 f3a795ae 2021-08-03 op if (!match(alias, "@"))
221 f3a795ae 2021-08-03 op printf("%s %s\n", alias, aliases[alias]);
223 f3a795ae 2021-08-03 op } else if (action == "domains") {
224 f3a795ae 2021-08-03 op for (domain in domains) {
225 f3a795ae 2021-08-03 op printf("%s\n", domains[domain]);
227 f3a795ae 2021-08-03 op } else if (action == "users") {
228 f3a795ae 2021-08-03 op for (user in users) {
229 f3a795ae 2021-08-03 op printf("%s %s\n", user, users[user]);
231 f3a795ae 2021-08-03 op } else if (action == "users.passwd") {
232 f3a795ae 2021-08-03 op for (user in users) {
233 f3a795ae 2021-08-03 op # user@doma.in:hash::::::
234 f3a795ae 2021-08-03 op # user@doma.in:hash::::::userdb_quota_rule=*:storage=1G
235 f3a795ae 2021-08-03 op printf("%s:%s::::::\n", user, users[user]);
237 f3a795ae 2021-08-03 op } else if (action == "users.mdirs") {
238 f3a795ae 2021-08-03 op for (user in users) {
239 f3a795ae 2021-08-03 op split(user, m, "@");
240 f3a795ae 2021-08-03 op # adjust the maildir path
241 f3a795ae 2021-08-03 op printf("/var/vmail/%s/%s/Maildir\n", m[2], m[1]);
244 f3a795ae 2021-08-03 op print "unknown action!\n" > "/dev/stderr"
250 f3a795ae 2021-08-03 op The AWK script needs the variable ‘action’ to be defined to dump the correct information. It can be provided with the ‘-v’ flag, but for extra-comfort I wrote also the following wrapper script:
252 f3a795ae 2021-08-03 op ```userctl the wrapper script
255 f3a795ae 2021-08-03 op if [ ! -f "userctl.awk" ]; then
256 f3a795ae 2021-08-03 op echo "Can't find userctl.awk!" >&2
260 f3a795ae 2021-08-03 op if [ ! -f "userdb" ]; then
261 f3a795ae 2021-08-03 op echo "Can't find userdb!" >&2
265 f3a795ae 2021-08-03 op # run <action>
268 f3a795ae 2021-08-03 op awk -f userctl.awk -v action="$1" userdb
272 f3a795ae 2021-08-03 op aliases) run "aliases" ;;
273 f3a795ae 2021-08-03 op virtuals) run "virtuals" ;;
274 f3a795ae 2021-08-03 op domains) run "domains" ;;
275 f3a795ae 2021-08-03 op users) run "users" ;;
276 f3a795ae 2021-08-03 op users.passwd) run "users.passwd" ;;
277 f3a795ae 2021-08-03 op users.mdirs) run "users.mdirs" ;;
279 f3a795ae 2021-08-03 op echo "USAGE: $0 <action>"
280 f3a795ae 2021-08-03 op echo "where action is one of"
281 f3a795ae 2021-08-03 op echo " - aliases"
282 f3a795ae 2021-08-03 op echo " - virtuals"
283 f3a795ae 2021-08-03 op echo " - domains"
284 f3a795ae 2021-08-03 op echo " - users"
285 f3a795ae 2021-08-03 op echo " - users.passwd"
286 f3a795ae 2021-08-03 op echo " - users.mdirs"
289 f3a795ae 2021-08-03 op echo "Unknown action $1" >&2
295 f3a795ae 2021-08-03 op Now that the framework is in place, the only missing piece is to use it to generate the files. I wrote yet another script to (re-)generate the tables and to create the maildir when a user is added.
297 f3a795ae 2021-08-03 op ```sync-userdb
302 f3a795ae 2021-08-03 op # On OpenBSD these are only /etc/mail/…
303 f3a795ae 2021-08-03 op ./userctl aliases > /usr/local/etc/mail/aliases
304 f3a795ae 2021-08-03 op ./userctl virtuals > /usr/local/etc/mail/virtuals
305 f3a795ae 2021-08-03 op ./userctl domains > /usr/local/etc/mail/domains
306 f3a795ae 2021-08-03 op ./userctl users > /usr/local/etc/mail/passwd
308 f3a795ae 2021-08-03 op ./userctl users.passwd > /usr/local/etc/dovecot/users
312 f3a795ae 2021-08-03 op if [ ! -d "$1" ]; then
314 f3a795ae 2021-08-03 op chown vmail:vmail "$1"
318 f3a795ae 2021-08-03 op # ensure the maildirs exists
319 f3a795ae 2021-08-03 op for dir in $(./userctl users.mdirs); do
320 f3a795ae 2021-08-03 op homedir=$(dirname "$dir")
321 f3a795ae 2021-08-03 op domdir=$(dirname "$homedir")
328 f3a795ae 2021-08-03 op # eventually add something like
329 f3a795ae 2021-08-03 op # service dovecot restart
330 f3a795ae 2021-08-03 op # service smtpd restart
331 f3a795ae 2021-08-03 op # for FreeBSD or
332 f3a795ae 2021-08-03 op # rcctl restart dovecot smtpd
333 f3a795ae 2021-08-03 op # for OpenBSD.
336 f3a795ae 2021-08-03 op ## Conclusion
338 f3a795ae 2021-08-03 op I don’t have a proper conclusion for this entry. Tools like this are usually almost always “work in progress”, as they are changed/extended over the time depending on what I need to do. One thing for sure, designing simple database files and managing them with AWK is lots of fun.
340 f3a795ae 2021-08-03 op As always, if you have comment, tips or noticed something that’s missing or not explained properly, don’t refrain from notifying me, so I can update this entry accordingly.
