1 I have a tiny i686 at home with OpenBSD where I run, amongst some other
2 things, an instance of unbound.
4 Last night I decided that I wanted a dashboard to collect some statistics
7 My first thought was the ELK stack. The only problem is the ram.
8 The little i686 has 1GB of ram, I don't know if it's enough to run
9 logstash, let alone the whole ELK.
11 A simple solution would be to collect the logs elsewhere, but I'm not
12 going to do this for various reason (lazyness being the first, followed
13 by the fact that having statistics about my dns queries isn't that
14 useful in my opinion, even if it's nice-to-have.)
16 Instead, my solution involves a bit of bash (don't hate me on this),
17 some fifos, tmux and ttyplot.
19 The primarly source of inspiration is [this
20 post](https://dataswamp.org/~solene/2019-07-29-ttyplot.html) that I red
21 some time ago: it's about plotting various system statistics with ttyplot.
25 
27 (note that I usually disable colors in xterm)
32 / various -------> multiple
33 unbound stats ------- fifos -------> ttyplot
34 \ -------> per tmux pane
37 The idea is to run `unbound-control stats` every once in a while,
38 multiplexing its output and draw each (interesting) stats with ttyplot
41 Why the fifos? Well, if I'm not wrong, every time you call
42 `unbound-control stats` it will clear the statistics, so you can't run
43 it *n* times to collect *n* different stats. And since the whole setup
44 requires only a couple of script, the easiest way was to use some fifos.
46 The whole setup requires three script:
52 ### `gen-dashboard.sh`
54 This is the startup script. I run it on my crontab as `@reboot
55 /path/to/gen-dashboard.sh`. It will create the required fifos, then
56 spawn a tmux session and create two windows and some panes.
62 for f in netstat queries hit miss time; do
68 tmux new-session -d -s $session
71 tmux new-window -t $session:1 -n 'logs'
72 tmux send-keys "/path/to/mystatd.sh" C-m
74 # create the dashboard
75 tmux select-window -t $session:0
77 # setup the layout of the panes
82 tmux split-window -v -p 66
83 tmux split-window -v -p 50
85 # load the correct ttyplot in the panes
87 tmux send-keys "/path/to/dashboard.sh netstat" C-m
90 tmux send-keys "/path/to/dashboard.sh queries" C-m
93 tmux send-keys "/path/to/dashboard.sh hit" C-m
96 tmux send-keys "/path/to/dashboard.sh miss" C-m
99 tmux send-keys "/path/to/dashboard.sh time" C-m
102 (A possible improvement may be to tell tmux which command to run when
103 creating a pane instead of sending the keys to the shell, but it works
106 There's nothing special about this script, so let's move to the next.
110 This script also isn't interesting, all it does is pull the data out of
111 the correct fifo and start ttyplot with the correct labels and units.
117 echo "missing dashboard type"
118 echo "usage: $0 <dashboard-name>"
126 done) | ttyplot -t "IN Bandwidth in KB/s" \
133 done) | ttyplot -t "DNS Queries/5s" \
140 done) | ttyplot -t "DNS cache hit/5s" \
147 done) | ttyplot -t "DNS cache miss/5s" \
154 done) | ttyplot -t "DNS query time avg/5s" \
159 printf "%s\n" "$1 is not a valid dashboard"
167 This is the (only?) interesting script. It's also the only one that
168 requires bash, because I'm lazy, it was already installed as dependecy of
169 something, and because of the `>(cmd)` construct. Rewriting the script
170 using only pure sh(1) constructs is left as an exercise to the reader
171 (hint: you need some extra fifo.)
177 grep "$1" | awk -F= '{print $2}' > /tmp/my-$2
182 unbound-control stats \
184 | tee >(filter queries= queries) \
185 | tee >(filter hit hit) \
186 | tee >(filter miss miss) \
187 | filter time.avg time
192 # netstat - ty solene@ for the awk
202 if(!index($4,":") && old>=0) {
210 }' | tee -a /tmp/my-netstat
216 The first piece collects the stat from unbound. Let's break it in pieces.
218 - `unbound-control stats` outputs the stats. Keep in mind that this
219 requires some privileges. I've solved this by creating a script
220 in /usr/local/bin that executes the command and allowed my user to
221 launch that script via `doas(1)`. Or you can run `mystatd.sh` as root.
223 - `grep -v thread0` removes the per-thread stats (since my unbound
224 uses only one thread). A more solid approach like `egrep -v ^thread`
226 - `tee >(filter queries= queries) |` duplicates the stream: one copy
227 goes to the subshell with `filter` and another copy goes on the pipes.
228 - `filter` is just a small function to grep the desired entry and send
229 it to `/tmp/my-$something`
231 The netstat bit filters the output of netstat (the awk is copied-pasted
232 from the previously linked post by solene@). You may want to change the
233 `^em0` to match your network device.
237 ## Possible improvements
239 - if you `SIGINT` `mystatd.sh` some of its subprocess still run. Maybe a
240 `trap` is needed. Since it is the only bash running on that system,
241 `pkill bash` is, albeit a bit aggressive, a working solution.
242 - replace bash. It's not difficult, but requires more fifos.
245 ## Final considerations
247 This was fun. Now I have a tmux session I remotely attach with cool
248 graphs. This doesn't cover the archiviation of the statistics tho.
249 I think it should be trivial to add (just one more `|tee -a` to a local
250 file, maybe a cronjob to do rotation, ...) but for the moment I'm happy
