System monitoring

Ok, my Raspberry Pi VPN router kept crashing (spoiler: it was both overheating and failing to load iptables rules) so I needed to try to monitor it to diagnose it. Then I went into a system monitoring rabbit hole.

Time series databases such as RRD are probably handy. RRD looks simple enough. You just run a job that measures something at intervals and it records it for you. The tides of hype have surfaced some more fancy and/or turnkey solutions.

netdata has been recommended to me as something with native Raspberry Pi support (i.e. it looks for overheating). Open source.

There are more elaborate solutions targeting big infrastructure and networks in particular such as icinga and opennms, but I have not used these as RRD ended up being sufficient.

If I wanted to monitor only server traffic logs, I could use goaccess.