When I deploy a Linux server, usually Ubuntu, I make sure to include some changes that differs from default. The default config is fine, but since the virtual hardware my server is running on is quite limiting, I need to do some fine tuning.
So, first up, let’s benchmark the performance with the default config, then the fine tuned config.
Setup /etc/sysctl.d/99-sysctl.conf
| Setting | Default | Fine tuned |
|---|---|---|
| vm.swappiness | 60 | 10 |
| vm.dirty_writeback_centisecs | 500 | 2000 |
| net.ipv4.tcp_fastopen | 1 | 3 |
| net.ipv4.tcp_congestion_control | cubic | bbr |
| net.core.default_qdisc | fq_codel | cake |
| net.ipv4.tcp_window_scaling | 1 | 1 |
| net.ipv4.tcp_mtu_probing | 0 | 1 |
| net.ipv4.tcp_low_latency | 0 | 1 |
| net.ipv4.tcp_slow_start_after_idle | 1 | 0 |
Do note that this config is tuned for better latency.
Server Hardware
My server (Virtual Private Server) is quite low powered, but cheap nontheless. Hardware goes as follows:
- OS: Ubuntu 22.04.4 LTS x86_64
- Host: KVM/QEMU
- Kernel: 5.15.0-116-generic
- CPU: QEMU Virtual version 2.5+ (1) @ 2.593GHz
- Memory: 957MiB
- Zram: 1.9G lzo-rle
Numbers
Do note that I’m testing this on my virtual machine
The test I will do is very simple. I have a program Vegeta that will thow as many GET requests at my site running in a virtual machine, that my computer can handle. The test is run 1 minut after server reboot. Example command used for this test:
echo "GET http://localhost:1313/" | ./vegeta attack -duration=1s -rate=10 | tee results.bin | ./vegeta report
| Duration | rate | Latency (mean) Default | Succes Ratio | Latency (mean) Fine tuned | Succes Ratio |
|---|---|---|---|---|---|
| 1s | 10 | 2.764s | 100% | 0.899s | 100% |
| 1s | 1.000 | 3.180s | 100% | 0.99s | 100% |
| 1s | 5.000 | 11.499s | 50.66% | 11.804s | 48.85% |
| 10s | 500 | 19.649s | 40.20% | 0.0200s | 100% |
| 10s | 1.000 | 0.0203s | 100% | 0.0201s | 100% |
| 60s | 500 | 0.0206s | 100% | 0.0200s | 100% |
Although there is not much we can do when we only have one virtual thread to work with, I would say it was worth researching and fine tuning. We don’t see much improvement, except for 10 seconds with 500 request/sec, and some marginal improvements.
If the server had more threads, or maybe even an actual CPU core, that would be better. But we have to work with what we have, and optimizing software before paying for new hardware, seems to be worth it.
Changes To /etc/sysctl.d/99-sysctl.conf
net.ipv4.tcp_fastopen=3
net.ipv4.tcp_congestion_control=bbr
net.core.default_qdisc=cake
net.ipv4.tcp_window_scaling=1
net.ipv4.tcp_mtu_probing=1
net.ipv4.tcp_low_latency=1
net.ipv4.tcp_slow_start_after_idle=0