Production server
This is how the official The Hub server runs (an OVH VPS with 6 CPU cores, Ubuntu). Everything is in deploy/, so you can build the same setup on your own Linux server.
The layout
| Part | Runs as | Cores | What it does |
|---|---|---|---|
| nginx | nginx | 4, 5 | HTTPS, the landing page and wiki, static files, rate limits, routing API calls to a pool |
| PHP pool sync | hub-php@sync | 0, 1 | sync, bot_events, typing, read: the polling every open app does every 1.5 s |
| PHP pool api | hub-php@api | 2, 3 | Everything else people click: messages, settings, spaces, roles |
| PHP pool media | hub-php@media | 4 | Uploads, downloads, GIF search, Discord import, plugin installs |
| MariaDB | mariadb | 5 | The database |
| PeerJS | hub-peerjs | 5 | Introduces people in calls (WebSocket at /peerjs/) |
| coturn | coturn | 5 | STUN/TURN, so calls work behind strict networks |
Each PHP pool is its own php-fpm with its own socket, worker limit, memory limit and timeouts, pinned to its cores with systemd's CPUAffinity. nginx picks the pool from the request's ?action=. That way a flood of one kind of request can't take the rest down: a thousand open apps polling sync can't slow message sending, and a pile of uploads can't hold up either.
Staying up under load
- Rate limits per pool and per address in nginx (
limit_req), answering 429 instead of queueing for ever, plus a cap on connections per address. - Bounded pools: each pool has a maximum number of workers sized to the server's memory, recycles workers every 2,000 requests, and kills any request that runs past its timeout.
- Restarts on their own: every service has
Restart=always, and a memory ceiling, so a leak restarts one pool instead of filling the machine. - Health checks every minute (
hub-health.timer): each pool must answer?action=health, including a database check; one that doesn't is restarted on its own. Restarts are logged in/var/log/thehub/health.log. - Releases switch in one step: each deploy is a new folder; the
currentlink moves to it and the pools reload gracefully, so requests already running finish on the old code. The last 5 releases are kept to roll back to.
Deploying
From the project folder, on a computer with the deploy key:
bash deploy/deploy.sh --setup # the first time: installs and configures the server
bash deploy/deploy.sh # every update after thatserver-setup.sh installs nginx, PHP, MariaDB, Node (for PeerJS) and coturn, creates the database and a config.local.php with this server's settings (PeerJS, TURN, the plugin shop), and opens the firewall for web and calls only. Server settings live in /srv/thehub/shared/, and survive every deploy.
Where things are
| Path | What |
|---|---|
/srv/thehub/current/ | The live release (the app, the website and the wiki) |
/srv/thehub/shared/ | config.local.php, database.php, data/ (cache, sessions, the plugin shop's files), uploads/, plugins/ |
/etc/thehub/fpm-*.conf | The three PHP pools |
/var/log/thehub/ | PHP errors and slow requests per pool, and the health log |
Useful commands
systemctl status 'hub-php@*' hub-peerjs mariadb coturn nginx
journalctl -u hub-php@sync -f
sudo -u thehub php /srv/thehub/current/tools/shop-publish.php --list