How Zelie works

One binary, four processes

Zelie is a single zelie binary. On a server it runs as four processes.

• The core runs as root. It talks to containerd, sets up networks and firewall rules, and manages disks. It has no network port. It accepts a small set of typed requests, such as "start this container", over a Unix socket. The kernel tells it who is on the other end. Only root and the panel are answered, and the SFTP server may use the file routes.
• The panel runs as an unprivileged user. It serves the interface and API, handles logins and receives GitHub webhooks. It has no network port either: the proxy hands it web traffic over a Unix socket.
• The proxy runs as its own unprivileged user and may only bind ports 80 and 443. It gets certificates from Let's Encrypt with CertMagic, terminates HTTPS and forwards each domain to the container that serves it. It requests certificates only for routed domains and forwards only to container addresses, never to the host.
• The SFTP server runs as its own unprivileged user. systemd holds its port and starts it when someone connects. It exits after five idle minutes. It asks the panel who may log in and passes every file operation to the core, which allows it the file routes only, and only on game server volumes.

A bug in the web interface gives an attacker an unprivileged account and a short list of allowed requests, not root. The panel can restart while every site keeps serving.

Every process sets PR_SET_THP_DISABLE and re-executes once at start. Otherwise the kernel folds small Go heaps into 2 MB pages that cannot be handed back.

Signing in

Every account needs a passkey or an authenticator app besides its password. The first administrator is created from a one-time link that zelie setup-link prints on the server.

Passwords are hashed with argon2id. The browser holds a random session token in a __Host- cookie, and the database keeps only a hash of it. The interface runs under a Content-Security-Policy that allows no inline code except its own, by hash.

Changing how an account signs in needs a second step from the last 15 minutes. Changing the password logs out every other browser.

Failed attempts are counted in the database, with an IPv6 /64 as one address. Thirty wrong passwords block an address for up to 15 minutes. Ten for one account block that account from that address only, so nobody who knows an email can lock its owner out. After a few failures, each try needs a proof of work that grows with the failures.

Containers

Every app and game server runs in its own container. Zelie talks to containerd directly, not through Docker, so restarting the runtime does not stop containers. Nothing runs privileged. Each container gets:

• a user namespace, so root inside is an unprivileged user on the host
• the default seccomp profile and a reduced set of capabilities
• limits on CPU, memory, disk and process count. Memory is a hard limit with no swap.
• an output log cleared at 64 MiB
• a disk allowance, measured once a minute: the app's volume limits added up, and at least 5 GB. Writes outside the volumes count too. An app past it is stopped.
• a place on its project's network only. It reaches the internet and its project's other containers, not other projects or the host. Nothing outside can connect to it except through a port Zelie forwards.

A container cannot connect to the host, apart from asking the DNS server on its network's gateway address. The one exception is host access, for a MariaDB installed on the server. It is off for every app, and only an administrator can turn it on. It opens TCP port 3306 only, on that app's own gateway address. MariaDB sees the container's 10.210.x.x address, so create its users for host 10.210.% with strong passwords.

Apps from GitHub

Zelie creates a GitHub App in your account with GitHub's manifest flow. The App's private key and webhook secret are encrypted with the panel's key. The App can read code and set commit statuses, nothing else.

On a push:

1. The code is built in a temporary container, with the repository's Dockerfile or with Railpack.
2. Tests run in the new image, in their own container, for at most ten minutes.
3. The new version must pass a health check. An app with a domain must answer HTTP with a status below 500. An app without one must stay up. An app with a domain and no volumes starts beside the old version. Any other app stops the old one first, and starts it again if the new one fails.
4. Traffic moves over. The images of the last five live versions are kept, so a rollback needs no build.

If a step fails, the old version keeps serving and the panel shows why. Each webhook is checked against the webhook secret first, and a repeated delivery is ignored.

Builds run BuildKit in an ordinary Zelie container, with its own user namespace and limits. The source archive is unpacked in a separate container with nothing else mounted, so crafted symlinks have nothing to reach. Every app has its own build cache.

Databases

MariaDB, PostgreSQL and Redis each run in their own container with a volume. A database never gets a public port. Each app is on its own network and the firewall drops traffic between them. Linking a database opens one way through, to its one port.

The password is sealed for the core. The panel stores it but cannot read it. An administrator may have it shown after a second step from the last 15 minutes. The core opens only app users' passwords this way, never a root password, and every display is logged.

The Data tab runs the engine's own client inside the database's container. The core builds each query itself and runs it read-only with a time limit. Desktop tools connect through an SSH tunnel to a port on 127.0.0.1, as a user made for them alone.

Backups

Backups are encrypted with age before they reach the disk. The key is the core's. A recovery file lets another server open them.

A backup is a tar archive. A file with holes is stored as a sparse entry in GNU tar's PAX 1.0 format, so backup and restore time follow the file's size on disk, not its length. tar, bsdtar and Go's archive/tar expand such entries, so a backup can be restored by hand. If a hole map would pass 1 MiB, the shortest holes are filled with zeros. A file that would need more zeros than it takes on disk, or a GiB, is refused with an error that names it.

A restore only reads archives Zelie made: its kept backups, their off-site copies and the stream of a clone. Uploaded volume backups are refused.

Game servers

The file manager works inside the core, which opens the server's volume as an os.Root. .., absolute paths and symbolic links cannot lead out of it, even if a link is swapped mid-request. Extracting stops at the first entry that would leave the folder, a link pointing out, a device file, or a size or entry count over the limit.

SFTP uses the same operations. A login is an SSH key added to an administrator's account, or the server's own SFTP password, stored as an argon2id hash. An account's password is never accepted. The server speaks only the sftp subsystem: no shell, no commands, no port forwarding. Every minute an open connection is checked with the panel, so removing a key or deleting the server ends it.

Players connect straight to the game server. The administrator gives Zelie a pool of ports and each server takes some. The core opens a port with an nftables rule that forwards to the container and one that lets that traffic through, and in ufw or another iptables firewall where present. The rules follow the container's address and go when the server is deleted. The SFTP port can never be given to a game server.

A game server runs as user 988 inside its container, as in Wings. To stop it, Zelie sends the egg's stop command or signal, and kills it if it still runs a minute later.

State and secrets

The panel keeps its state in SQLite. Variables marked secret are sealed with a public key whose private half only the core holds. The panel stores the sealed value, and the core opens it when it starts the container. A copy of the database therefore reveals no secret. This does not protect against a taken-over panel, which could deploy a version that prints its environment.

For builds, the core hands variables to BuildKit as mounted secrets, not build arguments, so values stay out of the image and its history.

The core's key is /var/lib/zelie/secrets.key. Losing it means losing the secrets, so keep a copy somewhere safe.

Updates

Releases are signed. Zelie downloads a new version, checks its signature, starts it and checks its health. If it does not come up, Zelie switches back. Apps and game servers keep running, since they live in containerd and not in the Zelie process.

Several servers

The paid edition can manage more than one server. Extra servers run the same binary in agent mode and connect out to the panel.