godot-binary-builder

Compile Godot with a chosen set of engine-level modules, for Windows, macOS, and Linux, and hand the binary to whoever asked for it.

Some of our tooling lives deep in the engine — C++ modules compiled into the core, not addons loaded at runtime. Not every project needs every module, and building the right combination by hand for three operating systems does not scale. This service takes a Godot version plus a list of modules and returns a binary.

Documentation

The project wiki covers getting started, building Godot, configuration, module administration, the API, deployment, security, and troubleshooting. Start there for current workflows; detailed design and implementation notes remain under docs/.

Status

Pre-alpha. It builds Godot for Linux, and the service around that now runs builds — but has never been run in a cluster.

gbb build --local runs the whole pipeline in one process — clone the engine, compile it in a container, collect the binary — and hands back a working editor. Since M3 the server does the same thing through HTTP: submit a build, watch its log stream, download the result, cancel it mid-compile. No endpoint returns 501 any more, and gbb speaks to a server rather than only to itself. The contract is docs/api.md.

What has not happened is a deployment. The manifests render, the image builds, and no build has ever run against the dind sidecar or the real object store — which means workspace_mount: volume, the mode every deployed build would use, is still reasoned about rather than demonstrated. Each gap is tracked to a milestone in ROADMAP.md.

Piece	State
Build spec, validation, cache key	Implemented
Module catalog and compatibility lists (Postgres)	Implemented, admin API not exercised against a deployed instance
Config loading, HTTP routing, static embedding	Implemented
Job state machine, log buffer	Implemented
Artifact storage (S3 + filesystem) and download delivery	Implemented, untested against the real bucket
Job queue, build handlers, SSE log streaming	Implemented, never run in a cluster
gbb against a server: build, list, status, fetch, modules	Implemented
Workspace assembly and SCons execution in a container	Implemented, Linux verified end to end
gbb build --local	Implemented
Module resolution and grafting	Implemented, fixtures not yet compiled into a real engine
Job persistence (Postgres)	Implemented
Authentication on every build route, session cookie, per-token quotas	Implemented
Rate limiting, audit log, artifact and log retention	Implemented, retention off by default; on at 30 days in the ConfigMap
Postgres-backed queue: leases, heartbeats, cancel-as-a-row	Implemented, exercised by two queues in one process, never by two pods
Frontend: submission form, build list, streaming log view	Implemented; exercised against two real builds (M4-10), browser-side steps outstanding
Admin UI: catalog, engine versions, compatibility, tokens, audit	Implemented (M9); not yet operated by hand in a browser
Docker SDK client and container config	Implemented
Server image (build/docker/server.Dockerfile)	Builds and runs
Kubernetes manifests	Render, but never applied to a cluster
Toolchain images: base, linux	Build; verified compiling Godot for x86_64 and arm64
Toolchain image: windows	Builds; verified compiling a Godot editor .exe
Toolchain image: macos	Never built — needs an Apple SDK; see docs/macos-sdk.md

Architecture

  Svelte UI ──▶ Go API ──▶ job queue ──▶ container runner ──▶ SCons
                                                                │
                             object store ◀─────────────────────┘
                                    │
  Svelte UI ◀── SSE logs, then a presigned download URL

Finished builds go to an S3-compatible object store at store.picklemustard.dev. Downloads redirect there rather than streaming through the API server — see docs/artifacts.md.

A build request is a BuildSpec — engine ref, platform/arch/target, module list. Everything downstream is a function of that value. Full detail in docs/architecture.md. The HTTP surface is written down in docs/api.md.

Cross-compilation happens in containers launched over the Docker Engine API, so one Linux host produces all three targets: native GCC for Linux, mingw-w64 for Windows, osxcross for macOS.

Deployment

Packaged as a container image and deployed to Kubernetes as a StatefulSet with a docker:dind sidecar that runs the builds. The server image carries no build toolchain — it orchestrates, it does not compile — but it does carry git and ssh, because module repositories are cloned host-side. It also needs a Postgres instance for the module catalog; the manifests do not provision one.

Follow the cluster configuration runbook to provision dependencies, customize an overlay, supply secrets, configure ingress, and verify the rollout. The base manifests require those steps before they can serve builds.

Requirements

• Go 1.26+
• Node 22+
• A reachable Docker daemon (DOCKER_HOST); podman's Docker-compatible socket works for local development
• Disk: a Godot mirror plus ccache runs to several GB

Quickstart

git clone ssh://git@git.picklemustard.dev:2222/StudioTeal/GodotBinaryBuilder.git
cd GodotBinaryBuilder

cp configs/config.example.yaml configs/config.yaml
# set database.dsn, or export GBB_DATABASE_DSN

make dev          # gbb-server on :8080, Vite on :5173

Open http://localhost:5173 in development — Vite proxies /api and /healthz to the Go server. For a production-shaped run:

make build        # frontend, then both binaries into bin/
./bin/gbb-server  # serves the API and the embedded frontend on :8080

The server connects to a Docker daemon at startup and logs what it found. On a host running podman rather than Docker:

systemctl --user enable --now podman.socket
export DOCKER_HOST=unix://$XDG_RUNTIME_DIR/podman/podman.sock

Without a reachable daemon the server still starts and logs a warning; only builds fail.

The CLI is a client for the same API. Every build route needs a token — -token, or $GBB_TOKEN; an admin issues one with POST /api/v1/admin/tokens (see docs/admin.md):

export GBB_TOKEN=gbb_…
./bin/gbb build --godot 4.4-stable --platform linuxbsd \
                --modules deep_engine_tool --dry-run

A browser exchanges the same token for a session cookie once, on first load, because EventSource cannot send an Authorization header and so a build's log could not otherwise be followed. The two catalog reads and /healthz stay open.

--local runs the build here instead of submitting it, which is the only way to get a binary today:

DOCKER=podman IMAGE_PREFIX=localhost/gbb/ ./scripts/build-images.sh
./bin/gbb build --godot 4.4-stable --platform linuxbsd --target editor --local

The first run clones the engine (~2 GB, several minutes); later runs reuse the mirror and the ccache volume. Binaries land under ./out. A podman host needs build.image_prefix: "localhost/gbb/", build.workspace_mount: "bind", and build.container_user: "0:0" — see docs/deployment.md for why each one is not optional.

--dry-run prints the validated spec, which works today. Submitting lands in M3.

Configuration

configs/config.yaml, with every value overridable by a GBB_-prefixed environment variable (GBB_ADDR, GBB_DATA_DIR, GBB_WORKERS, …). See configs/config.example.yaml for the annotated list.

Finished builds go to the object store configured under artifacts:. Credentials come from GBB_S3_ACCESS_KEY_ID and GBB_S3_SECRET_ACCESS_KEY, not the config file, so they can live in a Kubernetes Secret. Set artifacts.backend: fs for local development without a bucket.

Logging

At the default info the server narrates each build one line per stage -- claimed, preparing, compiling, packaging, succeeded or failed -- and logs nothing per HTTP request except a 4xx or 5xx. A build's compiler output is not printed to the console; it goes to <data_dir>/logs/<job id>.log, whose path appears on the compiling line, and logging.build_output_file: false turns that file off.

logging.level: trace (or GBB_LOG_LEVEL=trace, or --log-level trace) adds a line per request, the route guards chosen at startup, the SCons command line, and a tee of every running build's raw output to the console. It is a debugging measure, not a steady state.

The catalog

What users can pick — modules, their versions, the selectable Godot versions, and which modules each Godot version offers — lives in Postgres and is edited over the admin API, not in a config file. GBB_DATABASE_DSN is required and the server will not start without it.

Compatibility runs the way you would ask about it: each Godot major/minor has a stored list of the modules that work with it, generated once and then frozen, so a module's later release does not change what an older engine offers. See docs/modules.md to register a module and docs/admin.md to curate the lists.

Cross-compilation notes

make images builds the toolchain images. They must end up in the sidecar's image store, not the node's — push them to a registry the cluster can pull from. See docs/deployment.md.

Each image carries every architecture it can build, and the image's entrypoint (build/docker/entrypoint.sh) supplies the SCons arguments that depend on the toolchain rather than on the request. A build spec therefore differs only by platform= and arch= across all three.

Windows works out of the box: mingw-w64 comes from Debian packages. Linux arm64 works through Debian's crossbuild-essential-arm64. Windows on arm64 is refused at submission — Godot supports it, but Debian ships no aarch64 mingw, so no image here can build it.

macOS does not. osxcross needs an Apple SDK extracted from Xcode, and Apple's licence forbids us from redistributing it — so you must supply your own and it must never be committed or pushed to a shared registry. .gitignore blocks the paths involved. Read docs/macos-sdk.md before attempting it. scripts/build-images.sh skips the macOS image when no SDK is present.

Layout

Path	Contents
cmd/gbb-server	The service: API, queue, embedded frontend
cmd/gbb-gateway	Artifact downloads only: no build, migration, or upload role
cmd/gbb	CLI client
internal/spec	BuildSpec, validation, cache key
internal/registry	Godot version's module list → checkouts
internal/source	Engine checkout with modules grafted in
internal/runner	SCons execution over the Docker Engine API
internal/job	Job lifecycle, queue, log fan-out
internal/artifact	Artifact storage: S3 and filesystem backends
internal/store	Postgres: jobs, module catalog, compatibility lists, tokens
internal/httpapi	REST handlers and static file serving
build/docker	The server image, and the per-platform toolchain images
deploy/k8s	Kubernetes manifests (StatefulSet, dind sidecar, config)
web	SvelteKit frontend
configs	Annotated example configuration
docs	Architecture, deployment, artifacts, module registration, macOS SDK
scripts	Development and image-build helpers

Development

make check    # gofmt, go vet, go test
make web      # rebuild the frontend into the server's embed directory
make help     # list every target

The frontend is embedded into the server binary from internal/httpapi/dist, which make web populates from web/build. Only the directory itself is tracked in git, so go build works on a fresh checkout without running npm — the server then serves a placeholder page telling you to run make web.

Roadmap

ROADMAP.md — M0 scaffolding through M8 signed distributables.

License

MIT. See LICENSE.