1 Reference Public Logs
picklemustard edited this page 2026-09-07 15:49:27 -04:00

Sanitized public build progress

In anonymous mode, GET /api/v1/builds/{id}/logs needs neither login nor human verification. It serves a separate public projection from PostgreSQL. Every API replica can replay it without accessing a worker's files or in-memory log buffer.

The projection uses a closed vocabulary: preparation started, compilation started, compiler output received, warnings reported, errors reported, and artifact finalization. Each category is recorded at most once per attempt. The stream also reports the final public build status. This is a concise progress log, not a redacted transcript: compiler messages, filenames, source excerpts, repository URLs, credentials, IP addresses, administrative identities and storage locations are never copied into it. Arbitrary build output cannot be safely made public by replacing a few known secret formats.

Preparation diagnostics are sent only to the private sink. Compiler output is classified with bounded line buffering, including output split across writes. Only event kinds are persisted, with a database constraint rejecting other values. Unknown kinds also fail closed at the HTTP boundary. The public DTO advertises logs_sanitized: true and whether the stream is configured.

Raw logs remain available to administrators at GET /api/v1/admin/builds/{id}/logs using the current administrative token authentication. Authelia integration remains separate work. Legacy token mode retains its existing build-log contract. Historical raw files are not republished or automatically rewritten; builds without public events receive an explicit notice and their public outcome. Historical files and build records are preserved.

Persistence and reconnects

Migration 0004_public_logs.sql adds public_log_events. Writes lock the job row before allocating a per-job sequence and verify worker ID, attempt, unexpired lease, and nonterminal status. Duplicate category writes are idempotent. A terminal job cannot receive late events, so the API reads status before the final event batch and can end without losing a committed tail.

Events carry SSE id fields. Last-Event-ID resumes strictly after that sequence; malformed or negative cursors return 400. Reads are bounded to 256 events per batch. Active streams poll shared storage every 500 ms and send keepalives every 10 seconds. The browser keeps its existing lines when EventSource reconnects. The legacy stream still closes on transport errors because it has no cursor.

Public events contain no IP data and remain with the durable build history. Only six event kinds per attempt can be stored. They are not the full compiler log archive. Persistence failures are recorded administratively and can leave gaps in public progress, without stopping an admitted build or exposing raw output.

Shared stream limits and failures

Redis sorted-set leases enforce four public streams per canonical IP across API replicas, regardless of cookies or authorization headers. Lease keys hash the IP; random stream IDs expire after 30 seconds and renew every 10 seconds. Closed streams release their lease. Requests beyond the limit receive 429 and Retry-After; Redis failure refuses admission with 503. A lost lease or dependency failure ends the stream with a named unavailable event rather than claiming the build finished. A stalled writer has a ten-second write deadline. Responses use Cache-Control: no-store and disable proxy buffering.

The verification Redis ACL additionally needs TIME, ZREMRANGEBYSCORE, ZCARD, ZADD, ZSCORE, and ZREM, restricted to the existing gbb:human:* prefix. Configure the same trusted proxy chain as anonymous verification. General request limits remain per replica; durable bans and periodic ban-driven revocation are not implemented by these leases.

Validation and remaining integration

Tests cover adversarial and fragmented output, oversized lines, preparation-log separation, raw-log authorization, historical privacy, cursor replay, dependency failure, and frontend reconnect behavior. Real PostgreSQL/Redis tests cover concurrent writes, sequence allocation, stale/expired/terminal writer rejection, four-stream enforcement, expiry, and replay through two independent API/store instances. Run them with GBB_TEST_DSN and GBB_TEST_REDIS_URL pointing to isolated test services.

The current Docker queue writes these events. The Kubernetes controller must connect its phase transitions and compiler collection to this projection under its future lease-generation fence. Shared raw/sanitized diagnostic chunk archives, controller-crash log collection and real Cloudflare/Traefik SSE acceptance remain outstanding. No production migration or rollout was performed during development.