A Headless Browser Built from Scratch in Zig for AI Agents
Deep Technical Dive — Architecture, Internals & Benchmarks
Modern browsers are engineering marvels — Chrome has ~35 million lines of code. But most of that code serves visual rendering: CSS layout, compositing, rasterization, GPU acceleration, font shaping.
Even "headless Chrome" starts the full rendering pipeline. You're paying for a cinema ticket to read the subtitles.
Zig 0.15.2 is used — pre-1.0 but stable enough for production. The team explicitly chose it for "low-level system programming" with optimization headroom.
Lightpanda uses Servo's html5ever — the same spec-compliant HTML parser that powers Firefox's experimental engine.
Key insight: By using html5ever, Lightpanda avoids reimplementing the gnarliest part of browser development — spec-compliant HTML parsing with all its edge cases.
// Simplified flow Raw HTML bytes ↓ html5ever tokenizer ↓ html5ever tree builder ↓ Zig DOM tree ↓ Web API bindings ↓ V8 JavaScript context
Lightpanda embeds Google V8 — the same JS engine powering Chrome and Node.js — for full ECMAScript compliance.
fetch() and XMLHttpRequestV8's C++ API must be bridged to Zig. Lightpanda uses Zig's C ABI interop to create bindings that expose DOM objects as V8 JavaScript objects.
// Zig exposes DOM nodes to V8 // so JS like: document.getElementById('app') // resolves to a Zig-managed // DOM node, not a C++ object
Rather than implementing HTTP/1.1, HTTP/2, TLS, and connection pooling from scratch, Lightpanda delegates to libcurl — the battle-tested C library powering billions of HTTP requests daily.
One of Lightpanda's most useful features for automation:
// Via CDP, you can intercept // any network request: // Block images (save bandwidth) page.setRequestInterception(true) page.on('request', req => { if (req.resourceType() === 'image') req.abort() else req.continue() })
The Chrome DevTools Protocol is the standard wire protocol for controlling browsers programmatically. Lightpanda implements a CDP server so existing tools work unchanged.
// Connect Puppeteer to Lightpanda const browser = await puppeteer.connect({ browserWSEndpoint: 'ws://127.0.0.1:9222' }); const page = await browser.newPage(); await page.goto('https://example.com'); const title = await page.title();
Page — navigation, lifecycle eventsRuntime — JS evaluationDOM — tree inspection, manipulationNetwork — request interceptionTarget — page/context managementInput — keyboard/mouse eventsMultiple concurrent CDP connections are supported within a single Lightpanda process.
const puppeteer = require( 'puppeteer-core' ); const browser = await puppeteer.connect({ browserWSEndpoint: `ws://127.0.0.1:9222` }); // Use exactly like Chrome const page = await browser.newPage(); await page.goto('https://...'); await page.click('#submit');
const { chromium } = require( 'playwright' ); const browser = await chromium.connectOverCDP( 'http://127.0.0.1:9222' ); // Works, but Playwright may // use different code paths as // new APIs become available
Playwright's internal feature detection may route to different implementations as Lightpanda exposes more Web APIs. Test your specific workflows.
Why it matters: At cloud scale, 9x memory reduction means 9x more browser sessions per server. For a fleet running 10,000 concurrent sessions, that's the difference between 100 servers and 12.
| Subsystem | Chrome Headless | Lightpanda |
|---|---|---|
| CSS Layout (Blink) | Full layout tree computation | Eliminated |
| Rendering / Paint | Skia rasterization (skipped but loaded) | Never compiled in |
| GPU Process | Separate process, IPC overhead | Does not exist |
| Image Decode | PNG/JPEG/WebP decoders loaded | Eliminated |
| Font Shaping | HarfBuzz, FreeType loaded | Eliminated |
| Process Model | Multi-process (browser + renderer + GPU) | Single process |
| Memory Allocator | PartitionAlloc (general purpose) | Zig's explicit allocators |
Zig gives Lightpanda explicit control over every allocation. There is no hidden memory overhead from a garbage collector or runtime.
// Zig allocator pattern const allocator = std.heap .page_allocator; // Arena for a page session var arena = std.heap .ArenaAllocator .init(allocator); defer arena.deinit(); // All DOM nodes allocated here // When page closes: one free() const node = try arena .allocator() .create(DomNode);
Result: 24 MB peak for 100 pages vs Chrome's 207 MB. Each page context is a clean arena — no fragmentation, no leaks.
Lightpanda implements a custom DOM tree in Zig that supports the Web APIs needed for headless automation.
document.querySelector() / querySelectorAll()document.getElementById()document.createElement()element.innerHTML / textContentelement.getAttribute() / setAttribute()element.click()addEventListener() / dispatchEvent()window.location, window.historyThe full Web API surface has hundreds of interfaces with thousands of methods. Lightpanda's approach:
Unlike Chrome, which has no concept of robots.txt, Lightpanda has native support for respecting crawl directives.
# Start with robots.txt compliance lightpanda --obey_robots # Lightpanda will: # 1. Fetch /robots.txt first # 2. Parse Allow/Disallow rules # 3. Block navigation to # disallowed paths # 4. Respect Crawl-delay
This is a deliberate design choice that signals Lightpanda's positioning: not a scraping bypass tool, but infrastructure for responsible automation.
# Docker (amd64 + arm64) docker run -p 9222:9222 \ ghcr.io/lightpanda-io/browser # Binary (Linux x86_64, macOS arm64) curl -L https://github.com/ lightpanda-io/browser/ releases/latest \ -o lightpanda chmod +x lightpanda ./lightpanda --host 0.0.0.0 \ --port 9222
Managed infrastructure — no servers to run:
const browser = await puppeteer.connect({ browserWSEndpoint: 'wss://cloud.lightpanda.io' + '?token=YOUR_TOKEN' }); // Supports both Lightpanda // and Chrome backends // for maximum compatibility
The cloud option gives you both engines — use Lightpanda for speed, fall back to Chrome for full rendering when needed.
Chrome spawns separate OS processes for each tab (renderer, GPU, utility). Lightpanda runs everything in a single process with multiple CDP connections.
Chrome (1 page = ~4 processes) browser PID 1001 120 MB renderer PID 1002 45 MB gpu PID 1003 30 MB utility PID 1004 12 MB Total: ~207 MB for one page
Lightpanda (N pages = 1 process) lightpanda PID 2001 24 MB - CDP session 1 (page A) - CDP session 2 (page B) - CDP session 3 (page C) ... Total: ~24 MB for 100 pages
Trade-off: Single process means one crash affects all sessions. But for automation workloads, this is acceptable — sessions are short-lived and can be retried.
Lightpanda validates its implementation against the Web Platform Tests — the same test suite used by Chrome, Firefox, and Safari to ensure interoperability.
A browser that passes WPT for a given API means you can trust it behaves exactly like Chrome for that API. No surprises, no edge-case differences.
Full WPT coverage is a massive undertaking. Chrome has thousands of engineers; Lightpanda is a small team. Coverage will grow over time, and some niche APIs may never be implemented.
| Limitation | Impact | Workaround |
|---|---|---|
| No screenshots | Can't capture visual state | Use Chrome for visual testing |
| No CSS layout | getBoundingClientRect() unavailable | Use DOM selectors instead of coordinates |
| Incomplete Web APIs | Some sites may break | Report issues; coverage improves monthly |
| No media | Audio/video/WebRTC unsupported | Not needed for scraping/automation |
| Pre-1.0 Zig | Language may introduce breaking changes | Pinned to Zig 0.15.2 |
| Single process | One crash affects all sessions | Restart + retry (sessions are short-lived) |
# Clone and build git clone https://github.com/ lightpanda-io/browser.git cd browser zig build -Doptimize=ReleaseSafe
The build system is notable for orchestrating three languages:
build.zig // Zig build script → compiles Zig source → calls cargo for html5ever → links V8 static lib → links libcurl → single binary output
Zig's build system handles C/C++ and Rust dependencies natively — no CMake, no Makefiles, no autoconf. One build.zig file orchestrates everything.
| Tool | Engine | Memory (100pg) | Time (100pg) | JS Support |
|---|---|---|---|---|
| Lightpanda | Custom (Zig + V8) | 24 MB | 2.3s | Full (V8) |
| Chrome Headless | Blink + V8 | 207 MB | 25.2s | Full (V8) |
| Playwright (Chromium) | Blink + V8 | ~220 MB | ~27s | Full (V8) |
| jsdom (Node.js) | JS DOM simulation | ~80 MB | ~5s | Partial |
| Splash (Python) | QtWebKit | ~150 MB | ~15s | Full (WebKit) |
Lightpanda occupies a unique niche: full V8 JavaScript with the memory footprint of a DOM simulator. No other tool offers both.
AGPL-3.0 is a strong copyleft license. If you modify Lightpanda and offer it as a service, you must open-source your changes.
Business model: Open-source core + managed cloud. The AGPL ensures competitors can't take the code and offer a competing cloud service without contributing back.
github.com/lightpanda-io/browser • lightpanda.io • AGPL-3.0