Browser Harness — Self-Healing Browser Automation for LLM Agents

Introduction

Browser Harness is a self-healing automation framework from the Browser Use team. It enables LLM agents to interact with web pages, fill forms, navigate sites, and complete multi-step tasks in a real browser. When elements move or pages change, the harness automatically adapts its selectors and retries, making agent-driven web workflows robust in production.

What Browser Harness Does

• Provides LLM agents with a persistent browser session via Chrome DevTools Protocol (CDP)
• Automatically recovers from selector failures by re-identifying page elements using AI
• Supports multi-tab and multi-window workflows within a single agent session
• Integrates with Playwright for reliable cross-browser rendering and interaction
• Exposes a Python API for embedding browser actions into agent tool pipelines

Architecture Overview

Browser Harness wraps a Playwright-managed Chromium instance and exposes browser actions as callable tools for LLM agents. It maintains a persistent session state including cookies, local storage, and open tabs. When an action fails due to a changed DOM, the self-healing module captures a screenshot, sends it to the LLM for element re-identification, and retries the action with updated selectors. Communication between the agent and the browser happens over CDP, keeping latency low.

Self-Hosting & Configuration

• Install from PyPI with pip install browser-harness
• Requires a local Chromium or Chrome installation (or use the bundled Playwright browsers)
• Configure the LLM provider (OpenAI, Anthropic, or compatible) via environment variables
• Set headless or headed mode, viewport size, and timeout defaults in the config file
• Deploy in Docker for CI/CD pipelines with the included Dockerfile

Key Features

• Self-healing selectors automatically adapt when page layouts change
• Persistent browser sessions maintain login state across multiple agent turns
• Screenshot-based fallback lets the LLM visually identify elements when DOM queries fail
• Cloud browser support for running sessions on remote VPS or container infrastructure
• Open-source with active development and community contributions

Comparison with Similar Tools

• Browser Use — the original agent browser library; Browser Harness extends it with self-healing and persistence
• Playwright — powerful browser automation but requires manual selector maintenance; Browser Harness adds AI-driven recovery
• Selenium — traditional browser testing framework; Browser Harness is purpose-built for LLM agent workflows
• Stagehand — AI browser automation with a different API design; Browser Harness focuses on session persistence
• Pydoll — WebDriver-free automation; Browser Harness uses CDP with self-healing on top

FAQ

Q: Does Browser Harness work with any LLM provider? A: Yes, it supports OpenAI, Anthropic, and any OpenAI-compatible API for the self-healing element identification.

Q: Can I run Browser Harness in headless mode? A: Yes, headless mode is the default for server and CI environments. Switch to headed mode for debugging.

Q: How does the self-healing work? A: When a selector fails, the harness captures a screenshot, sends it to the configured LLM, and asks it to identify the target element. It then generates a new selector and retries the action.

Q: Is there a Docker image available? A: Yes, the repository includes a Dockerfile for containerized deployment with pre-installed browser dependencies.

Sources

• https://github.com/browser-use/browser-harness