Page Agent — In-Page AI Browser Automation by Alibaba

Introduction

Page Agent by Alibaba is an in-page GUI agent that translates natural language instructions into browser interactions. Unlike headless browser tools, it operates directly within the page context, giving it full access to the DOM, event system, and rendered layout for reliable automation.

What Page Agent Does

• Controls web interfaces through natural language commands
• Identifies and interacts with UI elements using visual and DOM-based understanding
• Fills forms, clicks buttons, navigates menus, and extracts data automatically
• Runs as a JavaScript library or browser extension within the page
• Integrates with LLMs via MCP for use in AI agent workflows

Architecture Overview

Page Agent injects a lightweight JavaScript agent into the target web page. The agent captures the page state, including DOM structure and visual layout, and sends it to a connected LLM for instruction interpretation. The LLM responds with structured actions (click, type, scroll) that the agent executes directly in the browser context, enabling precise interaction without external browser drivers.

Self-Hosting & Configuration

• Install via npm as a JavaScript library for programmatic use
• Load as a browser extension for interactive automation
• Configure the LLM backend with your preferred provider and API key
• Adjust action timeouts and retry policies through options
• Works with any modern Chromium-based browser

Key Features

• Natural language control of any web interface
• In-page execution without external WebDriver dependencies
• Visual and DOM-based element identification for robust targeting
• MCP server integration for multi-agent workflows
• TypeScript and JavaScript APIs for programmatic automation

Comparison with Similar Tools

• Browser Use — Python-based browser automation; Page Agent operates in-page with JavaScript for lower latency
• Playwright — programmatic browser control with code; Page Agent adds natural language interaction
• Stagehand — AI browser automation framework; Page Agent focuses on in-page GUI control
• Selenium — traditional WebDriver automation; Page Agent eliminates the need for an external driver

FAQ

Q: Does it work on any website? A: It works on any website accessible in a modern browser. Complex SPAs and dynamic pages are fully supported.

Q: Which LLM providers does it support? A: It supports OpenAI, Anthropic, and other providers through configurable API endpoints.

Q: Can it handle multi-step workflows? A: Yes, you can chain natural language instructions into multi-step automation sequences.

Q: Is it suitable for testing? A: It can be used for exploratory testing and UI verification, though dedicated testing frameworks may be preferred for CI pipelines.

Sources

• https://github.com/alibaba/page-agent