Midscene.js
Interact, Query and Assert by Natural Language
Whether you're brainstorming, testing, or automating UI interactions, Midscene.js provides the perfect balance of simplicity and functionality through natural language commands.
Action
Execute a series of actions by describing the steps in natural language using .ai or .aiAction methods.
Query
Extract customized data from the UI using .aiQuery. Simply describe the JSON format you want, and AI will understand the page.
Assert
Perform assertions on the page using .aiAssert with natural language descriptions, making test maintenance easier.
Visual Reports
Get detailed visual reports after each run, with animated replays and step-by-step analysis for better debugging.
Visualized Reports for Better Debugging
Midscene provides comprehensive visual reports after each automation run, making debugging and maintenance more intuitive and efficient.
- •Animated replay of each automation step
- •Interactive playground for prompt adjustment
- •Detailed step-by-step execution analysis
Simple Yet Powerful
// 👀 type keywords, perform a search
await ai('type "Headphones" in search box, hit Enter');
// 👀 find the items, return in JSON
const items = await aiQuery(
"{itemTitle: string, price: Number}[], find item in list and corresponding price"
);
console.log("headphones in stock", items);
// 👀 assert by natural language
await aiAssert("There is a category filter on the left");Multiple Ways to Integrate
YAML Scripts
Write automation scripts in YAML format for a more declarative approach to UI testing.
Chrome Extension
Control desktop Chrome directly through the Midscene extension for quick automation tasks.
Puppeteer Integration
Seamlessly integrate with Puppeteer for powerful browser automation capabilities.
Playwright Support
Use Midscene with Playwright for cross-browser testing and automation.
Community & Resources
About Midscene
Midscene is an open-source UI automation tool that makes testing and interaction with web applications intuitive through natural language. It supports both general-purpose LLMs and specialized open-source models for enhanced performance and privacy.