Repository: web-infra-dev/midscene

Stars: 12688

CLAUDE.md Copy Chapter

AGENTS.md

README.md Copy Chapter

<p align="center">
<img alt="Midscene.js" width="260" src="https://github.com/user-attachments/assets/f60de3c1-dd6f-4213-97a1-85bf7c6e79e4">
</p>

<h1 align="center">Midscene.js</h1>
<div align="center">

English | 简体中文

<strong>Official Website</strong>: <a href="https://midscenejs.com/">https://midscenejs.com/</a>

<a href="https://trendshift.io/repositories/12524" target="_blank"><img src="https://trendshift.io/api/badge/repositories/12524" alt="web-infra-dev%2Fmidscene | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>

</div>

<p align="center">
AI-powered, vision-driven UI automation for every platform.
</p>

<p align="center">
<a href="https://www.npmjs.com/package/@midscene/web"><img src="https://img.shields.io/npm/v/@midscene/web?style=flat-square&color=00a8f0" alt="npm version" /></a>
<a href="https://huggingface.co/ByteDance-Seed/UI-TARS-1.5-7B"><img src="https://img.shields.io/badge/UI%20TARS%20Models-yellow" alt="hugging face model" /></a>
<a href="https://npm-compare.com/@midscene/web/#timeRange=THREE_YEARS"><img src="https://img.shields.io/npm/dm/@midscene/web.svg?style=flat-square&color=00a8f0" alt="downloads" /></a>
<a href="https://github.com/web-infra-dev/midscene/blob/main/LICENSE"><img src="https://img.shields.io/badge/License-MIT-blue.svg?style=flat-square&color=00a8f0" alt="License" />
<a href="https://discord.gg/2JyBHxszE4"><img src="https://img.shields.io/discord/1328277792730779648?style=flat-square&color=7289DA&label=Discord&logo=discord&logoColor=white" alt="discord" /></a>
<a href="https://x.com/midscene_ai"><img src="https://img.shields.io/twitter/follow/midscene_ai?style=flat-square" alt="twitter" /></a>
<a href="https://deepwiki.com/web-infra-dev/midscene">
<img alt="Ask DeepWiki.com" src="https://devin.ai/assets/deepwiki-badge.png" style="height: 18px; vertical-align: middle;" />
</a>
</p>

📣 Midscene Skills is here! Copy Chapter

Use Midscene Skills to control any platform with OpenClaw

Showcases Copy Chapter

* Web Automation - Automatically register the GitHub form in a web browser and pass all field validations
* iOS Automation - Meituan coffee order
* iOS Automation - Auto-like the first @midscene_ai tweet
* Android Automation - DCar: Xiaomi SU7 specs
* Android Automation - Booking a hotel for Christmas
* MCP Integration - Midscene MCP UI prepatch release
* robotic arm + vision + voice for in-vehicle testing

💡 Features Copy Chapter

Write Automation with Natural Language Copy Section

Web & Mobile App & Any Interface Copy Section

For Developers Copy Section

👉 Zero-code Quick Experience Copy Chapter

- Chrome Extension: Start in-browser experience immediately through the Chrome Extension, without writing any code.
- Android Playground: There is also a built-in Android playground to control your local Android device.
- iOS Playground: There is also a built-in iOS playground to control your local iOS device.

✨ Driven by Visual Language Model Copy Chapter

Midscene.js is all-in on the pure-vision route for UI actions: element localization and interactions are based on screenshots only. It supports visual-language models like Qwen3-VL, Doubao-1.6-vision, gemini-3-pro, and UI-TARS. For data extraction and page understanding, you can still opt in to include DOM when needed.

* Pure-vision localization for UI actions; the DOM extraction mode is removed.
* Works across web, mobile, desktop, and even <canvas> surfaces.
* Far fewer tokens by skipping DOM for actions, which cuts cost and speeds up runs.
* DOM can still be included for data extraction and page understanding when needed.
* Strong open-source options for self-hosting.

Read more about Model Strategy

📄 Resources Copy Chapter

* Official Website: https://midscenejs.com
* Documentation: https://midscenejs.com
* Sample Projects: https://github.com/web-infra-dev/midscene-example
* API Reference: https://midscenejs.com/api
* GitHub: https://github.com/web-infra-dev/midscene

🤝 Community Copy Chapter

* Discord
* Follow us on X
* Lark Group(飞书交流群)

🌟 Awesome Midscene Copy Chapter

Community projects that extend Midscene.js capabilities:

* midscene-ios - iOS Mirror automation support for Midscene
* midscene-pc - PC operation device for Windows, macOS, and Linux
* midscene-pc-docker - Docker image with Midscene-PC server pre-installed
* Midscene-Python - Python SDK for Midscene automation
* midscene-java by @Master-Frank - Java SDK for Midscene automation
* midscene-java by @alstafeev - Java SDK for Midscene automation

📝 Credits Copy Chapter

We would like to thank the following projects:

- Rsbuild and Rslib for the build tool.
- UI-TARS for the open-source agent model UI-TARS.
- Qwen-VL for the open-source VL model Qwen-VL.
- scrcpy and yume-chan allow us to control Android devices with browser.
- appium-adb for the javascript bridge of adb.
- appium-webdriveragent for the javascript operate XCTest。
- YADB for the yadb tool which improves the performance of text input.
- libnut-core for the cross-platform native keyboard and mouse control.
- Puppeteer for browser automation and control.
- Playwright for browser automation and control and testing.

📖 Citation Copy Chapter

If you use Midscene.js in your research or project, please cite:

@software{Midscene.js,
  author = {Xiao Zhou, Tao Yu, YiBing Lin},
  title = {Midscene.js: Your AI Operator for Web, Android, iOS, Automation & Testing.},
  year = {2025},
  publisher = {GitHub},
  url = {https://github.com/web-infra-dev/midscene}
}

✨ Star History Copy Chapter


📝 License Copy Chapter

Midscene.js is MIT licensed.

---

<div align="center">
If this project helps you or inspires you, please give us a star
</div>