#Try the GUI Agent

Use the Chrome extension to try the GUI Agent in a browser Playground without setting up a test project. This guide walks you through model configuration, running a natural-language instruction, checking its result, and reviewing execution.

To write YAML tests directly, go to Midscene Test. For mobile and desktop Playgrounds, see other platforms.

#Configure a model

Before using the Chrome extension, prepare a multimodal model with UI localization capabilities. If you have not chosen a model or connection method, start with the Model Selection Guide.

The following example uses qwen3.7-plus through Alibaba Cloud:

export MIDSCENE_MODEL_BASE_URL="https://dashscope.aliyuncs.com/compatible-mode/v1"
export MIDSCENE_MODEL_API_KEY="your-api-key"
export MIDSCENE_MODEL_NAME="qwen3.7-plus"
export MIDSCENE_MODEL_FAMILY="qwen3"

Replace your-api-key with your API Key.

To use another model, such as Doubao, GLM, Gemini, or GPT-5, see Use a model API key.

For all configuration options, see Model configuration.

If you use a ChatGPT subscription, follow Use a ChatGPT subscription to obtain the complete model configuration, then continue with extension installation. Choose either connection method.

After installing the Chrome extension, paste this configuration into its settings.

#Install the Chrome extension

1. Use the button below to install Midscene from the Chrome Web Store:

   

2. Open Midscene from the Chrome extensions list. The Midscene sidebar appears on the right side of the browser.

3. Click the settings icon in the sidebar, paste the complete configuration from Configure a model, and save it.

#Perform an action and check the result

Open the TodoMVC example, keep the page visible, and try the following API categories in the Midscene sidebar.

#1. Complete a goal (aiAct)

Select aiAct, then enter and run:

Add a todo named "Try Midscene", then mark it as completed.

The Agent observes the page, finds the input, adds the todo, and checks it off. When execution finishes, you should see the todo marked as completed.

#2. Check the result (aiAssert)

Select aiAssert, then enter and run:

The list contains a todo named "Try Midscene", and it is marked as completed.

The assertion passes if the condition is true. Otherwise, the result shows an error with the model's reason.

#3. Extract data (aiQuery, optional)

Select aiQuery, then enter and run:

Visible todos in the list, return {name: string, completed: boolean}[].

The result contains todo names and completion states. The same interface can be operated, checked, and read.

#Review execution

After running an instruction, inspect the steps and screenshots in the Playground's result area and compare them with the final page state. If execution fails, inspect the failed step and error first. For model connection issues, see Model advanced configuration and debugging.

Here is another complete example: filling out a GitHub sign-up form with the Chrome extension. Use the report link below the video to explore execution replay.

Prompt: Fill out the GitHub sign-up form and pass validation, but do not submit it.

Midscene generates a complete report for every task so developers can review the operation process. See the report for the demo above: report.html

#Next steps

• Understand key APIs: Read The Basics.
• Write YAML tests: Create a project, run cases, and view reports with Midscene Test.
• Integrate into existing tests: Follow Integrate with Playwright or Integrate with Puppeteer to create an Agent and call APIs such as aiAct and aiAssert.
• Let an AI coding agent operate apps: Install Midscene Skills.

#Use Midscene on other platforms

Midscene provides the same complete automation capabilities on Android, iOS, HarmonyOS, and desktop, with a dedicated Playground for each platform. Before using a Playground, prepare the device environment for that platform. For example, Android requires adb to be installed and configured. See the following guides for platform requirements, Playground launch instructions, and troubleshooting.

#Troubleshooting