Desktop control with a memory.

Install once. Keep the session.

This repository is a standalone OpenCode plugin. It contains the plugin entrypoint, the local MCP server, and the native runtime integration. The plugin registers cua_repl; the model sees only js and js_reset.

git clone git@github.com:rashidtvmr/opencode-computer-use.git
cd opencode-computer-use
npm install

{
  "$schema": "https://opencode.ai/config.json",
  "plugin": [
    ["/absolute/path/to/opencode-computer-use", { "backend": "auto" }]
  ]
}

opencode mcp list

{
  "$schema": "https://opencode.ai/config.json",
  "plugins": [
    {
      "package": "/absolute/path/to/opencode-computer-use",
      "options": { "backend": "auto" }
    }
  ]
}

Start with discovery.

Use the first call to see what the native runtime can actually see. Do not guess an app name or assume that a screenshot is available.

var apps = await cua.listApps({ emit: false });
var chrome = await cua.getApp("Google Chrome");
await chrome.getAXState({ textLimit: 12000 });

The generated snippet is intentionally small. Bindings persist between calls, so use var when a later call may need to reassign the same name.

One surface. Three layers.

1. OpenCode plugin
   Loads the local package and registers the MCP server without exposing raw desktop tools.
2. cua_repl MCP server
   Provides a persistent JavaScript evaluation surface, output bounds, request validation, and worker isolation.
3. Native runtime
   Runs open-computer-use as a separate child process. OS accessibility, screen capture, and input permissions remain there.

The Codex-shaped API.

The model receives a small async API inside persistent JavaScript:

• cua.getState(options?)List the current desktop apps and emit bounded output.
• cua.listApps(options?)Return structured app records without emitting by default.
• cua.getApp(nameOrPid)Bind an app and emit its initial accessibility state.
• app.getAXState(options?)Read a fresh semantic tree before choosing an element index.
• app.getScreenshot(options?)Return local image bytes and optionally emit them with nodeRepl.
• app.click(target, options?)Use an element index first, or a bounded coordinate when necessary.
• app.typeText(text)Send literal text through the native runtime.
• app.pressKey(key)Send a named key or supported modifier chord.
• app.performSecondaryAction(index, action)Invoke an advertised accessibility action such as a context action.

Use nodeRepl.write(value) for extra text and await nodeRepl.emitImage(bytesOrDataUrl) for local images. Both are bounded before crossing the worker boundary.

Make risky capabilities explicit.

{
  "backend": "auto",
  "serverName": "cua_repl",
  "timeoutMs": 30000,
  "maxTextChars": 200000,
  "maxImageBytes": 2097152,
  "allowGlobalPointerFallbacks": false
}

• backendauto, native, or a fixed custom command.
• serverNameRename the generated MCP server when a project needs isolation.
• timeoutMsBounds JavaScript evaluation. A timeout resets the worker session.
• maxTextCharsAggregate text budget for one tool result.
• maxImageBytesAggregate image budget for one tool result.
• allowGlobalPointerFallbacksOff by default. Enable only when system-level pointer movement is explicitly intended.

Observe, act, verify.

1. Read getAXState() before selecting an element.
2. Prefer a fresh element index over a remembered coordinate.
3. Batch deterministic actions and the next state read in one js call.
4. After navigation, dialogs, resizing, or rerendering, read state again.
5. Use js_reset when bindings need to be discarded, not when an app needs restarting.

var app = await cua.getApp("Example App");
await app.getAXState();
var state = await app.click(12);
await app.getAXState();
nodeRepl.write(state);

Know what the host can do.

macOS generally needs Accessibility and Screen Recording. Windows needs UI Automation in the interactive desktop session. Linux needs AT-SPI plus the native runtime's input and capture services. A secure desktop, lock screen, UAC surface, or missing permission is not bypassed.

Small surface. Strong boundaries.

• No arbitrary shellThe model-facing API does not expose run_shell or a host command escape.
• Restricted VMFilesystem, process, fetch, dynamic import, code generation, and host built-ins are blocked.
• Worker isolationCPU-bound JavaScript can be terminated without wedging the MCP server.
• Bounded transportInput frames, output text, images, content items, and native responses are bounded.
• Approval preservingConsequential actions still pass through OpenCode and OS permission boundaries.
• Safe defaultsGlobal pointer fallback is disabled until explicitly enabled.

Read SECURITY.md before using the session against banking, password managers, administrator tools, or unattended workloads.

When the session is quiet.