Touchpoint

@Touchpoint-Labs

About Touchpoint

Playwright for the entire OS. Give AI agents the ability to see, find, and interact with UI elements in any desktop application — native apps, Electron apps, and browsers. Cross-platform: Linux, macOS, Windows.

#desktop-automation #accessibility #ui-automation #cross-platform #mcp-server #ai #ai-agents

Config

Add this server to your MCP-compatible client using the configuration below.

{
  "mcpServers": {
    "touchpoint": {
      "command": "touchpoint-mcp",
      "args": []
    }
  }
}

Tools

`apps`, `windows`, `find`, `elements`, `get_element`

`screenshot` (returns image data the LLM can see)

`click` (left/right/double), `set_value`, `set_numeric_value`, `focus`, `action`

`type_text`, `press_key` (single key or combo)

`mouse_move`, `scroll`

`activate_window`

`wait_for`, `wait_for_app`, `wait_for_window`

Overview

What is Touchpoint?

Touchpoint is a cross-platform Python library that reads and interacts with desktop UI through native accessibility APIs, shipping with an MCP server so LLM agents can control any desktop app. It provides structured element data (names, roles, states, positions) without needing pixel scraping or vision models.

How to use Touchpoint?

Install with pip install touchpoint-py, then configure the MCP server in your client by adding the touchpoint-mcp command (see client setup details in the README). Use the library API for direct automation: import touchpoint as tp, then call functions like tp.find(), tp.click(), tp.elements(), and tp.wait_for().

Key features of Touchpoint

Cross-platform (Linux, macOS, Windows) via native accessibility backends
Integrates browsers and Electron apps via Chrome DevTools Protocol (CDP)
MCP server with 19 tools for LLM agents (discovery, actions, keyboard, mouse, waiting)
Structured element data: names, roles, states, and positions
Output formats: flat, tree, JSON for LLM-friendly consumption
Built-in wait and action functions for reliable automation

Use cases of Touchpoint

Automating desktop workflows with AI agents (fill forms, extract data)
Letting LLM agents control any desktop app (native, browser, Electron)
Building robust UI automation without OCR or vision models
Cross-platform testing of desktop applications

FAQ from Touchpoint

What platforms does Touchpoint support?

Linux (AT-SPI2), macOS (Accessibility API), and Windows (UI Automation). Platform-specific dependencies are installed automatically via pip environment markers.

Does Touchpoint require vision models or OCR?

No. It reads the real accessibility tree directly, providing fast and reliable structured data without model inference.

How do I use Touchpoint with Claude Desktop?

Add the MCP server to your Claude config file: set command to touchpoint-mcp. If using a virtualenv, use the full path to the binary. The MCP server includes built-in instructions for LLM agents.

Can Touchpoint control browser or Electron apps?

Yes, via Chrome DevTools Protocol (CDP). Launch the browser or Electron app with --remote-debugging-port=9222 and configure CDP discovery or explicit ports. Touchpoint merges native UI and web content in a single elements() call.

What are the runtime requirements?

Python 3.10+. On Linux, xdotool is needed for input (usually pre-installed). On macOS, grant Accessibility permission in System Settings. No dependencies are needed on Windows—it uses built-in COM APIs.

Frequently asked questions

What platforms does Touchpoint support?

Linux (AT-SPI2), macOS (Accessibility API), and Windows (UI Automation). Platform-specific dependencies are installed automatically via pip environment markers.

Does Touchpoint require vision models or OCR?

No. It reads the real accessibility tree directly, providing fast and reliable structured data without model inference.

How do I use Touchpoint with Claude Desktop?

Add the MCP server to your Claude config file: set `command` to `touchpoint-mcp`. If using a virtualenv, use the full path to the binary. The MCP server includes built-in instructions for LLM agents.

Can Touchpoint control browser or Electron apps?

Yes, via Chrome DevTools Protocol (CDP). Launch the browser or Electron app with `--remote-debugging-port=9222` and configure CDP discovery or explicit ports. Touchpoint merges native UI and web content in a single `elements()` call.

What are the runtime requirements?

Python 3.10+. On Linux, `xdotool` is needed for input (usually pre-installed). On macOS, grant Accessibility permission in System Settings. No dependencies are needed on Windows—it uses built-in COM APIs.