---
title: Agent skills
description: Print the instructions that let an AI agent drive a screen-relay desktop from the CLI.
url: "https://screen-relay.dev/docs/agent-skills"
docs_index: /llms.txt
lastUpdated: 2026-11-18
---

> For an index of all documentation, see [/llms.txt](/llms.txt).

Skills are instruction files that ship inside the CLI. The `core` skill tells an agent how to start a session, take screenshots, send input, and recover from an error. Skills match the installed version, and reading them never contacts a session.

## Load the core skill

```sh
screen-relay skills get core
```

An agent should start there. The other skills are listed by `screen-relay skills list`.

- `screen-relay skills list` lists the visible skills. It is the default action.
- `screen-relay skills get <name>` prints one skill's workflow. A hidden skill loads when you name it.
- `screen-relay skills path [name]` prints the skills directory, or the directory of a named skill.

The `--full` option, which adds reference and template files, is covered in the [CLI reference](/docs/cli#skills).

```console
$ screen-relay skills list
core              Control a screen-relay desktop from an agent using screenshots and input commands.
vercel-sandbox    Run and control screen-relay inside a Vercel Sandbox.
```

An unknown name fails with reason `not_found` and suggests `screen-relay skills list`.

With `--json`, `list` returns `{"skills":[...]}`, `get` returns `{"skill":{"name","content"}}`, and `path` returns `{"path":"..."}`.

## Skills

| Skill            | What it covers                                                                                                                                                                                     |
| ---------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `core`           | Where each command runs, starting a session, the screenshot and input loop, coordinates, JSON output, and error recovery. Its reference files cover the CLI, config, Wayland, and troubleshooting. |
| `vercel-sandbox` | Running and controlling screen-relay inside a Vercel Sandbox.                                                                                                                                      |
| `screen-relay`   | Hidden. Points an agent that finds the skills directory to `screen-relay skills get core`.                                                                                                         |

## Agent-aware help

When screen-relay detects an AI agent, `screen-relay --help` opens with a `Start here (for AI agents):` block that lists the skills commands. Each command's help says where it runs. The [CLI reference](/docs/cli#help-layouts) explains how detection works and how to force either layout.

## Agent loop

Start the session on the VM with `screen-relay up -d`. It returns once the desktop and engine are ready. Then repeat: take a screenshot, decide from the image, act, and take another screenshot to verify.

```sh
screen-relay up -d
screen-relay screenshot /tmp/before.png
screen-relay click 0.5 0.5
screen-relay type "echo hello"
screen-relay key Enter
screen-relay screenshot /tmp/after.png
```

- `up` and `run` run on the VM. `status`, `screenshot`, `click`, `move`, `scroll`, `key`, `type`, and `input` reach the engine from anywhere with `--url` and a token file. See [Remote access](/docs/remote-access).
- Coordinates are fractions of the screen, from `0` to `1`, measured from the top-left. Pixel `480,810` on a 1920x1080 screenshot is `screen-relay click 0.25 0.75`. `screen-relay screenshot file.png --json` returns `width` and `height`.
- A screenshot writes a PNG, default `screen.png`, or PNG bytes to stdout with `-`. Success means the engine saved the frame, not that an app responded.
- Click a window to focus it before typing. Use `type` for text, and `key` for shortcuts and Escape.
- Input success means the engine applied the events, not that the application reacted. Take a screenshot to confirm.
- Only one controller sends input at a time. A browser viewer is view-only until someone chooses **Take control** in it, so the CLI can work while a person watches. A viewer that owns input, or a control session, makes input return `busy` with `Another controller holds control of the desktop.` Screenshots still work. Wait for it to release control or close, then retry once.
- `screen-relay run -- APP` returns the app's pid at once, before its window appears. Poll screenshots until the window shows, and read the app's output with `screen-relay logs apps`.

A long `screen-relay type` is split into batches. When one fails:

- `data.applied` in the JSON error counts the events the engine confirmed.
- After `busy`, `invalid_input`, or `unsupported_input`, resend only the remaining text.
- After `timeout`, `unreachable`, or `disconnected`, take a screenshot first. The last batch may already be on screen.

## JSON and errors

Add `--json` to any command. Match `reason` in a failed reply instead of the message, and run the `next` command when one is set. The [CLI reference](/docs/cli#error-reasons) lists every reason with its recovery step.

---

For a semantic overview of all documentation, see [/sitemap.md](/sitemap.md)

For an index of all available documentation, see [/llms.txt](/llms.txt)

For agent-facing discovery, including API and MCP surfaces, see [/agents.md](/agents.md)