clawtv

# ClawTV

AI-powered Apple TV remote that can see the screen and navigate any app autonomously using vision + remote control.

## When to use (trigger phrases)

Use this skill immediately when the user asks:

- "play [movie/show] on Apple TV"
- "search for [content] on [Netflix/Plex/YouTube/etc]"
- "open [app] on the TV"
- "turn on subtitles" / "go to TV settings"
- "what's playing on Apple TV?"
- "take a screenshot of the TV"
- Any Apple TV navigation or control task

## Quick start

```bash
# AI agent mode — tell it what you want in plain English
python3 ~/Developer/clawtv/clawtv.py do "open Plex and play Fight Club"
python3 ~/Developer/clawtv/clawtv.py do "search for Stranger Things on Netflix"
python3 ~/Developer/clawtv/clawtv.py do "go to settings and turn on subtitles"

# Direct Plex control (instant, no vision loop)
python3 ~/Developer/clawtv/clawtv.py plex play "Fight Club"
python3 ~/Developer/clawtv/clawtv.py plex play "Westworld" -s 2 -e 6

# Manual remote commands
python3 ~/Developer/clawtv/clawtv.py cmd up down select
python3 ~/Developer/clawtv/clawtv.py type "breaking bad"
python3 ~/Developer/clawtv/clawtv.py launch com.plexapp.plex

# Screenshot
python3 ~/Developer/clawtv/clawtv.py screenshot

# Status
python3 ~/Developer/clawtv/clawtv.py playing
```

## Required Credentials

### ANTHROPIC_API_KEY (Required for AI/Vision Mode)

The `do` command (AI agent mode) requires an Anthropic API key to access Claude's vision API for screenshot analysis and navigation decisions.

```bash
export ANTHROPIC_API_KEY=sk-ant-...
```

Or add to your shell profile (~/.zshrc, ~/.bashrc).

**Note:** This is only required for the `do` command. Manual commands (`cmd`, `type`, `launch`) and `plex` direct control do not use the API.

### Plex Credentials (Optional)

For instant Plex playback without the vision loop, add these to `~/.clawtv/config.json`:

```json
{
  "plex_url": "http://192.168.1.100:32400",
  "plex_token": "your-plex-token",
  "plex_client": "Living Room"
}
``...

# ClawTV

**AI-powered Apple TV remote that can see your screen.**

Tell it what you want in plain English and it navigates any app autonomously using vision + remote control. No per-app APIs needed — it works with every app on your Apple TV.

```
python clawtv.py do "open Plex and play Fight Club"
```

ClawTV takes screenshots of your Apple TV, sends them to Claude's vision API to understand what's on screen, then sends remote commands to navigate — just like a person would. It repeats this see-think-act loop until it accomplishes your goal.

## Demo

```
$ python clawtv.py do "find Fight Club on Plex and play it"

--- Step 1 ---
  AI: {"action": "launch", "app": "com.plexapp.plex"}
  -> launched: com.plexapp.plex

--- Step 2 ---
  AI: {"action": "remote", "commands": ["up"]}
  -> up
  (sees Plex sidebar, Search is highlighted)

--- Step 3 ---
  AI: {"action": "remote", "commands": ["select"]}
  -> select
  (sees search screen with keyboard)

--- Step 4 ---
  AI: {"action": "type", "text": "fight club"}
  -> typed: fight club
  (sees search results)

--- Step 5 ---
  AI: {"action": "remote", "commands": ["down", "down"]}
  -> down -> down
  (Fight Club 1999 is highlighted)

--- Step 6 ---
  AI: {"action": "remote", "commands": ["select"]}
  -> select
  (sees movie details page — 1080p, 5.1, R rated)

--- Step 7 ---
  AI: {"action": "remote", "commands": ["select"]}
  -> select

  Done: Fight Club is now playing in 1080p
```

## How It Works

```
┌─────────────┐    screenshot    ┌─────────────┐    analyze     ┌─────────────┐
│  Apple TV    │ ──────────────> │   ClawTV     │ ────────────> │   Claude     │
│  (screen)    │ <────────────── │   (agent)    │ <──────────── │   (vision)   │
└─────────────┘  remote control  └─────────────┘   commands     └─────────────┘
```

1. **See** — Takes a screenshot of the Apple TV
2. **Think** — Sends the screenshot to Claude, which analyzes the UI and decides what to do
3. **Act** — Sends remote commands (navigate, select, type) vi...