> ## Documentation Index
> Fetch the complete documentation index at: https://figranium.dev/docs/llms.txt
> Use this file to discover all available pages before exploring further.
# CAPTCHA Solving
> Solve reCAPTCHA v2/v3, hCaptcha, and Cloudflare Turnstile challenges from Agent mode using the built-in local solver, with an optional remote endpoint.
Agent mode can detect and solve CAPTCHA challenges without leaving the task flow. Figranium ships a **resource-adaptive local solver** that runs an OWL-ViT or Florence-2 model against the task's own headless browser and injects the resulting token. If you configure a YesCaptcha/AntiCaptcha-compatible remote endpoint, the block tries it first and falls back to the local solver on failure.
Supported challenges: reCAPTCHA v2, reCAPTCHA v3, hCaptcha, Cloudflare Turnstile.
You have two ways to invoke it:
* An explicit **Solve Captcha** block that you drop at the exact step in your flow where a challenge appears.
* A task-level **Auto-Solve Captchas** toggle that runs the same detection pass after every `navigate`, `click`, and `type` action, for tasks that hit CAPTCHAs at unpredictable points.
Both routes use the same underlying solver and are only available in Agent mode.
***
## How solving works
When the block runs, Figranium:
Scans the page (or the container you scoped it to) for a supported widget and reads its site key, action, and other parameters. If nothing is found, the block errors out with `no CAPTCHA challenge found on the page`.
If `CAPTCHA_SOLVER_URL` is set, submits a task to that endpoint and waits up to `CAPTCHA_REMOTE_TIMEOUT_MS` for a token. Failures are logged and the block falls through to the local solver.
Solves the challenge inside the task's own browser session using the selected model tier. For reCAPTCHA v2 and hCaptcha image challenges, a grid engine handles 3×3 and 4×4 layouts and replacement tiles. Turnstile and checkbox-only test-key flows are model-free.
Writes the token into the page's response field and fires the site's own callback so the form thinks a real user solved it.
Local image solving requires at least **2 GiB effective cgroup memory**. Hosts below that threshold can still use a configured remote endpoint; set `SKIP_LOCAL_CAPTCHA_MODEL=true` to disable local probing entirely.
***
## The `solve_captcha` block
Drop this block into a task at the step where you expect a challenge. See [Action Blocks — Solve Captcha](/docs/action-blocks#solve-captcha) for the full field reference; the shape is:
```json theme={null}
{
"type": "solve_captcha",
"captchaType": "recaptcha_v2",
"selector": "#recaptcha-container",
"varName": "captchaResult",
"timeout": 120000
}
```
* `captchaType` is optional — omit it to auto-detect from the page. Supported values: `recaptcha_v2`, `recaptcha_v3`, `hcaptcha`, `turnstile`.
* `selector` is optional — omit it to scan the whole page.
* `varName` stores `{ success, challenge, duration, provider, model?, device?, attempts }` so you can log or branch on the outcome.
* `timeout` is the terminal deadline in milliseconds (default `120000`). Provider errors are returned immediately rather than being reported as timeouts.
**Example — submit a form, then solve the reCAPTCHA it renders:**
```json theme={null}
[
{ "type": "click", "value": "button[type='submit']" },
{ "type": "wait_selector", "value": "iframe[src*='recaptcha']" },
{
"type": "solve_captcha",
"captchaType": "recaptcha_v2",
"varName": "captchaResult"
},
{ "type": "click", "value": "button.continue" }
]
```
***
## Auto-Solve Captchas (task setting)
Toggle **Auto-Solve Captchas** in the task settings **Behavior** tab. When enabled, Figranium runs the same detection pass automatically after every `navigate`, `click`, and `type` action in that task.
* Default: **off**. Tasks that don't need CAPTCHA handling pay no extra latency.
* When no challenge is present on the page, the auto-pass silently no-ops.
* Auto-solve and explicit `solve_captcha` blocks work together — turning auto-solve on doesn't disable your explicit blocks.
Use this when you don't know exactly where a CAPTCHA will appear in a flow (for example, sites that rate-limit into a challenge only sometimes). If the CAPTCHA is always at the same step, prefer an explicit block for readability.
***
## Local solver models
The local solver auto-selects a model tier based on host memory. Weights are fetched on first use into persistent `data/captcha-model/`; nothing is bundled into the Docker image. Every artifact is pinned to an exact upstream commit, size, and SHA-256 digest, and inference loads with remote access disabled.
| **Tier** | **Auto-selected when** | **Artifact size** |
| :------------- | :-------------------------- | :---------------- |
| **OWL-ViT** | 2–7.99 GiB effective memory | \~159 MB |
| **Florence-2** | 8 GiB+ effective memory | \~361 MB |
Only the active tier is retained at steady state.
### Optional Apple Silicon companion
On Apple Silicon (Docker Desktop or native macOS), an optional companion process runs inference on Apple unified memory and CoreML/MLX acceleration, keeping model weights out of the container image:
```bash theme={null}
npm run captcha:companion:install # pinned Python 3.10+ venv + generated token
npm run captcha:companion:start # native Figranium, loopback only
npm run captcha:companion:start:docker # Docker Desktop, authenticated external bind
```
OWL-ViT uses ONNX Runtime's CoreML execution provider. Florence-2 stages a pinned 4-bit MLX build and probes it; if MLX inference fails, the companion falls back to a verified native ONNX build automatically. The companion exposes only authenticated `GET /v1/health` and `POST /v1/detect`, caps request and image sizes, serializes inference, and applies inference timeouts.
Docker Desktop discovers the companion at `http://host.docker.internal:11438`; native macOS uses loopback. Override with `CAPTCHA_COMPANION_URL` if needed. The bearer token is shared through the `data/` mount by default, or set `CAPTCHA_COMPANION_TOKEN` explicitly.
***
## Optional remote endpoint
The local solver works standalone. If you have a YesCaptcha/AntiCaptcha-compatible endpoint, set these variables to try it first and fall back to the local solver on failure.
| **Variable** | **Default** | **Description** |
| :------------------------------------------- | :---------------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `CAPTCHA_SOLVER_URL` | — | YesCaptcha/AntiCaptcha-compatible endpoint. When set, remote solving is attempted first. |
| `CAPTCHA_SOLVER_KEY` | — | Client key for `CAPTCHA_SOLVER_URL`. |
| `CAPTCHA_REMOTE_TIMEOUT_MS` | action deadline minus local reserve | Maximum time allocated to the remote route before falling back locally. |
| `CAPTCHA_LOCAL_FALLBACK_MIN_MS` | `15000` | Portion of the action deadline reserved for the local route. |
| `CAPTCHA_REMOTE_FORWARD_PROXY` | `false` | Send the active browser proxy and user agent to compatible proxy-backed remote tasks. Unsupported provider/task combinations fail over locally without changing IP. |
| `CAPTCHA_REMOTE_FORWARD_CONTEXT` | `false` | Send origin-scoped cookies, locale, timezone, viewport, and user agent to a custom endpoint that advertises `browserContext` version 1. |
| `OHMYCAPTCHA_URL` / `OHMYCAPTCHA_CLIENT_KEY` | — | Deprecated aliases for `CAPTCHA_SOLVER_URL` / `CAPTCHA_SOLVER_KEY`. |
Standard AntiCaptcha/YesCaptcha payloads never receive cookies. Proxy and browser-context forwarding are disabled by default because they disclose sensitive connection and session data to the configured endpoint; both are separate explicit opt-ins. Context is only sent to custom endpoints advertising `browserContext` version 1, and cookies are filtered to the active page origin. Secrets are redacted from terminal errors and logs.
***
## Local solver tuning
| **Variable** | **Default** | **Description** |
| :---------------------------- | :---------------- | :--------------------------------------------------------------------------------------------------------------------------- |
| `CAPTCHA_MODEL_TIER` | `auto` | Model selection: `auto`, `owlvit`, or `florence2`. Forcing `florence2` requires 8 GiB+ effective memory and 2 GiB available. |
| `CAPTCHA_MODEL_DEVICE` | `auto` | Inference device policy: `auto` or `cpu`. |
| `SKIP_LOCAL_CAPTCHA_MODEL` | `false` | Unconditionally disable local model detection, download, startup, reconciliation, and fallback. Cached weights are retained. |
| `CAPTCHA_OWLVIT_THRESHOLD` | calibrated `0.12` | Optional OWL-ViT confidence override in the range 0–1. |
| `CAPTCHA_FLORENCE2_THRESHOLD` | calibrated `0.18` | Optional Florence-2 confidence override in the range 0–1. |
**Docker Compose example:**
```yaml docker-compose.yml theme={null}
services:
figranium:
environment:
# Optional: try a remote solver first, fall back to the local model
# - CAPTCHA_SOLVER_URL=https://solver.example.com
# - CAPTCHA_SOLVER_KEY=your-client-key
# Local model tuning (defaults work on most hosts)
# - CAPTCHA_MODEL_TIER=auto # auto|owlvit|florence2
# - CAPTCHA_MODEL_DEVICE=auto # auto|cpu
# - SKIP_LOCAL_CAPTCHA_MODEL=true # disable local entirely
# Deadline budgeting between remote and local routes
# - CAPTCHA_REMOTE_TIMEOUT_MS=45000
# - CAPTCHA_LOCAL_FALLBACK_MIN_MS=15000
```
***
## Return value
If you set `varName`, the variable holds:
```json theme={null}
{
"success": true,
"challenge": "recaptcha_v2",
"duration": 42317,
"provider": "local",
"model": "owlvit",
"device": "cpu",
"attempts": [
{ "provider": "remote", "status": "unavailable", "duration": 0, "error": "CAPTCHA_SOLVER_URL is not configured" },
{ "provider": "local", "status": "success", "duration": 42200 }
]
}
```
* `success`: `true` when the token was injected, `false` otherwise.
* `challenge`: The detected challenge type (`recaptcha_v2`, `recaptcha_v3`, `hcaptcha`, or `turnstile`).
* `duration`: Milliseconds the solve took end-to-end.
* `provider`: Which route produced the token (`remote` or `local`).
* `model`, `device`: Set when the local solver produced the token (`owlvit`/`florence2`, `cpu`/`coreml`/`companion`, ...).
* `attempts`: Per-route routing outcomes (no credentials or tokens included).
Use this to log solve times, or to branch with an [If](/docs/control-flow) block on `{$captchaResult.success}`.
***
## Troubleshooting
Both the remote endpoint (if configured) and the local solver failed. Check `attempts` in the return value for the per-route error. Common causes: `CAPTCHA_SOLVER_URL` unreachable, host memory below 2 GiB with `SKIP_LOCAL_CAPTCHA_MODEL` implicitly disabling local, or a challenge variant not yet supported.
Local image solving requires at least 2 GiB effective cgroup memory. On smaller hosts, configure a remote endpoint via `CAPTCHA_SOLVER_URL` / `CAPTCHA_SOLVER_KEY`.
Florence-2 requires 8 GiB+ effective memory and 2 GiB available. Drop back to `auto` or `owlvit`.
Raise the block's `timeout` field (default `120000`). reCAPTCHA v2 image challenges routinely take longer than a minute. If you're mostly hitting the remote route, adjust `CAPTCHA_REMOTE_TIMEOUT_MS` and `CAPTCHA_LOCAL_FALLBACK_MIN_MS` to reallocate the budget.
Verify the companion is running (`npm run captcha:companion:start` or the `:docker` variant), that `data/captcha-companion-token` is shared between the container and the companion (or `CAPTCHA_COMPANION_TOKEN` is set on both sides), and that `CAPTCHA_COMPANION_URL` points at a reachable host — Docker Desktop uses `http://host.docker.internal:11438`.
Only solve CAPTCHAs on sites you own or have explicit permission to automate. Solving third-party CAPTCHAs may violate Terms of Service and applicable laws in your jurisdiction.