CAPTCHA Solving

Agent mode can detect and solve CAPTCHA challenges without leaving the task flow. Figranium ships a resource-adaptive local solver that runs an OWL-ViT or Florence-2 model against the task’s own headless browser and injects the resulting token. If you configure a YesCaptcha/AntiCaptcha-compatible remote endpoint, the block tries it first and falls back to the local solver on failure. Supported challenges: reCAPTCHA v2, reCAPTCHA v3, hCaptcha, Cloudflare Turnstile. You have two ways to invoke it:

An explicit Solve Captcha block that you drop at the exact step in your flow where a challenge appears.
A task-level Auto-Solve Captchas toggle that runs the same detection pass after every navigate, click, and type action, for tasks that hit CAPTCHAs at unpredictable points.

Both routes use the same underlying solver and are only available in Agent mode.

How solving works

When the block runs, Figranium:

Detect the challenge

Scans the page (or the container you scoped it to) for a supported widget and reads its site key, action, and other parameters. If nothing is found, the block errors out with no CAPTCHA challenge found on the page.

Try the remote endpoint (if configured)

If CAPTCHA_SOLVER_URL is set, submits a task to that endpoint and waits up to CAPTCHA_REMOTE_TIMEOUT_MS for a token. Failures are logged and the block falls through to the local solver.

Run the local active-browser solver

Solves the challenge inside the task’s own browser session using the selected model tier. For reCAPTCHA v2 and hCaptcha image challenges, a grid engine handles 3×3 and 4×4 layouts and replacement tiles. Turnstile and checkbox-only test-key flows are model-free.

Inject the token

Writes the token into the page’s response field and fires the site’s own callback so the form thinks a real user solved it.

Local image solving requires at least 2 GiB effective cgroup memory. Hosts below that threshold can still use a configured remote endpoint; set SKIP_LOCAL_CAPTCHA_MODEL=true to disable local probing entirely.

The `solve_captcha` block

Drop this block into a task at the step where you expect a challenge. See Action Blocks — Solve Captcha for the full field reference; the shape is:

{
  "type": "solve_captcha",
  "captchaType": "recaptcha_v2",
  "selector": "#recaptcha-container",
  "varName": "captchaResult",
  "timeout": 120000
}

captchaType is optional — omit it to auto-detect from the page. Supported values: recaptcha_v2, recaptcha_v3, hcaptcha, turnstile.
selector is optional — omit it to scan the whole page.
varName stores { success, challenge, duration, provider, model?, device?, attempts } so you can log or branch on the outcome.
timeout is the terminal deadline in milliseconds (default 120000). Provider errors are returned immediately rather than being reported as timeouts.

Example — submit a form, then solve the reCAPTCHA it renders:

[
  { "type": "click", "value": "button[type='submit']" },
  { "type": "wait_selector", "value": "iframe[src*='recaptcha']" },
  {
    "type": "solve_captcha",
    "captchaType": "recaptcha_v2",
    "varName": "captchaResult"
  },
  { "type": "click", "value": "button.continue" }
]

Auto-Solve Captchas (task setting)

Toggle Auto-Solve Captchas in the task settings Behavior tab. When enabled, Figranium runs the same detection pass automatically after every navigate, click, and type action in that task.

Default: off. Tasks that don’t need CAPTCHA handling pay no extra latency.
When no challenge is present on the page, the auto-pass silently no-ops.
Auto-solve and explicit solve_captcha blocks work together — turning auto-solve on doesn’t disable your explicit blocks.

Use this when you don’t know exactly where a CAPTCHA will appear in a flow (for example, sites that rate-limit into a challenge only sometimes). If the CAPTCHA is always at the same step, prefer an explicit block for readability.

Local solver models

The local solver auto-selects a model tier based on host memory. Weights are fetched on first use into persistent data/captcha-model/; nothing is bundled into the Docker image. Every artifact is pinned to an exact upstream commit, size, and SHA-256 digest, and inference loads with remote access disabled.

Tier	Auto-selected when	Artifact size
OWL-ViT	2–7.99 GiB effective memory	~159 MB
Florence-2	8 GiB+ effective memory	~361 MB

Only the active tier is retained at steady state.

Optional Apple Silicon companion

On Apple Silicon (Docker Desktop or native macOS), an optional companion process runs inference on Apple unified memory and CoreML/MLX acceleration, keeping model weights out of the container image:

npm run captcha:companion:install       # pinned Python 3.10+ venv + generated token
npm run captcha:companion:start         # native Figranium, loopback only
npm run captcha:companion:start:docker  # Docker Desktop, authenticated external bind

OWL-ViT uses ONNX Runtime’s CoreML execution provider. Florence-2 stages a pinned 4-bit MLX build and probes it; if MLX inference fails, the companion falls back to a verified native ONNX build automatically. The companion exposes only authenticated GET /v1/health and POST /v1/detect, caps request and image sizes, serializes inference, and applies inference timeouts. Docker Desktop discovers the companion at http://host.docker.internal:11438; native macOS uses loopback. Override with CAPTCHA_COMPANION_URL if needed. The bearer token is shared through the data/ mount by default, or set CAPTCHA_COMPANION_TOKEN explicitly.

Optional remote endpoint

The local solver works standalone. If you have a YesCaptcha/AntiCaptcha-compatible endpoint, set these variables to try it first and fall back to the local solver on failure.

Variable	Default	Description
`CAPTCHA_SOLVER_URL`	—	YesCaptcha/AntiCaptcha-compatible endpoint. When set, remote solving is attempted first.
`CAPTCHA_SOLVER_KEY`	—	Client key for `CAPTCHA_SOLVER_URL`.
`CAPTCHA_REMOTE_TIMEOUT_MS`	action deadline minus local reserve	Maximum time allocated to the remote route before falling back locally.
`CAPTCHA_LOCAL_FALLBACK_MIN_MS`	`15000`	Portion of the action deadline reserved for the local route.
`CAPTCHA_REMOTE_FORWARD_PROXY`	`false`	Send the active browser proxy and user agent to compatible proxy-backed remote tasks. Unsupported provider/task combinations fail over locally without changing IP.
`CAPTCHA_REMOTE_FORWARD_CONTEXT`	`false`	Send origin-scoped cookies, locale, timezone, viewport, and user agent to a custom endpoint that advertises `browserContext` version 1.
`OHMYCAPTCHA_URL` / `OHMYCAPTCHA_CLIENT_KEY`	—	Deprecated aliases for `CAPTCHA_SOLVER_URL` / `CAPTCHA_SOLVER_KEY`.

Standard AntiCaptcha/YesCaptcha payloads never receive cookies. Proxy and browser-context forwarding are disabled by default because they disclose sensitive connection and session data to the configured endpoint; both are separate explicit opt-ins. Context is only sent to custom endpoints advertising browserContext version 1, and cookies are filtered to the active page origin. Secrets are redacted from terminal errors and logs.

Local solver tuning

Variable	Default	Description
`CAPTCHA_MODEL_TIER`	`auto`	Model selection: `auto`, `owlvit`, or `florence2`. Forcing `florence2` requires 8 GiB+ effective memory and 2 GiB available.
`CAPTCHA_MODEL_DEVICE`	`auto`	Inference device policy: `auto` or `cpu`.
`SKIP_LOCAL_CAPTCHA_MODEL`	`false`	Unconditionally disable local model detection, download, startup, reconciliation, and fallback. Cached weights are retained.
`CAPTCHA_OWLVIT_THRESHOLD`	calibrated `0.12`	Optional OWL-ViT confidence override in the range 0–1.
`CAPTCHA_FLORENCE2_THRESHOLD`	calibrated `0.18`	Optional Florence-2 confidence override in the range 0–1.

Docker Compose example:

docker-compose.yml

services:
  figranium:
    environment:
      # Optional: try a remote solver first, fall back to the local model
      # - CAPTCHA_SOLVER_URL=https://solver.example.com
      # - CAPTCHA_SOLVER_KEY=your-client-key

      # Local model tuning (defaults work on most hosts)
      # - CAPTCHA_MODEL_TIER=auto        # auto|owlvit|florence2
      # - CAPTCHA_MODEL_DEVICE=auto      # auto|cpu
      # - SKIP_LOCAL_CAPTCHA_MODEL=true  # disable local entirely

      # Deadline budgeting between remote and local routes
      # - CAPTCHA_REMOTE_TIMEOUT_MS=45000
      # - CAPTCHA_LOCAL_FALLBACK_MIN_MS=15000

Return value

If you set varName, the variable holds:

{
  "success": true,
  "challenge": "recaptcha_v2",
  "duration": 42317,
  "provider": "local",
  "model": "owlvit",
  "device": "cpu",
  "attempts": [
    { "provider": "remote", "status": "unavailable", "duration": 0, "error": "CAPTCHA_SOLVER_URL is not configured" },
    { "provider": "local", "status": "success", "duration": 42200 }
  ]
}

success: true when the token was injected, false otherwise.
challenge: The detected challenge type (recaptcha_v2, recaptcha_v3, hcaptcha, or turnstile).
duration: Milliseconds the solve took end-to-end.
provider: Which route produced the token (remote or local).
model, device: Set when the local solver produced the token (owlvit/florence2, cpu/coreml/companion, …).
attempts: Per-route routing outcomes (no credentials or tokens included).

Use this to log solve times, or to branch with an If block on {$captchaResult.success}.

Troubleshooting

Block errors with 'no solver route succeeded'

Both the remote endpoint (if configured) and the local solver failed. Check attempts in the return value for the per-route error. Common causes: CAPTCHA_SOLVER_URL unreachable, host memory below 2 GiB with SKIP_LOCAL_CAPTCHA_MODEL implicitly disabling local, or a challenge variant not yet supported.

Local solver never runs on a small host

Local image solving requires at least 2 GiB effective cgroup memory. On smaller hosts, configure a remote endpoint via CAPTCHA_SOLVER_URL / CAPTCHA_SOLVER_KEY.

Forcing CAPTCHA_MODEL_TIER=florence2 fails to start

Florence-2 requires 8 GiB+ effective memory and 2 GiB available. Drop back to auto or owlvit.

Solve times out at the default deadline

Raise the block’s timeout field (default 120000). reCAPTCHA v2 image challenges routinely take longer than a minute. If you’re mostly hitting the remote route, adjust CAPTCHA_REMOTE_TIMEOUT_MS and CAPTCHA_LOCAL_FALLBACK_MIN_MS to reallocate the budget.

Apple Silicon companion not detected

Verify the companion is running (npm run captcha:companion:start or the :docker variant), that data/captcha-companion-token is shared between the container and the companion (or CAPTCHA_COMPANION_TOKEN is set on both sides), and that CAPTCHA_COMPANION_URL points at a reachable host — Docker Desktop uses http://host.docker.internal:11438.

Only solve CAPTCHAs on sites you own or have explicit permission to automate. Solving third-party CAPTCHAs may violate Terms of Service and applicable laws in your jurisdiction.

Intro

Installation

Basics

Features

Agent Logic

API

JavaScript SDK

Integrations

Ops

Changelog

How solving works

The `solve_captcha` block

Auto-Solve Captchas (task setting)

Local solver models

Optional Apple Silicon companion

Optional remote endpoint

Local solver tuning

Return value

Troubleshooting

​How solving works

​The solve_captcha block

​Auto-Solve Captchas (task setting)

​Local solver models

​Optional Apple Silicon companion

​Optional remote endpoint

​Local solver tuning

​Return value

​Troubleshooting

How solving works

The `solve_captcha` block

Auto-Solve Captchas (task setting)

Local solver models

Optional Apple Silicon companion

Optional remote endpoint

Local solver tuning

Return value

Troubleshooting