AI Extraction Script Generation

​

UI Workflow

1. Open your task in the Spatial Editor.
2. Click the Extraction Script block at the bottom of the canvas.
3. In the modal toolbar, click the Generate button (represented by a sparkle icon).
4. Enter a plain-language prompt describing your target data — for example:
   • “Extract all product titles, current prices, original prices, and star ratings as an array of objects.”
   • “Get the article headline, author name, publication date, and main body text.”
5. Click Generate or press Enter.
6. The AI generates the JavaScript code and inserts it directly into the extraction script code editor.
7. Review the generated script, edit if necessary, and save your task.

​

API Endpoint: POST /api/tasks/generate-script

​

Request

POST /api/tasks/generate-script HTTP/1.1
Content-Type: application/json
Authorization: Bearer <your-api-key>

{
  "description": "Extract table rows containing order ID, customer name, total amount, and order status."
}

​

Response

{
  "script": "return Array.from(document.querySelectorAll('table tr.order-row')).map(row => ({\n  orderId: row.querySelector('.order-id')?.textContent?.trim() || '',\n  customer: row.querySelector('.customer-name')?.textContent?.trim() || '',\n  total: row.querySelector('.total-amount')?.textContent?.trim() || '',\n  status: row.querySelector('.status-badge')?.textContent?.trim() || ''\n}));"
}

​

Error Responses

• 400 Bad Request: Missing description parameter or no AI API keys configured.
• 502 Bad Gateway: All configured AI providers failed to generate a script.

​

Prompt Rules and Execution Constraints

1. Explicit Return: The script must always use the return statement to output structured data (a single JSON object or an array of JSON objects).
2. Standard DOM APIs: Uses standard browser DOM methods (document.querySelector, document.querySelectorAll, getAttribute, textContent, etc.).
3. No Function Definition Wrapping: The script body is generated directly without being wrapped inside function() { ... } or () => { ... }.
4. No Top-Level Async/Await: page.evaluate() executes synchronously within page scope; top-level async/await is omitted.
5. Clean Code Output: Figranium automatically strips markdown code fences (```javascript) from the AI provider’s raw output before inserting it into the editor or API response.

​

AI Provider Architecture & Fallback Order

Gemini  ───►  OpenAI  ───►  Claude  ───►  Ollama
(v1beta)     (gpt-4o)    (claude-3)    (local)

1. Google Gemini: Tried first using the configured Gemini key and model (gemini-2.0-flash or gemini-1.5-flash).
2. OpenAI: Used if Gemini fails or is unconfigured (gpt-4o-mini or custom OpenAI model).
3. Anthropic Claude: Used if OpenAI fails or is unconfigured (claude-3-5-haiku or custom Claude model).
4. Ollama: Used as the final fallback for self-hosted, local LLM execution.

​

Configuring AI Providers

• Navigate to Settings > System > API Keys and add at least one provider key (Gemini, OpenAI, Claude, or Ollama endpoint).
• Configure preferred default models in Settings > System > AI Models.

​

DOM Context & Extraction Compatibility

• DOM Cleaning: Figranium cleans non-essential DOM tags (<script>, <style>, <svg>, <iframe>) before passing the page to the extraction context. This keeps memory usage low and execution fast.
• Shadow DOM: If enabled, Shadow DOM boundaries are preserved as <template data-shadowroot="open">, allowing generated scripts to traverse shadow roots seamlessly.
• Output Formats:
  • JSON: If your task format is set to json, the returned object or array is formatted directly as JSON.
  • CSV: If your task format is set to csv, ensure the generated script returns an Array of Objects. Figranium will convert the keys into CSV column headers automatically.

​

Examples of Prompts & Generated Output

​

Example 1: E-Commerce Product List

“Extract all product cards on the page into an array with title, price, image link, and product URL.”

return Array.from(document.querySelectorAll('.product-card')).map(card => ({
  title: card.querySelector('.product-title')?.textContent?.trim() || '',
  price: card.querySelector('.price-tag')?.textContent?.trim() || '',
  image: card.querySelector('img')?.src || '',
  url: card.querySelector('a')?.href || ''
}));

​

Example 2: Single Article Metadata

“Get the article title, author name, publish date, and full reading time.”

return {
  title: document.querySelector('h1.article-title')?.textContent?.trim() || '',
  author: document.querySelector('.byline .author-name')?.textContent?.trim() || '',
  publishedAt: document.querySelector('time')?.getAttribute('datetime') || document.querySelector('.publish-date')?.textContent?.trim() || '',
  readingTime: document.querySelector('.read-time')?.textContent?.trim() || ''
};

​

Tips for Better Script Generation

1. Be Specific About Selectors or Context: If you know class names or IDs on the target page, mention them in your prompt (e.g., “Extract rows inside #results-table”).
2. Specify Target Data Types: Clarify if you want clean text, attribute values (like href or src), or numbers.
3. Specify List vs. Single Object: Indicate whether you are extracting a list ("an array of objects...") or a single record ("an object containing...").