Talkr AI Documentation

Scrape a web page and add it to the knowledge base, using the same chunking/embedding pipeline as uploaded files

The URL is validated synchronously (must be http(s) and resolve to a public IP — private/internal addresses are rejected as an SSRF guard) before a pending document record is created; the actual fetch happens in a background job, and every redirect hop it follows is re-validated against the same guard. Poll Get a Document or List Documents to watch it progress from pending to completed or failed, same as an uploaded file.

Header Parameters

authorization?string|null

X-API-Key?string|null

Request Body

application/json

TypeScript Definitions

Use the request body type in TypeScript.

Request schema for scraping a URL into the knowledge base.

url*Url

URL of the web page to fetch and add to the knowledge base

retrieval_mode?Retrieval Mode

Retrieval mode: 'chunked' for vector search or 'full_document' for full text retrieval

Default"chunked"

collection_uuid?|

Collection to add this document to. Defaults to the organization's Default collection when omitted.

max_depth?Max Depth

How many link-levels deep to follow from the given URL. 0 (default) scrapes only the given page. 1 also scrapes pages it directly links to (same hostname only), etc. Silently capped at 5 regardless of the value sent.

Range0 <= value

Default0

max_pages?Max Pages

Maximum total pages to scrape when max_depth > 0. Ignored when max_depth is 0. Silently capped at 50 regardless of the value sent.

Range1 <= value

Default1

Response Body

`application/json`

curl -X POST "https://example.com/api/v1/knowledge-base/documents/from-url" \  -H "Content-Type: application/json" \  -d '{    "url": "string"  }'

{  "id": 0,  "document_uuid": "string",  "filename": "string",  "file_size_bytes": 0,  "file_hash": "string",  "mime_type": "string",  "processing_status": "string",  "processing_error": "string",  "total_chunks": 0,  "retrieval_mode": "chunked",  "custom_metadata": {},  "docling_metadata": {},  "source_url": "string",  "collection_uuid": "string",  "created_at": "2019-08-24T14:15:22Z",  "updated_at": "2019-08-24T14:15:22Z",  "organization_id": 0,  "created_by": 0,  "is_active": true}

Add a Document from a URL

Header Parameters

Request Body

Response Body

200application/json

404

422application/json

`application/json`

`application/json`