Tools Backend · cold-reader map

A tool call is a guarded HTTP hop from the LLM runtime into a Cloudflare Worker registry.

Read it as six boxes: caller -> edge route -> middleware -> Hono router -> tool handler -> service binding.

Production route

https://tool.fpai.io

Test route

https://tool-test.fpai.io

Dev route resolution

localhost:8787 or workers.dev

caller

skill_toolcall_curl parses curl, injects auth, rewrites known hosts for runtime.

worker

app.ts gates every request with request-id, timing, CORS, auth.

registry

registry.ts maps tool names and aliases to typed handlers.

01 · mental model

01 One request path, six responsibilities

The system is easiest to understand as a conveyor belt. Each stage adds one guardrail or turns the request into a typed tool result.

Callercurl parser

Edgeroute host

Middleware4 gates

Router/:toolName

HandlerToolDefinition

Bindingupstream service

The LLM sees stable URLs; the backend can transparently rewrite prod/test hosts for dev, test, or local Wrangler.
The Worker never runs a free-form command; every public tool name resolves through a registry entry.
Responses are normalized into data, batch, or async polling shapes before returning to the caller.

02 · request path diagram

02 The graph: from curl text to service binding

Each box is tied to a source file and one request-path responsibility.

runtime caller

skill_toolcall_curl

https only

edge

route host

prod/test/dev

middleware 1

request-id

X-Request-Id

middleware 2

timing

durationMs

middleware 3

cors

optional

middleware 4

auth

IGNISAUTH first

hono

app.ts

/:toolName

dispatch

registry

toolMap

handler

Rendi frames

batch result

handler

Gemini analyse

data result

dependencies

bindings

services + KV + R2

app.use order verified in app.ts tool aliases verified in registry.ts bindings verified in wrangler.toml

03 · middleware chain detail

03 Four gates run before any handler

Source-verified order from app.ts: request-id, timing, optional CORS, then auth.

request-id

Reads X-Request-Id when present.
Falls back to crypto.randomUUID().
Stored in Hono context for response meta.

timing

Captures wall-clock duration with Date.now().
Logs method, path, tool, status, durationMs.
Redacts email unless LOG_USER_EMAIL=true.

cors

Created only when CORS_ORIGINS exists.
Allows GET, POST, OPTIONS.
Allows Content-Type, Authorization, request id, worker key.

auth

Bypasses OPTIONS and public health paths.
Prefers IGNISAUTH JWT.
Falls back to static worker token.

Short-circuit points: unsupported content-type returns 415; unknown tools return TOOL_NOT_FOUND; missing async KV returns ASYNC_NOT_CONFIGURED.

04 · auth gate

04 Preferred identity is signed IGNISAUTH; token auth is only fallback

The backend caller can mint a short-lived RS256 JWT. The Worker validates issuer, audience, and email before setting request context.

Primary path: IGNISAUTH

Header name is IGNISAUTH.
Key source is MAI_IGNISAUTH_PUBLIC_KEY_PEM.
Expected issuer is mai-backend.
Expected audience is tool.fpai.io.

jwtVerify(token, key, { issuer, audience }) payload.email -> c.set("userEmail", email)

Fallback path: static token

Reads TOOLS_BACKEND_TOKENS or one TOOLS_BACKEND_TOKEN.
Accepts Authorization: Bearer ....
Also accepts X-Worker-Auth-Key.
Returns UNAUTHORIZED on missing or invalid token.

expectedTokens.includes(provided) AUTH_DISABLED=true bypasses in dev

session_id canvas_id turn_id env

05 · routing and registry

05 app.ts validates the request; registry.ts chooses the handler

A tool name is just a URL segment until getTool() resolves it to a ToolDefinition.

Route	Behavior	Error shape
`GET /`	Returns service name plus tool metadata.	None on happy path.
`GET /tools`	Returns `listTools().map(t => t.meta)`.	None on happy path.
`GET /tools/:toolName`	Returns one metadata object from `getTool()`.	`TOOL_NOT_FOUND`
`POST /:toolName`	Checks content-type, builds ctx, invokes `tool.run()`.	`UNSUPPORTED_MEDIA_TYPE`
`GET /tasks/:taskId`	Loads TASKS_KV and calls `tool.poll()` when needed.	`TASK_NOT_FOUND`

Main alias extract-frame -> extractFramesRendiTool

Typo-compatible alias extra-frame -> extractFramesRendiTool

Legacy path extract-frame-legacy -> extractFrameTool

extract-frameextract-frames-rendimedia-analyseadd-subtitleaudio-utilsvoice-cloneaudio-isolationvoiceover-clonegenerate-speech-clonefetch-social-videounified-utilsvideo-split3d-modelvideo-probeseedance-preprocess

06 · tool handlers

06 Two concrete handlers show the result contract

extract-frame returns visible batch media. media-analyse returns text data from Gemini through the Vertex proxy.

extract-frame via Rendi FFmpeg

Accepts video_url, url, or video_id.
Requires non-empty numeric timestamps.
Polls Rendi up to 60 attempts at 2000 ms.
Uploads each output URL through MEDIA_UPLOADER.

buildFfmpegCommand(timestamps)

POST /run-ffmpeg-command

GET /commands/:command_id

uploadUrlToR2AsFileId(...)

kind: "batch"
can_show: true
successful: [{ file_id, timestamp, width, height }]
failed: [{ error, message }]

media-analyse via Vertex / Gemini

Defaults task to analyze.
Supports query with user_query, query, or question.
Resolves URL or provided GCS URI before model call.
Maps Vertex 429 to a 429 upstream error.

resolveGcsUri(videoUrl, gcsUri)

buildAnalyzeOptions or buildQueryOptions

callVertexProxy(proxyOptions)

parseProxyResponse(resultJson)

kind: "data"
can_show: false
data: { text } or { answer }
meta: { task, model_id }

07 · bindings and source notes

07 Bindings are the Worker-side dependency map

wrangler.toml keeps the public Worker small: handlers call named service bindings instead of embedding upstream URLs everywhere.

Binding	Type	Used for	Primary section
`GCS_FILE_UPLOADER`	service	Prepare media for Gemini / GCS workflows.	media-analyse helpers
`VERTEX_AI_PROXY`	service	Gemini video analysis and Q&A proxy.	media-analyse
`MEDIA_UPLOADER`	service	Turn returned frame URLs into durable file_ids.	extract-frame
`CLOUDINARY_VIDEO_PROCESSOR`	service	Legacy extract-frame implementation path.	extract-frame-legacy
`UNIFIED_GENERATION_API`	service	Legacy and shared video generation paths.	other handlers
`VIDEO_CACHE_KV`	KV	Cache source URL to GCS URI mappings.	media pipeline
`TASKS_KV`	KV	Store async task status and poll data.	/tasks/:taskId
`R2_CDN`	R2	Shared CDN storage bucket for media outputs.	file result delivery

Source files checked

tools-backend/app.ts
tools-backend/registry.ts
tools-backend/wrangler.toml
tools-backend/middleware/auth.ts
backend/tools/skill_toolcall_curl.py

* The important invariant: the LLM gets a simple URL, but the backend keeps authority in code paths: host allowlist, auth injection, middleware gates, registry dispatch, typed handler results.