Remote Tool Calling
Remote Tool Calling (RTC) is the core protocol of RTC Agent. It reverses the traditional invocation direction: instead of the frontend calling backend APIs, AI calls frontend tools — file read/write, script execution, and business API calls all happen within the user’s browser.
Why RTC
Section titled “Why RTC”| Traditional | RTC | |
|---|---|---|
| Tool Execution | Server | Frontend |
| Data Flow | To the cloud 🔒 | Stays on user’s device 🔐 |
| Privacy | Data passes through server | End-to-end, data never leaves the browser |
| Observability | Black box | Fully visible to the user |
Full Lifecycle
Section titled “Full Lifecycle”A single RTC call goes through the following steps:
💡 Key Design: After pushing the RTC, the server suspends the Turn and saves a Checkpoint to Redis. Even if the server restarts, it can recover from the Checkpoint — the user is completely unaware.
Built-in Tools
Section titled “Built-in Tools”The RTC protocol defines 8 built-in tools covering file system operations, precise editing, and script execution:
| Tool | Function | Key Parameters | Status |
|---|---|---|---|
🔍 ls | List directory contents | path (default /) | Available |
📖 read | Read file contents | path, offset / limit (pagination) | Available |
✏️ write | Write to file (full overwrite) | path, content | Available |
🔧 edit | Precise text replacement | path, old_string, new_string, replace_all | Available |
🔎 grep | Search by content | pattern (regex), path | Available |
📁 find | Search by name | pattern (glob), path | Available |
⚡ script | Execute JavaScript | action (save / run / eval), code | Available |
💬 askUser | Ask user questions (interrupts turn) | questions (1-4 multiple choice) | Available |
💡 The
writetool performs full-content overwrites. For partial edits, use theedittool (precise string replacement) or thertcAgent.fs.write()API in thescripttool.
These 8 tools give the AI a complete file operation interface — operating on the frontend virtual file system just like working with a local terminal.
💡 All RTC tools follow the same lifecycle: the server creates an RTC record, saves a checkpoint, pauses the turn, pushes an event to the frontend, the frontend executes the tool, submits the result, the server restores the checkpoint, and reasoning continues.
State Machine
Section titled “State Machine”Each RTC call has a well-defined lifecycle state:
| State | Prompt Seen by AI | Description |
|---|---|---|
Pending | [Tool Pending] | Waiting for frontend to receive |
Sent | [Tool Pending] | Delivered to frontend |
Executing | [Tool Pending] | Currently executing |
Completed | Tool output | Execution successful, result returned |
Failed | [Tool Error] | Execution failed |
Timeout | [Tool Timeout] | Execution timed out |
Rejected | [Tool Rejected] | User rejected execution |
Permission Matrix
Section titled “Permission Matrix”Different work modes determine whether tool execution requires user confirmation. See Work Modes for the full permission matrix.
📌 Core Principle: Read-only operations are always safely allowed;
writeonly requires confirmation in the highest-alertness mode;scriptcan execute arbitrary code, so it requires user confirmation in all modes except bypass.
Checkpoint Mechanism
Section titled “Checkpoint Mechanism”The core reliability guarantee of RTC is the Checkpoint:
| Feature | Description |
|---|---|
| Storage Location | Redis, TTL 24 hours |
| Crash Recovery | Can recover from Checkpoint after server restart |
| User Experience | Perceived as “AI waiting for tool result before continuing” |
Reliability Guarantee
Section titled “Reliability Guarantee”RTC result reporting follows the 100% delivery principle:
Why must it be 100% delivered? Because the AI is waiting for the tool result to continue reasoning. If the result is lost, the AI will wait indefinitely.
| Mechanism | Description |
|---|---|
| Infinite Retry | Failed submissions are automatically retried, with no give-up mechanism |
| Idempotency Guarantee | Repeated submissions with the same client_id return success |
| Exponential Backoff | Retry intervals: 1s → 2s → 4s → … → capped at 30s |
Sequential Execution
Section titled “Sequential Execution”RTC calls within the same session are strictly serialized to avoid file conflicts caused by concurrency:
The next RTC is processed only after the current one completes. This is transparent to the user — the AI naturally completes each operation in sequence.
Worker Queue Model
Section titled “Worker Queue Model”The rtc-queue uses a hold-lock mode to handle consecutive work for the same Session: after completing a work item, the Worker retains the session lock and continues claiming more work from the same Session until the queue is empty, then releases the lock. This ensures that turn-loop agents (e.g., executing multiple RTC tool calls in sequence) don’t need to re-compete for the lock between operations, improving efficiency for consecutive processing.
💡 This is an internal optimization, completely transparent to users. Users simply experience the coherence of AI operations — no unnecessary waits between multiple tool calls.
Next Steps
Section titled “Next Steps”- Virtual File System — Learn about the file system that RTC tools operate on
- Work Modes — Learn about the mode switching behind the permission matrix
- Session Management — Learn about RTC context within sessions