Agent-IR explores a question: could a compact instruction format make a local model's tool calls easier to validate and execute? The design combines grammar-constrained output with a small register-based runtime in Rust.
This article describes the architecture and illustrative code. The instruction stream is still text that needs parsing; the aim is to reduce format complexity. Token savings and latency improvements need to be measured against a JSON baseline before drawing performance conclusions.
1. Grammatical Compilation via GBNF Constrained Decoding
A grammar can constrain the shape of generated tool calls. Agent-IR explores GBNF constrained decoding, limiting the next allowed tokens to those compatible with an instruction grammar.
That addresses syntax. The runtime still needs to validate register indices, argument types, permissions, and whether an instruction makes sense for the task.
Instead of generating a verbose string token payload, the model selects native register commands sequentially:
A shorter instruction format may reduce output length, but the actual token count depends on the tokenizer and task. A valid instruction is also not necessarily a correct tool call; both need separate evaluation.
2. Low-Overhead Runtime: The Memory-Safe Rust Virtual Machine
The proposed runtime parses instructions into a small register structure. This sketch shows the dispatch mechanism; a complete implementation would replace panics and unchecked indexing with validation and recoverable errors.
The separation makes the execution rules explicit and testable. It does not make the model's decisions deterministic or provide a security boundary by itself.
3. Bounding Execution Safety: The Academic Horizon
The next steps are to test the design against explicit execution limits and a reproducible baseline:
- Instruction Budget Boundaries: Implementing an explicit instruction-step budget loop inside the VM context layer to guarantee that recursive model logic pathways cannot trigger an infinite processing crash.
- Quantitative Latency Telemetry: Compiling precise dataset performance curves tracking token footprint reduction ratios and compute consumption efficiency gains against standard JSON-RPC baselines across variable parameter scales.
The idea is to make the boundary between generated instructions and execution small enough to inspect and test. The next useful result will be an evaluation of correctness, token usage, and latency against equivalent JSON tool calls.