🚀 Programmatic Tool Calling with AI SDK

A Universal LLM Optimization POC for Any Model

Programmatic Tool Calling is a novel approach to dramatically reduce LLM inference costs and latency by replacing traditional multi-round-trip tool calling with code generation and sandboxed execution.

💡 Inspired by Anthropic’s beta capabilities announced November 2025. This project extends that paradigm to work with any LLM through the Vercel AI SDK, including 100+ models via the AI Gateway.

---

🎯 The Problem

Traditional LLM tool calling is inherently inefficient, especially with MCP:

User: "Get data for users 1-5 and find the highest scorer"

Traditional Approach (N round-trips):
┌─────────────────────────────────────────────────────────────┐
│ Round 1: LLM → getUser(1) → result → LLM (context grows)    │
│ Round 2: LLM → getUser(2) → result → LLM (context grows)    │
│ Round 3: LLM → getUser(3) → result → LLM (context grows)    │
│ Round 4: LLM → getUser(4) → result → LLM (context grows)    │
│ Round 5: LLM → getUser(5) → result → LLM (context grows)    │
│ Round 6: LLM → final answer                                  │
└─────────────────────────────────────────────────────────────┘
                         ⬇️
            6 LLM calls × full context each
            Accumulated results pollute context
            High latency, high token cost

✨ The Solution

PTC transforms tool orchestration into a single code generation + execution:

Programmatic Approach (1 round-trip):
┌─────────────────────────────────────────────────────────────┐
│ Round 1: LLM generates JavaScript:                          │
│   const users = await Promise.all([                         │
│     getUser({ id: '1' }), getUser({ id: '2' }),             │
│     getUser({ id: '3' }), getUser({ id: '4' }),             │
│     getUser({ id: '5' })                                    │
│   ]);                                                        │
│   return users.sort((a,b) => b.score - a.score)[0];         │
│                                                              │
│ → Execute in Sandbox → Return final result only             │
│                                                              │
│ Round 2: LLM receives final answer, responds to user        │
└─────────────────────────────────────────────────────────────┘
                         ⬇️
            2 LLM calls total
            Intermediate results never enter context