OpenAI's and Anthropic's chat APIs take text in and give text out, even though the model underneath samples token IDs at every step. There is precedent for token-ID input (OpenAI's legacy Completions prompt field, vLLM's prompt_token_ids), but no major hosted chat API exposes it end to end. This post analyzes the current input/output shapes and proposes a concrete, opt-in token-ID mode using the same string-or-array interface pattern from the token-native storage post.

Token-Native LLM APIs: Input and Output as Token IDs

Inference-free SPLADE nearly matches full SPLADE quality at 13× lower query latency, no GPU needed at query time. Benchmarked against full SPLADE and BM25 on Qdrant and pyserini.

Inference-Free SPLADE: Full Quality, 13× Faster Queries

Agents read and write in tokens (BPE). But our DB engines are still designed for humans (UTF-8). Persist payloads as BPE token IDs with a static entropy coder and you get ~3.3× lossless compression, zero-cost persistence of LLM output, and a representation agents already speak.

Token-Native Storage: Read and Write in the Language Models Already Speak

Search is everywhere and is one of the hardest problems in CS. Here's why I'm obsessed with it.

Why I'm Obsessed With Search

Reflections on a year of growth, travel, and pursuing mastery

2025: Year in Review

I Reverse-Engineered Exa.ai Infrastructure Cost with Napkin Math

Exploring the JSON embeddings to for matching new products with existing ones