The Router Thesis: When Models Became Commodities

By July 2024, GPT-4o Mini delivered comparable quality for sixty cents. The same capability, one hundred times cheaper, in sixteen months. ^[1] When DeepSeek V3 arrived in December 2024, it matched frontier-model performance at roughly a dollar per million tokens, accomplished through architectural efficiencies that shook every assumption about what it cost to build at the frontier. ^{[1] [2]}

This is not, at its core, a story about price. It is a story about what happens to markets when the most valuable thing in them becomes cheap and abundant almost overnight.

The Stanford AI Index 2025 puts precise numbers on the shift: inference cost for a system performing at the level of GPT-3.5 fell 280-fold between November 2022 and October 2024. ^{[3] [4]} Hugging Face, the open-source model repository, passed one million hosted models in September 2024 and two million by August 2025, with one thousand to two thousand new models uploaded every day. ^{[5] [6]} The performance gap between the best open-source models and the best closed commercial models, once a chasm of seventeen percentage points on the MMLU benchmark, had narrowed to roughly 1.7 percentage points by 2025. ^{[7] [2]} Andrej Karpathy, speaking at Y Combinator's AI Startup School in June 2025, framed the moment with a piece of infrastructure metaphor that has since traveled widely: OpenRouter, he said, is the "transfer switch" of AI, in the same sense that a physical transfer switch lets a building move its power supply between the grid, solar, and a generator. ^{[8] [9] [10]} The analogy captures something real. When there are many equivalent sources of a resource and the cost of choosing wrong is significant, you need a switching mechanism.

The AI industry now has one. Several, actually. And understanding why they exist, how they work, and what it means that a finance company called Ramp has just launched one at router.com while a payments company called Stripe has spent a reported seven and a half to eight billion dollars acquiring the leading marketplace version, requires understanding what happens when an industry's core resource becomes commoditized faster than the systems built to consume it can adapt. ^{[11] [12] [13]}

To understand routing, it helps to understand the problem that made it necessary rather than merely convenient.

Building an AI-powered product in 2022 was, from a model selection standpoint, a relatively simple decision. OpenAI had the most capable models. You used them. The cost was high, but the choice was clear. By 2024, the decision had become genuinely difficult. GPT-4o, Claude 3.5 Sonnet, Gemini 1.5 Pro, DeepSeek V3, Llama 4, Qwen: each had different strengths, different pricing, different context windows, different latency profiles, different terms of service, and different rates of improvement. The AI Index 2025 describes the era as "AI's Great Flattening," a moment when the top ten frontier models are separated by roughly 0.2 capability units on composite benchmarks, meaning that for most tasks, the question is no longer which model is capable but which is cheapest, fastest, or best suited to this particular kind of query. ^{[14] [7]}

The price spread across this landscape is enormous. As of 2026, the gap between the least expensive usable model and the most capable frontier model runs to roughly one hundred times on a per-token basis. ^{[15] [16]} This creates an obvious incentive for optimization, but optimization requires a mechanism. An application hardcoded to use a single model provider is leaving money on the table when simpler queries could be served by cheaper models, and it is also fragile: one provider outage, one rate-limit cascade, one pricing change, and the system breaks or becomes expensive in unpredictable ways.

The research literature began formalizing this problem in 2023. A paper called Tryage, submitted to arXiv in August 2023 by Surya Narayanan Hari and Matt Thomson, introduced the concept of a language model router: a system that analyzes incoming prompts and dispatches each to the model in a library most likely to handle it well, based on an objective function that integrates performance predictions with user constraints including model size, security requirements, and output readability. ^[17] Across heterogeneous datasets covering code, clinical data, and patents, Tryage identified the optimal model with 50.9 percent accuracy, compared to 23.6 percent for GPT-3.5 Turbo attempting the same dispatch task. ^[17] A month later, a group of researchers from IBM Research, MIT, and other institutions published a formalization of the problem as a collection of binary classification tasks, demonstrating it across eighteen open-source models ranging from three billion to seventy billion parameters. ^[18] Routing had become a recognized field of study.

The mechanism deserves precise explanation, because it is less mysterious than it sounds and more consequential than it appears.

A router is a control-plane component that sits between an application and multiple model backends. The application calls the router as if calling a single model API. The router, rather than forwarding the request blindly, evaluates it against a set of decision signals and selects which backend should actually handle it.

The evaluation happens across four dimensions simultaneously, combined in a scoring formula: cost, latency, quality, and security governance. ^[19] Cost scoring rewards cheaper models, normalized against the range of available options. Latency scoring rewards models that can respond within the task's acceptable window, accounting for time-to-first-token, per-token generation speed, and network overhead. Quality scoring estimates how well a model is likely to perform on this type of query, using benchmark scores, task-specific evaluation suites, or historical accuracy signals from prior similar requests. Security governance acts as a hard gate rather than a soft weight: if a request involves data classified as sensitive, models without appropriate data-handling agreements are simply removed from the candidate pool before scoring begins. ^[19] The router selects the highest-scoring model that clears all hard constraints, forwards the request, logs the outcome, and feeds that outcome back into future scoring estimates.

The overhead introduced by this process is, in practice, negligible relative to the latency of model inference itself. Rule-based routing adds less than one millisecond. Embedding-based routing adds roughly five milliseconds. Even a machine learning classifier, the most computationally expensive variant, adds fifty to one hundred milliseconds against a typical model response time of five hundred to two thousand milliseconds. ^[16] The routing decision is fast enough to be invisible to the user while being commercially significant at scale.

The academic grounding for learned routing was substantially advanced by RouteLLM, a framework developed by researchers from UC Berkeley, Anyscale, and Canva, submitted to arXiv in June 2024 and subsequently published at ICLR 2025 as a peer-reviewed conference paper. ^{[20] [21]} RouteLLM trained routing models on eighty thousand human preference battles from the Chatbot Arena dataset and demonstrated that a matrix factorization router could achieve ninety-five percent of GPT-4's performance while sending only twenty-six percent of requests to GPT-4, at a forty-eight percent cost reduction. With augmented training data, the same router required only fourteen percent of calls to reach the ninety-five percent quality threshold, achieving a seventy-five percent cost reduction compared to a random baseline. ^{[22] [23]} On the GSM8K math benchmark, the cost reduction reached eighty-five percent while maintaining ninety-five percent of GPT-4's accuracy. ^[22] The message was striking: most queries are easier than the hardest model available, and routing exploits that fact systematically rather than accidentally.

The market response to these dynamics produced a distinct class of product: the AI gateway or router, sitting between applications and the sprawl of model providers. Several companies now occupy this space with materially different philosophies.

OpenRouter, founded in early 2023 by Alex Atallah, the former co-founder and CTO of OpenSea, officially launched under that name in August 2023. ^{[24] [25] [26] [27]} Atallah's framing for what the company does is deliberately infrastructural: "AI is pretty unique. It's a battleground with lines being drawn around the models, and the models are just not available in all clouds. Gemini is only in Google Cloud. Anthropic is in AWS and Google, but not Azure. Grok is in Azure and from X, but not Google Cloud. We track them and sort of allow you to orchestrate across all of the clouds. OpenRouter is building the connective tissue." ^[28] The platform aggregates three hundred to five hundred models from sixty or more providers through a single OpenAI-compatible API endpoint, with routing options that include manual model selection, cheapest-available automatic routing using its floor suffix, fastest-response selection, and retrieval-augmented variants. ^{[24] [29] [30]} Its revenue model, notable for its structure, charges a 5.5 percent fee on credit top-ups rather than marking up inference itself, preserving provider list pricing for users. ^{[31] [32]}

The growth rate of OpenRouter has been striking. By May 2026, the platform was processing roughly twenty-five trillion tokens per week, serving more than eight million developers, with Sacra estimating annualized revenue of around $140 million. ^{[33] [34] [35]} Its Series B in March 2026, at a $1.3 billion valuation, came just months before the company's acquisition by Stripe, announced August 19, 2026. ^{[36] [11]} The official Stripe press release, signed by CEO Patrick Collison, does not disclose a price; media reports citing unnamed sources indicated figures ranging from $7.5 billion to more than $8 billion. ^{[11] [12] [13]} Collison's statement frames the logic in explicitly financial terms: "Tokens are the central currency for companies building with AI, and it's clear that the real-world economic potential will depend on making good use of scarce compute resources. Stripe is building the economic infrastructure for AI, and together with OpenRouter we'll help businesses maximize profitability by routing their requests intelligently and spending their tokens efficiently." ^[11]

Where OpenRouter is a hosted marketplace, LiteLLM is infrastructure for teams who want to run the routing layer themselves. The project, available on GitHub under the name BerriAI/litellm with more than 56,900 stars, describes itself as a Rust-core AI gateway with a Python SDK that supports one hundred or more providers in OpenAI-compatible format. ^[37] Its operational capabilities go substantially beyond simple model selection. LiteLLM supports multiple load-balancing strategies including simple shuffle, least-busy, usage-based, latency-based (with configurable time-to-live averaging windows), and cost-based routing. ^{[38] [39] [40]} It implements traffic mirroring, which allows teams to silently evaluate a candidate model against a copy of live traffic without affecting the primary request's latency or result. ^[38] Cooldowns isolate failing deployments rather than taking down entire model groups: a deployment that hits a 429 rate-limit response or fails more than fifty percent of calls within a minute is placed in a cooldown state, automatically recovering when the period expires, while other deployments in the group continue serving. ^[38] Fallback chains allow escalation from one model group to another when all deployments in the primary group have been exhausted, with specialized paths for context-window overflows and content-policy violations. ^{[40] [41]} Named production users include Netflix, Lemonade, and Rocket Money. ^{[42] [43]} Its stable release cadence reached v1.97.0 on August 16, 2026, with active pre-release development continuing. ^[44]

Martian, founded in November 2023 by Shriyash Upadhyay and Etan Ginsberg, both AI researchers from the University of Pennsylvania, takes a different technical approach. ^[45] Rather than building a hosted marketplace or a generic gateway, Martian built what it calls model mapping: a system that uses model compression, quantization, and distillation to understand the internal mechanics of black-box language models well enough to predict their behavior on a given query without running the full model. ^{[45] [46]} The result is a router that can estimate which model will produce the best output for a specific prompt, at the lowest cost, without requiring exhaustive live comparisons. Accenture invested in Martian in September 2024, integrating the technology into Accenture's "switchboard" services for enterprise clients. ^{[46] [47]} Martian has raised $9 million in seed funding from NEA, Prosus Ventures, and General Catalyst, and reports usage by engineers at more than three hundred companies. ^{[45] [48]}

Unify, a Y Combinator W23 company founded by Daniel Lenton, a PhD graduate from Imperial College London's Dyson Robotics Lab, uses a different approach again: a learned neural network router trained on exhaustive benchmarks run on each new model, with GPT-as-judge evaluation, that learns which models perform best on which categories of tasks. ^{[49] [50]} New providers are incorporated within a day or two of availability. ^[49] Unify raised $8 million in May 2024 from investors including SignalFire, Microsoft's venture arm M12, and Y Combinator, reporting approximately four thousand engineer signups from companies including Tesla, Meta, and DeepMind. ^{[49] [51]}

Portkey, founded in 2023 by Rohit Agarwal and Ayush Garg, positioned itself as the most comprehensive control plane of the group, combining gateway routing with observability, governance, prompt management, and compliance. ^{[52] [53]} Processing over one trillion tokens and 150 million AI requests daily by May 2026, and managing more than $180 million in annualized LLM spend for enterprises, Portkey's feature set includes circuit breakers, canary testing, conditional routing by user tier or region, role-based access control, virtual keys with per-team budget limits, ISO, SOC 2, HIPAA, and GDPR compliance with custom business associate agreements, and regional data planes for data residency requirements. ^{[54] [53] [55]} The company raised a $15 million Series A in February 2026 at a valuation estimated between $60 and $70 million, led by Elevation Capital. ^{[56] [57]} In April 2026, Palo Alto Networks announced its intent to acquire Portkey, closing the deal in May 2026 at a reported valuation of $120 to $140 million, approximately double the February figure. ^{[53] [58] [59]} Portkey has since been integrated into Palo Alto's Prisma AIRS platform. ^{[60] [61]} That a cybersecurity company paid a significant premium for an AI gateway within weeks of a $15 million Series A signals something about the speed at which routing infrastructure is being recognized as a security and compliance problem, not merely a cost optimization tool.

The most pointed statement about what routing means for enterprises came not from an AI company but from a financial technology company. On August 19, 2026, Ramp launched Router.com with the tagline: "The best model changes. Your application shouldn't." ^{[62] [63] [64]}

Ramp, founded in March 2019 by Eric Glyman, Karim Atiyeh, and Gene Lee, is a financial operations platform serving more than seventy thousand companies with corporate cards, expense management, accounts payable automation, and procurement tools. ^[65]) ^[66] It reached a $44 billion valuation at its $750 million Series F financing in June 2026. ^[66] It is not, on the face of it, an AI infrastructure company. But the logic of its foray into routing is internally coherent in a way that clarifies the broader stakes.

Ramp built an internal LLM router approximately three years before the public launch, initially to reduce the inference costs of AI products it was building for its seventy thousand customers. ^{[67] [68]} Veeral Patel, Ramp's Head of Applied AI and a founding engineer, described the genesis on LinkedIn: "3 years ago, we built an internal LLM router at Ramp that powers AI products for 70,000 customers. Back then it was mostly about saving money. Now it feels obvious: the best model changes constantly. GPT, Claude, Gemini, Grok, Qwen, DeepSeek, Kimi, GLM, prices and capabilities move every week." ^[67] The internal system, which Ramp's CTO Rahul Sengottuvelu confirmed reduced the company's own inference costs by thirty percent while simultaneously improving reliability, now processes 2.75 trillion tokens monthly across Ramp's production products. ^{[63] [64] [69]}

The public version, Router.com, implements several routing strategies that illustrate the practical sophistication of production-grade routing. The Flex Tier strategy monitors observed latency across standard and flexible service tiers and routes to the cheaper tier when it is demonstrably as fast as the standard offering, passing the savings to the caller. ^[63] Shadow Models sends a sample of live requests to candidate models in the background while the primary model continues serving users, allowing teams to evaluate new models on real workloads without affecting latency or quality for end users. ^[63] Benchmark Routing allows teams to define what matters for a particular workload by choosing up to three benchmarks and weighting them, with Router ranking models accordingly. ^[63] The NVIDIA Switchyard strategy reserves the expensive model for only those steps in a workflow that genuinely require it: an internal coding agent example handled fifty-eight turns with a cheaper model called Luna and escalated only the final five to the more capable Sol. ^[63] The underlying mechanism uses Thompson Sampling and exponentially weighted moving averages to learn real-time latency and failure distributions across models and service tiers, enabling routing decisions that account for current system state rather than static performance tables. ^[70]

Ramp also built its own evaluation benchmark, Ramp SWE-Bench, from real production engineering work, precisely because public leaderboards could not answer the specific cost-per-unit-of-work questions that mattered for routing decisions. ^{[62] [63]} As new models and routing strategies prove themselves on real workloads, Router incorporates them automatically. Early customers using Router reported average cost reductions of forty percent; the product is free through 2026, with users paying list price for the model tokens they consume. ^{[62] [64]}

The broader framing is where Ramp's contribution becomes genuinely distinctive. CEO Eric Glyman, in the official Series F announcement, articulated what the company is attempting: "For 500 years, business ran on two pillars of spend: people and vendors. In the last 24 months, a third arrived, intelligence, paid by the token and invisible to every system we've built to manage cost. Ramp is the infrastructure for the third pillar." ^{[71] [66]} In parallel with Router, Ramp launched AI Token Spend Management in July 2026, a product that connects to Anthropic, OpenAI, Cursor, and Gemini to give finance teams visibility over AI spending that had previously been invisible until month-end reconciliation. ^[72] Since June 2025, AI token spend across Ramp's customer base has increased 20.7 times. Average monthly AI token spend per customer jumped thirteen times between January 2025 and mid-2026. One in three businesses using the tool identified opportunities to shift work from expensive frontier models to more efficient alternatives performing equally well for the same tasks. ^{[72] [73]}

The implication is that routing, from Ramp's vantage point, is not primarily a developer productivity tool. It is a financial governance instrument, analogous to the spend controls Ramp already builds for employee expenses and vendor payments, applied to a new category of expenditure that CFOs neither planned for nor have tools to manage. CTO Rahul Sengottuvelu's framing in the Router.com press release is deliberately CFO-facing: "AI is the fastest-growing line item at most companies, and the one they can least measure. Router puts every token in one place and sends each request to the model that delivers the right performance at the right cost." ^[62] The tagline "Built for CTOs. Loved by CFOs" ^[64] is not accidental.

Each generation of internet infrastructure has included an abstraction layer that emerged to manage a glut of interchangeable underlying resources. The pattern is consistent enough to be instructive.

Load balancers began as DNS round-robin hacks in the 1990s, became dedicated hardware appliances from F5 Networks, Cisco, and Citrix in the 2000s (costing between $100,000 and $250,000 per installation, offering health checks, SSL offloading, and sticky sessions), and were subsequently commoditized first by open-source options like HAProxy and Nginx, then by cloud-native managed services like AWS Elastic Load Balancing, which offered auto-scaling, multi-zone high availability, and pay-per-use pricing. ^[74] The managed service won because it made the complexity of running a load balancer someone else's problem.

Content delivery networks arose in the late 1990s as a geographic abstraction over origin servers, allowing content owners to pay CDN operators to deliver content from proximity to end users without maintaining their own global infrastructure. ^[75] API gateways emerged for microservices architectures to centralize authentication, rate limiting, request routing, and observability that would otherwise be duplicated across every service in a distributed system. ^{[76] [77]} Cloud providers commoditized raw compute the same way, with AWS introducing EC2 and S3 in 2006, catalyzing cloud-native application design by abstracting physical servers behind virtual ones billed by the hour. ^[78]

In each case, the pattern is similar: a resource that was once scarce and differentiated becomes abundant and interchangeable, and a routing or abstraction layer emerges to manage the selection problem. The layer captures value not by controlling the underlying resource but by owning the selection decision and the operational complexity that comes with it. Google Cloud's announcement of a Model Routing feature for its API Gateway, published August 4, 2026, the same week as the Stripe-OpenRouter acquisition, frames this explicitly: "a managed alternative to client-side proxies such as LiteLLM, providing centralized infrastructure to move routing logic to network edge." ^{[79] [80]} The largest cloud provider in the world is treating model routing as a managed infrastructure service on the same tier as load balancing.

The Stripe-OpenRouter acquisition is the clearest evidence that this transition is complete. Menlo Ventures, which led OpenRouter's Series A, described the company as "the unified interface for LLMs, launched in 2023 with four models, now supporting over 500 models on over 80 providers," and quoted Karpathy's transfer-switch analogy as the most succinct summary of the platform's role. ^{[81] [8]} Atallah's own statement at announcement captured the philosophy: "We believe intelligence will be multi-model: no single model will be optimal for every task, and developers need a neutral layer to orchestrate and manage them all. Joining Stripe lets us accelerate that mission and bring the full AI ecosystem to every business." ^[11] Stripe's interest extends beyond the routing mechanics: the company has been building token-based billing infrastructure in partnership with OpenRouter since January 2026, and the acquisition closes a loop between routing (which model handles the request) and billing (who pays for the token, at what rate). ^{[82] [83]} Routing as payment rail is not a metaphor; at Stripe's scale, it is a product strategy.

OpenRouter, he said, is the "transfer switch" of AI, in the same sense that a physical transfer switch lets a building move its power supply between the grid, solar, and a generator.

The production reality of routing systems is more demanding than the simple version suggests, and the operational literature is candid about the gap.

IBM Research engineers, writing in July 2026, argued that routing is typically presented as a classification problem but in practice becomes a complex systems optimization problem. ^[84] An instructive example: analyzing a CodeAct agent across 417 AppWorld tasks, Claude Sonnet 4.6 cost $79 total ($0.19 per task) while GPT-4.1 cost $155 ($0.37 per task) despite GPT-4.1's lower nominal token pricing. The difference emerged from caching behavior: large context chunks reused across execution steps lowered Sonnet's effective input costs substantially. A router optimizing on price sheets alone would have made the wrong decision. ^[84] Cost, in other words, is not just model pricing; it is model pricing in the context of a specific workload's memory and caching patterns.

Complexity is not just task difficulty either. A query asking to "summarize this contract" appears simple but may trigger document retrieval, compliance checking, tool invocation, and multiple refinement rounds. A highly technical prompt may be efficiently handled by a small specialized model. Difficulty is often invisible at routing time, and routers must simultaneously balance cost, quality, latency, model specialization, and reliability, while enterprise deployments add compliance requirements, data residency rules, privacy constraints, and approved model lists. ^{[84] [85]}

LiteLLM's documented routing capabilities illustrate what production plumbing actually looks like: load balancing strategies, retry policies configurable per exception type (rate-limit errors, authentication failures, timeout errors, content policy violations each warrant different retry logic), cooldowns at the individual deployment level rather than the entire model group (preserving availability when one deployment fails without taking down alternatives), traffic mirroring for silent evaluation, and Redis-backed distributed state management to ensure consistent rate-limit tracking across multiple proxy instances behind a load balancer. ^{[38] [86] [40]} This is infrastructure engineering, not AI engineering.

The observability challenge is equally consequential. Research published in 2026 argues that routing systems require what the authors call "route receipts," runtime records of the serving conditions that produced a given response, including which model was selected, at what service tier, through which execution path, with which tools invoked. ^[87] Current platforms document some routing decisions but none surveyed expose a portable per-answer receipt covering the full route, including tool-search traces and unified decision context. ^[87] When a customer-support agent produces a wrong answer, knowing the model identifier is not sufficient for diagnosis; the team needs to know whether the request was served by the expected model, whether it used priority or standard capacity, whether a fallback triggered, and whether it used retrieved context or cached context. ^{[87] [88]} Without that visibility, debugging production failures is guesswork.

The case against routing has enough substance to require engagement rather than dismissal.

The sharpest version comes from a case study published in Towards Data Science in June 2026, describing a product team that built a routing layer to reduce AI costs and discovered, three months later, that it had damaged the product in ways aggregate dashboards could not detect. ^[88] The team trained a small classifier on approximately two hundred thousand historical customer-support queries, labeling each as simple or complex, and routed the roughly sixty-five percent labeled simple to a cheaper model at about one-quarter the per-token cost. Within eight weeks, the inference bill fell to forty percent of its previous level. Aggregate human review samples remained green. The offline regression suite showed equivalent quality on ninety-four percent of holdout examples. In-product satisfaction signals were stable.

The problem emerged three months later. The cheaper model performed well on eighty percent of routed queries but failed structurally on the remaining twenty percent, queries whose surface form looked simple but whose underlying intent was complex. A billing inquiry phrased as "where is my charge from" could mask a fraud investigation, a reconciliation problem, or a billing-cycle dispute. The frontier model hedged and handled the nested intent; the cheaper model answered the surface question confidently and incorrectly. Customers disengaged and called human support instead. Cohort satisfaction fell at the ninety-day follow-up. The inferred cost of quality loss was conservatively four to five times the inference savings: roughly $100,000 per month saved versus $400,000 to $500,000 per month in retention and support costs. ^[88] The team's measurement infrastructure had been built to validate the routing logic, not to detect the specific failure modes that routing introduced.

This is not an argument against routing in principle. It is an argument that routing adds a control-plane layer whose failure modes are distinct from, and harder to observe than, the failure modes of the application itself. Routing can improve aggregate system performance while making individual failure modes less visible, precisely because the routing decision is not the same as the execution decision. ^{[87] [89]}

A separate technical concern: routing can destroy the economics of prompt caching. When a request routes to a different model or provider than the previous request in a session, cached context must be recomputed. In workloads with high cache hit rates, the cache-miss cost can exceed the per-token savings from routing to a cheaper model. IBM Research's analysis of the CodeAct agent workload found that Sonnet's lower cache-read pricing effectively inverted the expected cost ordering relative to nominal token rates. ^{[84] [85]} A routing system that optimizes on token price but ignores cache economics is optimizing on a faulty signal.

Research published in arXiv in early 2026 also documented adversarial vulnerabilities in ML-based routers. A study of four router architectures (BERT classifier, matrix factorization, similarity-weighted ranking, and a causal LLM classifier) against adversarial inputs designed to manipulate routing decisions found that causal LLM routers showed attack success rates near one hundred percent in cost escalation scenarios, where crafted inputs cause expensive models to be selected for simple queries. Gray-box attacks raised jailbreak success rates from 42.8 percent to 73.4 percent by triggering routing to less safety-constrained models. ^[90] The proposed defense, called RerouteGuard, achieved greater than ninety-nine percent detection accuracy with less than 2.5 percent false positive rates on legitimate queries, but adding a detection layer requires its own engineering and introduces the very observability and monitoring complexity the original routing layer obscured. ^[90]

A recent empirical study in the routing literature is also humbling for advocates of sophisticated learned routers. Research published in May 2026 found that a well-tuned k-nearest-neighbors approach, the simplest possible learned method, matched or outperformed state-of-the-art neural routers on standard benchmarks, achieving average AUC scores of 52.68 against 51.71 for multilayer perceptrons and 51.82 for graph neural networks, while running thirteen to fourteen times faster. ^{[91] [92]} Under distribution shift (queries that differ from the training distribution), the kNN approach degraded by only 2.63 points while more complex methods degraded by 6.67 points. The authors argue that routing problems have a low intrinsic dimensionality (effective dimensions between two and twenty-eight, far below the nominal dimensionality of the embedding space), which makes non-parametric methods like kNN structurally well-suited and complex parametric methods prone to overfitting. ^[91] The takeaway for practitioners is straightforward: before investing in sophisticated routing infrastructure, test the simplest possible approach. It may be sufficient.

Finally, there is the question of whether routing constitutes a durable competitive advantage. The honest answer is that it probably does not on its own. A competitor can replicate routing logic more readily than they can replicate customer relationships, proprietary data accumulated through usage, or deep integration into customer workflows. The defensibility framework that matters for AI businesses places velocity and user experience at the bottom (six to eighteen months of protection), workflow stickiness in the middle (two to five years), and compounding proprietary data and structural lock-in at the top (three to ten or more years). ^[93] Routing sits between the first and second levels. Ramp's router is interesting not because the routing logic is irreplicable but because it is embedded in a platform that already manages the expense cards, the accounts payable workflows, the vendor management systems, and the procurement approvals of seventy thousand companies. ^{[94] [95]} The router is not the moat; the integrated financial operations platform is the moat. The router is what makes the platform more useful to companies managing AI token spend as a line item alongside headcount and vendor costs.

The emergence of routing as deliberate infrastructure rather than ad-hoc optimization reflects something larger than the routing decision itself.

When AI capabilities were concentrated in one or two providers, the question of which model to use was resolved by the market. When capabilities became commoditized across dozens of providers, the question of which model to use became an optimization problem requiring its own system layer. The router is the market's structural response to its own success at producing capable models. It is what happens when supply exceeds the ability of demand to evaluate it.

The geography of token consumption on OpenRouter tells an adjacent story. Chinese-origin models, including DeepSeek, Xiaomi, Tencent, MiniMax, and Qwen, captured roughly forty-four percent of token volume across the top ten models on OpenRouter by June 2026, up from near zero a year earlier. US model share collapsed from approximately seventy percent in June 2025 to thirty percent by June 2026. ^[96] A CNBC investigation published in July 2026 found that Chinese-origin models had captured forty-six percent of US enterprise token usage routed through OpenRouter. ^[97] The routing layer is not simply a cost optimization tool; it is a conduit through which geopolitical AI competition expresses itself in commercial workloads. Whatever position a government or enterprise wants to take about which models it uses, that position must be enforced at the router level, because applications written against an OpenAI-compatible API endpoint will use whatever model the router selects unless the router is configured with explicit restrictions.

Portkey's acquisition by Palo Alto Networks, a cybersecurity company, makes more sense in this light. ^{[53] [58]} The question of which model handles a request is, increasingly, a governance question about what data leaves the enterprise perimeter, under which contractual terms, subject to which retention policies, and in compliance with which regulatory frameworks. A router configured incorrectly can route sensitive data to a model provider with incompatible data handling practices. A router configured correctly enforces data residency, approved model lists, and compliance requirements at the point of every inference call. That is a security problem, and security companies understand security problems.

The infrastructure analogy points toward where this market is likely to go. Load balancers did not remain independent products; they consolidated into cloud platform services, with managed offerings from AWS, Azure, and GCP capturing the majority of the market. API gateways followed a similar arc. Open-source implementations, commercial vendors, and managed cloud offerings are likely to compete in routing on the same terms, with consolidation around a small number of dominant platforms as scale economies favor integrated stacks that optimize across routing, billing, and observability simultaneously. Stripe's acquisition of OpenRouter is, on this reading, not just a financial bet on a specific company but a structural bet that in an AI-first economy, the ability to route tokens efficiently and bill for them accurately will be as valuable as the ability to route payments and settle them accurately. ^{[11] [81]}

The bet may be right. It may also be premature. The routing market is real and growing. The operational complexity it adds is also real. The enterprises that benefit most will be those that treat routing as the infrastructure decision it has become rather than the developer convenience it started as, and that build the observability and governance systems required to know when routing is working and when, quietly and expensively, it is not.

The best model changes constantly. The application that depends on only one of them is already fragile. The question is what happens in the layer between them, and who controls it.