Open-Weight LLM Inference Providers for a Self-Hosted LiteLLM Router (Sep 2026)

I ran a comparative research pass (via Tavily) on open-weight inference providers as candidate routes behind my self-hosted LiteLLM gateway, targeting Llama 3.3 70B, Qwen3 (70B class), and DeepSeek-V3 class models. This is the full report, lightly edited for publication. Data is as of September 2026 — pricing and rate limits move fast, so re-verify before committing a production route.

Executive summary

Key findings: DeepInfra is consistently the lowest cost for Llama 3.3 70B with strong uptime; Groq and SiliconFlow show the highest throughput / low latency for large models; Fireworks and SiliconFlow provide the largest context windows and long-context pricing options; Together AI is relatively expensive and has documented lower recent uptime; Novita shows mid/low throughput on Llama 3.3 but competitive catalog breadth; OpenRouter is a routing/aggregation layer useful for cost arbitration but has free-tier and per-day limits that constrain production primary routing.

Normalized comparison tables (per-model snapshots)

Note: input/output pricing normalized to USD per 1M tokens; throughput is representative median tokens/sec where evidence exists; context length is maximum supported per provider/model in evidence. Cells marked “n/a” indicate no usable evidence in the dataset.

Llama 3.3 70B (per-provider)

Provider	Input $ /1M	Output $ /1M	Throughput (tokens/s)	Context
DeepInfra	$0.10	$0.32	16 (reported)	128k
Together AI	$1.04	$1.04	n/a	128k (model supports 128k)
Fireworks AI	$0.90	$0.90	145 (median reported)	131k (Instruct)
Novita AI	$0.135	$0.40	32–39 (reported)	131k
Groq	$0.59	$0.79	174–394 (reported range)	128k/131k
OpenRouter (as router)	varies (aggregator)	varies	depends on provider	free model limits apply
SiliconFlow	$0.14	$0.14	680 (claimed)	131k

DeepSeek-V3 class (per-provider)

Provider	Input $ /1M	Output $ /1M	Throughput (tokens/s)	Context
DeepInfra	$0.24 (input)	$0.84–$0.90 (output)	~21 (reported)	64k (DeepSeek V3 listing)
Fireworks AI	$0.22–$1.32 (tiers)	$0.007–$0.044 (tiers)	n/a	262k (V3.1 listing)
Novita AI	$0.27	$1.10	n/a	64k–131k (catalog entries)
SiliconFlow	$0.27	$1.10	n/a	164k (DeepSeek-V3.1)

Qwen3 (70B class) — snapshot (where available)

Provider	Input $ /1M	Output $ /1M	Throughput (tokens/s)	Context
Groq	$0.29	$0.59	662 (Qwen3 32B reported)	131k
Fireworks AI	n/a	n/a	n/a	Qwen3 entries with 131k–262k contexts
SiliconFlow	$0.06 (Qwen3-8B)	$0.06	n/a	131k (Qwen3-8B)

Provider synopses, operational notes and recommended routing role

DeepInfra — strengths and fit

Price leader for Llama 3.3 70B (input $0.10 / output $0.32 per 1M) and broad catalog including DeepSeek and Qwen variants; offers an OpenAI-compatible API per provider listings. [1], [27]
Concurrent request defaults documented (200 concurrent per model) and 429 behavior on limits. [2]
Strong published recent uptime on core API and many models (90-day model uptimes 99.1–100% for sampled models). [3]
Recommendation: Primary route for cost-sensitive Llama 3.3 workloads and production fallbacks where predictable price + good uptime are priorities; consider as primary for cost-sensitive high-volume inference and as primary or fallback for long-context DeepSeek use when paired with DeepInfra’s cache tiers. [1], [18], [3]

Groq — strengths and fit

High throughput and low latency for large models; multiple sources report Llama 3.3 throughput in the hundreds of tokens/sec and sub-250ms TTFT on median measurements. Groq per-model pricing is higher (e.g., Llama 3.3 input $0.59 / output $0.79). [11], [22], [23]
Explicit rate-limit headers and 429 behavior supported; free tier available with modest daily/request caps. [12], [24]
Recommendation: Primary route for latency-sensitive, real-time assistant and high-throughput bulk processing where cost budget permits; fallback only if cost must be minimized. [11], [12]

SiliconFlow — strengths and fit

Claims competitive on-demand pricing for some Qwen and Llama variants and very high throughput/low TTFT for Llama 3.3 in a bench comparison (claimed 680 tokens/s, 300 ms TTFT). Offers large DeepSeek V3 context variants (164k). [16], [17], [28]
Recommendation: Strong candidate for primary route on latency-sensitive or high-throughput profiles if internal verification reproduces bench claims; otherwise treat as high-value fallback pending verification. [16], [28]

Fireworks AI — strengths and fit

Offers very large context windows on many models (up to 1,048,576 for vision/selected models) and tiered serverless vs reserved throughput pricing; reported mid/high throughput on Llama 3.3 (median ~145 tokens/s) in community reviews. [7], [8], [23]
Pricing for large models and premium tiers can be high; reserved throughput options suit predictable high-volume use. [8]
Recommendation: Primary for long-context RAG tasks and specialized large-context workloads; use reserved tiers for production throughput SLAs, otherwise as fallback for occasional long-context needs. [7], [8]

Together AI — strengths and fit

Broad quantization support (Reference FP16, Turbo FP8, Lite INT4) and dedicated endpoints / cluster options; pricing for Llama 3.3 70B is relatively high at $1.04 /1M (input/output) and platform requires a $5 minimum credit purchase (no free tier). Recent uptime traces show lower availability in the dataset. [5], [4], [6]
Recommendation: Use as fallback or dedicated endpoint only when specific Together features (quantization tier, cluster reservation) are required; not recommended as primary cost route for Llama 3.3 given price and documented recent uptime. [5], [4], [6]

Novita AI — strengths and fit

Catalog includes Llama 3.3 and DeepSeek; Novita pricing for Llama 3.3 is mid-range ($0.135/$0.40) but reported throughput for Llama 3.3 in available data is modest (~32–39 tokens/s). Limits on request input tokens reported in a tool listing (12,288 for one Llama 3.3 listing). [10], [9], [28]
Recommendation: Secondary/fallback route for workloads requiring Novita’s specific catalog or hardware choices; not a primary low-latency route for Llama 3.3. [10], [9]

OpenRouter — strengths and fit

Acts as an aggregator/routing layer that can route to the cheapest provider for a model and offers BYOK with a fee; free model routing exists but with restrictive per-day/per-minute caps on free accounts. In-region (US/EU) routing is available on paid Business plans. [14], [15], [25]
Recommendation: Use OpenRouter as a cost-arbitration layer or secondary routing plane for failover and cost optimization, but not as sole primary path when high sustained throughput or strict per-second SLAs are required because free/anon limits are strict. [14], [25]

Decision criteria and configurable thresholds (qualitative)

Use these comparative axes to choose primary vs fallback:

Cost: prefer providers with lowest input+output $/1M for the targeted model (DeepInfra leads for Llama 3.3). [1]
p95 latency and TTFT: choose providers with sub-~300ms TTFT and high median tokens/sec for latency-sensitive assistants (Groq, SiliconFlow claims). [11], [23], [28]
Sustained throughput: select providers that publish or demonstrate high sustained tokens/sec under concurrency (Groq, reported DeepSeek results on H200 show high scaling for DeepSeek). [11], [19]
Regional & compliance needs: prefer providers with published residency options (OpenRouter in-region routing on business tiers; DeepInfra lists US-only residency). [14], [3]
Rate limits & free tiers: ensure provider limits match expected concurrency; Groq and DeepInfra document limits; OpenRouter free caps are small. [12], [2], [25]

Adjust weights depending on profile (e.g., high weight on cost for cost-sensitive, on p95 latency for low-latency).

Evidence gaps

No unified, directly comparable end-to-end OpenAI-compatibility fidelity score per provider in the dataset (required for the requested 0–10 API-compatibility scoring).
Missing consistent provider-side published SLA documents across the set (few explicit SLA terms were supplied).
Sparse cross-provider, same-testbed Qwen3 (70B-class) throughput numbers suitable for head-to-head normalization.
Incomplete published egress/storage fee detail for many providers in the provided findings.

Reproducible test plan (high level)

From a LiteLLM gateway instance, call each provider’s default inference endpoint for the model slug (request FP8/best default) with: batch sizes 1 and 8, prompt lengths short (32 tokens), medium (512), long (4k/64k where supported), concurrency sweep (1, 8, 32, 128), record TTFT, per-token generation times, error rates, retries; run 3× samples per condition, warm vs cold start separation, and measure sustained throughput over 60–300s windows. Normalize tokens/sec to per-provider reported quantization and report USD per 1M tokens using the provider’s input+output rates. Use exponential backoff per provider 429 guidance.

Recommendations (concise)

Low-latency, real-time assistant: Groq or SiliconFlow (primary) if verified, otherwise Groq primary, DeepInfra fallback. [11], [28]
Cost-sensitive high-volume inference: DeepInfra primary; OpenRouter used as aggregator for opportunistic routing to cheapest provider for bursts. [1], [14]
High-throughput bulk processing: Groq primary; Fireworks reserved throughput or DeepInfra flex/cached tiers for cost balance. [11], [8], [18]
Long-context RAG: Fireworks or SiliconFlow for large windows; DeepInfra for DeepSeek cached tiers when cost matters. [7], [17], [18]
Multi-region redundancy/failover: Combine DeepInfra (strong uptime) + OpenRouter for multi-provider routing; add Groq or SiliconFlow as low-latency regional failover where available. [3], [14], [11]

Works Cited

Research generated 2026-09-14 with Tavily (model: pro); figures reflect public listings and community benchmarks at that date.