Lanes

Cheap, fast inference for open models

Run open-weight models on an OpenAI-compatible API — 2–10× cheaper than the managed APIs we track, depending on model.

By signing in you agree to the Terms of Service and Acceptable Use.

Free trial credit on sign-up · no card

curl https://34.134.52.0.nip.io/v1/chat/completions \
  -H "Authorization: Bearer lanes-sk-…" \
  -H "Content-Type: application/json" \
  -d '{"model":"qwen3-8b","messages":[{"role":"user","content":"Hi"}]}'

from openai import OpenAI
lanes = OpenAI(base_url="https://34.134.52.0.nip.io/v1", api_key="lanes-sk-…")
r = lanes.chat.completions.create(
    model="qwen3-8b", messages=[{"role": "user", "content": "Hi"}])
print(r.choices[0].message.content)

import OpenAI from "openai";
const lanes = new OpenAI({ baseURL: "https://34.134.52.0.nip.io/v1", apiKey: "lanes-sk-…" });
const r = await lanes.chat.completions.create({
  model: "qwen3-8b", messages: [{ role: "user", content: "Hi" }] });
console.log(r.choices[0].message.content);

Offerings

Prices are per 1M tokens. Live now, more on the way.

Models

Model	Status	Details	Lanes	$ / 1M in	$ / 1M out
qwen	live	Qwen3 4B · reasoning-capable · AWQ 4-bit · NVIDIA L4	realtimebatch	$0.015	$0.200
qwen3-8b	live	Qwen3 8B · stronger general reasoning · AWQ 4-bit · NVIDIA L4	realtimebatch	$0.030	$0.350
qwen3-27b	soon	Qwen3.6 27B · flagship dense · AWQ 4-bit · NVIDIA A100 80GB · 32k in / 8k out · thinking opt-in	realtimebatch	$0.050	$0.320
qwen3-32b	soon	Qwen3 32B · flagship dense · 32GB-class GPU	realtimebatch	soon	soon
qwen3-coder	soon	Qwen3 Coder 30B · code + tool use · MoE	realtimebatch	soon	soon
llama3-70b	soon	Llama 3.3 70B · frontier-class open weights · A100	realtimebatch	soon	soon
deepseek-v3	soon	DeepSeek V3 · large MoE · long context · batch-first	realtimebatch	soon	soon

Two lanes

active

Realtime — interactive responses. What you get by default. Measured first-token p95 (nonced, per pricing/basis.json): qwen 0.515s; qwen3-8b 1.076s. Models not listed have no measured figure yet, so we make no latency claim for them.

active

Batch — results within 12h. Per model: input is the lower of $0.05/1M and the realtime input rate, output is half the realtime output price floored at $0.13/1M (our measured cost floor) and never above realtime. You pay only for jobs that complete. For large, non-urgent workloads.

qwen: batch $0.015 in / $0.130 out per 1M
qwen3-8b: batch $0.030 in / $0.175 out per 1M

Realtime vs batch — which do I want?

Realtime: chat, agents, anything a person or a live app is waiting on.
Batch: overnight evals, bulk classification/extraction, dataset generation — cheaper because it rides spare capacity.

Agent-native live

Structured output — pass a JSON schema in response_format; get guaranteed-valid JSON back.
Tool / function calling — standard OpenAI tools + tool_choice; the model returns tool_calls.
Drop-in OpenAI-compatible — any OpenAI SDK, streaming, with prefix caching on.

How it saves you money

The price isn't a discount — it's the real cost of running open models on well-utilised hardware. measured

What makes it cheap measured

Open-weight models — You run open models (Qwen3), not a frontier lab's closed model — so there's no proprietary-API premium in the price.
4-bit quantization (AWQ) — The model is compressed to 4-bit weights: ~2.8× more throughput per GPU at essentially the same quality (measured ~90% on GSM8K vs full precision).
Inexpensive GPU capacity, kept busy — We serve on low-cost cloud GPUs (an L4 for the small models, an A100 for the large one) on standard provisioning from one provider today — the price stays near raw hardware cost because the cards are kept busy, not because anyone subsidises them. Multi-provider spot capacity with automatic failover is on the bench (below), not in production.
Continuous batching — Many requests share one GPU at the same time (vLLM), so the fixed cost of the card spreads across everyone instead of sitting idle between calls.
Prefix caching — When requests share a prompt prefix — system prompts, few-shot examples, RAG context — we compute it once and reuse it (measured ~2.65× on shared workloads). That saving is one reason the flat rate is low; today repeated input is billed at the SAME rate as fresh input — differential cached-token pricing is on the bench.
The batch lane — If your jobs can wait for spare capacity, they run within 12 hours at 29–40% below the realtime price (blended 3:1 in:out, live models — derived from the rates we actually bill). You pay for the tokens we generate, like any other request.

Net effect: our blended price lands 2–10× below the managed-API anchors we track (blended 3:1 in:out), depending on the model — so the savings are structural, not a temporary promo.

On the bench research

Directions we're testing. These are not part of today's price — some of our own experiments parked them on cost, and we'll only ship a lever once it actually beats what we run now.

Multi-provider spot / preemptible GPU capacity with automatic failover (today: one provider, standard provisioning).
CPU expert-streaming (WASTE) for frontier-size models — quality parity, but not yet price-competitive.
Cross-model KV-cache transfer / small-model prefill — helps only at capability gaps too small to matter, so far.
Two-tier cached-vs-fresh input pricing and prefix-sorted batch scheduling (measured ~2.65× on shared prefixes).