Back to library

🚦Use AI Gateways: OpenRouter, Portkey, Helicone

Stop choosing a gateway because a blog post said so. By the end you can pick OpenRouter, Portkey, Helicone, or self-host for a real multi-region app and defend it on failover, cost, and observability.

Applied14 drops~2-week path · 5–8 min/daytechnology

Phase 1What an AI Gateway Actually Solves

See the three problems a gateway actually solves

4 drops

  1. The three problems a gateway actually solves

    6 min

    An AI gateway isn't a router — it's three products in one trench coat: failover, cost tracking, and observability. Most teams discover they needed the second two only after an outage forced them to discover the first.

  2. When you don't need a gateway

    5 min

    If you have one provider, low volume, and no compliance asks, a gateway is overhead — not insurance. The right call is to call the provider SDK directly and revisit when one of the three problems actually appears.

  3. How a gateway sits in the request path

    7 min

    A gateway is just an HTTP proxy with an LLM-shaped API. Your app calls the gateway like it called OpenAI; the gateway calls the actual provider and returns the response. Everything else — retries, caching, logging — is what it does in between.

  4. The failover-cost-observability triangle

    6 min

    Every gateway optimizes for one corner of the triangle: OpenRouter for failover, Portkey for cost+governance, Helicone for observability. Pick by which corner is most painful, not which vendor pitched best.

Phase 2Wiring OpenRouter Into One Endpoint

Wire OpenRouter into one endpoint with a fallback

5 drops

  1. OpenRouter in 10 lines: drop-in OpenAI replacement

    8 min

    OpenRouter speaks the OpenAI Chat Completions API. Swapping in OpenRouter is literally a base URL and an API key change — your existing SDK code keeps working unchanged.

  2. Configure your first fallback chain

    8 min

    OpenRouter lets you declare a fallback chain in the request itself: `models: ['anthropic/claude-3.5-sonnet', 'openai/gpt-4o']`. If the first fails or rate-limits, it tries the second automatically — no app-side retry logic.

  3. Force a failure and watch failover behave

    8 min

    The only way to trust failover is to break the primary on purpose. Set the primary to an invalid model name or an expired key, keep the fallback valid, and confirm the request still succeeds — that's the gateway earning its keep.

  4. See per-request cost in the gateway dashboard

    7 min

    OpenRouter's dashboard surfaces per-request cost in dollars (not tokens) within seconds. The instant you route through it, your finance question 'what does this feature cost?' becomes answerable per endpoint, per user, per day.

  5. Wire gateway env vars into your real app

    9 min

    The migration from scratch file to real app is one environment variable per provider. Keep the original OpenAI key as a fallback path (or remove it entirely) — the gateway becomes the single point of LLM access for your app.

Phase 3Choosing Among Real Gateway Options

Compare OpenRouter, Portkey, Helicone, and self-host

4 drops

  1. OpenRouter — when 'simplest path to multi-provider' wins

    8 min

    OpenRouter is the right pick when your top problem is 'I want failover across providers without managing multiple SDKs.' It's the simplest gateway by a wide margin — and that simplicity is the feature.

  2. Portkey — when governance and policy enforcement matter

    9 min

    Portkey is the gateway for teams whose top problem is 'we have 8 engineers calling LLMs, we need budgets, role-based access, prompt registries, and audit logs — not just routing.' It's a control plane, not a router.

  3. Helicone — when observability is the headline problem

    8 min

    Helicone treats every LLM call as a data point: logged, indexed, replayable, diffable. It's the gateway for teams whose top problem is 'production is breaking and we can't tell which prompts caused what outputs.'

  4. Self-hosted gateways — LiteLLM and when DIY pays off

    9 min

    Self-hosted gateways like LiteLLM (and Kong AI, Cloudflare AI Gateway) give you full control of the proxy, no third-party vendor in the request path, and zero per-call SaaS cost — at the price of operating another service.

Phase 4Defending a Gateway Choice at Scale

Defend a gateway choice for a 5-provider multi-region app

1 drop

  1. Pick a gateway for a 5-provider multi-region app

    22 min

    A gateway decision at this scale is a one-page memo: which corner of the triangle dominates, which gateway optimizes for it, what the alternatives are, and what the failure mode is if you're wrong. Writing the memo IS the work.

Frequently asked questions

What is an AI gateway and how is it different from an LLM router?
This is covered in the “Use AI Gateways: OpenRouter, Portkey, Helicone” learning path. Start with daily 5-minute micro-lessons that build from fundamentals to hands-on application.
When do I actually need a gateway versus just calling OpenAI directly?
This is covered in the “Use AI Gateways: OpenRouter, Portkey, Helicone” learning path. Start with daily 5-minute micro-lessons that build from fundamentals to hands-on application.
OpenRouter vs Portkey vs Helicone — which one should I pick?
This is covered in the “Use AI Gateways: OpenRouter, Portkey, Helicone” learning path. Start with daily 5-minute micro-lessons that build from fundamentals to hands-on application.
Does an AI gateway add latency to every LLM call?
This is covered in the “Use AI Gateways: OpenRouter, Portkey, Helicone” learning path. Start with daily 5-minute micro-lessons that build from fundamentals to hands-on application.
Can I self-host an AI gateway, and is it worth the operational cost?
This is covered in the “Use AI Gateways: OpenRouter, Portkey, Helicone” learning path. Start with daily 5-minute micro-lessons that build from fundamentals to hands-on application.

Related paths

🐍Python Decorators Introduction

Build one mental model for Python decorators that covers closures, argument passing, functools.wraps, and stacking — then ship a working caching or logging decorator from scratch in under 30 lines.

Applied~2-week path · 5-8 min/day

🦀Rust Lifetimes Explained

Stop reading `'a` as line noise and start reading it as scope arithmetic — one failing snippet at a time — until you can thread lifetimes through a small parser or iterator adapter without fighting the borrow checker.

Applied~2-week path · 5-8 min/day

☸️Kubernetes Core Concepts

Stop drowning in 30+ resource types. Build the mental model one primitive at a time -- pods, deployments, services, ingress, config -- then deploy a real app with rolling updates and health checks.

Applied~2-week path · 5-8 min/day

📈Big O Intuition

Stop treating Big O as math you memorized for an interview — build the intuition to spot O(n²) disasters, pick the right data structure without thinking, and rewrite a slow function from O(n²) to O(n) in under five minutes.

Foundations~2-week path · 5-8 min/day