01
Trim context deliberately
Remove repeated history and irrelevant retrieved passages before a request. Keep the source material and evaluate quality after each change.
Field notes
Measure the work. Then tune the prompt.
These guides explain common ways teams reason about inference costs. They are informational workflows, not Relay features or live dashboards.
Cost techniques
Start with the request you actually need.
02
Set output budgets
Choose a maximum output length that fits the task. Long answers can cost more and may add review work.
03
Cache exact repeats
For deterministic tasks, consider an application cache keyed to the full normalized request. Set an expiry and avoid caching personalized or sensitive responses without a clear policy.
04
Normalize structured payloads
When a provider supports structured output, send only the fields the model needs and validate the result before downstream use.
05
Route by measured task fit
Compare models on representative inputs, then choose the least expensive model that meets your quality and latency needs. Do not assume a model is reachable because it appears in a catalog.
Price history
Treat every price as a dated observation.
A price can change with provider list rates, routing and capacity. Keep the source, date, billing unit, input and output rates, cache treatment and image rates together. A snapshot is useful for comparison, but it is not a live feed and should not be displayed as a fresh quote.
Relay’s public catalog labels the capture date of the gateway list prices it is derived from. Each Relay price is set per model, is shown next to that list price, and is the rate requests are debited at; a model with no gateway price shows no price.
Rankings
Popularity is not a fit score.
Request volume can show what other users tried during a measurement window. It does not establish model quality for your workload, availability for your account, or a stable preference. Compare a small, representative set of tasks and note the sample size and date.
Controlled comparison
Run an evaluation you can explain.
- Choose representative inputs and a clear quality rubric.
- Keep prompts, context, parameters and evaluation conditions fixed.
- Record provider, model identifier, price source and date, latency, errors, and measured output quality.
- Review failures and edge cases before changing production traffic.
Relay does not currently provide an experiment runner, live ranking feed or pricing-history dashboard. These steps can be followed with your own evaluation harness.
Need a specific provider or model?
Contact Relay to ask about current route access and early access readiness.