Your Custom LLMs.
Your AWS.
Running 24/7.

Your Custom LLMs.
Your AWS.
Running 24/7.

Bring your weights, set SLO and cost targets.
Rivvr runs, optimizes, scales, and recovers your inference inside your AWS account, so your ML team can ship.

Running your own LLM is ongoing work

Running your own LLM is ongoing work

Self-hostingWith Rivvr
AWS capacityGPU shortages and quota limits block scaling.Provisions from available types; holds capacity only while it's scarce.
Infra tuningManual tuning goes stale as traffic changes.Continuously re-tunes every layer of the serving stack from live traffic.
CostYou pay for idle GPUs sized for peak.Right-sized VMs that scale with traffic and to zero — you pay for what serves.
Traffic burstsAutoscaling reacts late; bursts break SLOs.Forecasts and scales ahead of demand; holds SLOs through bursts.
FailuresAn engineer detects and restores each failure.Detects, replaces replicas, and recovers automatically.
Your teamKeeping it running takes a platform team.You set targets. Rivvr runs inference. Your team ships product.

Deploy any vLLM-compatible model

Your checkpoint and config, not a catalog model. Rivvr load-tests and tunes it, finds suitable GPUs, then serves an OpenAI-compatible endpoint for your application.

Auto load-testing
Your model or
LoRA adapter
TTFT p50≤ 1,100 ms
TTFT p90≤ 1,800 ms
TPS p50≥ 50 tok/s
TPS p90≥ 50 tok/s
Cost≤ $30/hr
SLO compliance≥ 95%
OpenAI-compatible Endpoint
from openai import OpenAI
 
client = OpenAI(
base_url="https://rivvr.ai/v1",
)
 
client.chat.completions.create(
model="your-model",
messages=[{
"role": "user",
"content": "Hi",
}],
)

Rivvr keeps your custom SLOs

Continuous optimization at every layer, from live traffic. Rivvr re-tunes vLLM, bin-packs models across GPUs, forecasts traffic to scale ahead of demand, and recovers from failures. No redeployment, no manual flags.

AWS VPC
Your SLOs and budget
TTFT p50
≤ 1,100 ms
840 ms
TTFT p90
≤ 1,800 ms
1,280 ms
TPS p50
≥ 50 tok/s
64 tok/s
TPS p90
≥ 50 tok/s
57 tok/s
Cost
≤ $30/hr
$14.19/hr
SLO compliance
≥ 95%
97.6%
Mixed VM Fleet
Live metrics
Requests
forecast 38/s
42 req/s
Queue depth
requests waiting
1
Tokens / request
prompt / output
2,048 / 180
GPU utilization
fleet average
62%
KV cache usage
fleet average
70%
KV hit rate
of 78% possible
72%
4×L40SOn-demand
g6e.12xlarge$10.49/hr
4×L4Spot
g6.12xlarge$1.84/hr
1×L40SOn-demand
g6e.xlarge$1.86/hr
03 Engine
Prompts grew to 2k tokens
Re-tuned onto a g6e.xlarge

Control performance and costs

Set the targets. See them held. SLO compliance, latency, and cost per model on live traffic — change a target and Rivvr reconfigures inference without manual tuning.

Monitoringqwen35-9b-support-prodSLO compliance97.3%≥ 95%
TTFTtarget p90 ≤ 1,800 ms
p50 769·p90 1,328 ms
TPStarget p90 ≥ 50 tok/s
p50 64·p90 56 tok/s
Input costper 1M tokens
$0.056
Output costper 1M tokens
$0.41

Benchmarks

Benchmarks

10× lower cost during 10× bursts

Compared with SageMaker HyperPod for Qwen3 14B FP8, across both input and output tokens.

Rivvrllm-dSageMaker HyperPod
Qwen3 14B FP8 · Scaling through traffic bursts
Cost, $ / 1M tokens · Lower is better
Input
Output
No bursts
$0.040
$0.057
$0.076
$0.080
$0.114
$0.152
2x bursts
$0.050
$0.125
$0.156
$0.100
$0.250
$0.313
4x bursts
$0.060
$0.210
$0.263
$0.120
$0.420
$0.525
10x bursts
$0.075
$0.600
$0.750
$0.150
$1.200
$1.500

13.4× faster cold starts

Rivvr is ready in 39 seconds on a cold node, compared with 8 minutes 42 seconds for vLLM + EKS.

Scroll the plot to compare startup times →

Image pullModel loadingvLLM start
Time to ready · Lower is better
0m2m4m6m8m10m
0.450s39s8m 42s
Rivvr
warm node
Rivvr
cold node
vLLM + EKS

73% fewer reserved GPU-hours

Same capacity availability as an ODCR, with no commitment. Rivvr holds the node only while capacity is scarce, then releases it.

Reserved GPU-hours in a month · Lower is better
AWS ODCR
720 h
Rivvr
194 h
Day 151015202530
Reserved GPU capacityNo GPUs available in AWS

One platform to manage inference

One platform to manage inference

RivvrRivvr
Overview
Inference
Model Storage
Monitoring
Project Settings
production
Runtime
Model Storage
Canonical Project Storage for model and LoRA artifacts.
5 artifacts·3 models·2 LoRA
Storage Overview
Project artifacts available for model and LoRA source selection.
ArtifactTypeFilesChecksIn UseUpdatedActions
qwen35-9b
ModelUnchanged
Checked 3 hours ago
ReadyIn useyesterday
support-adapter-v1-1-4
LoRAUnchanged
Checked 3 hours ago
ReadyIn useyesterday
qwen35-9b-support-fused-v1-1-3
ModelUnchanged
Checked 4 days ago
1 warningAvailable4 days ago
support-adapter-v1-1-1
LoRAUnchanged
Checked 4 days ago
ReadyAvailable4 days ago
qwen3-14b
ModelUnchanged
Checked 9 days ago
ReadyIn use9 days ago

Manage inference from your code, terminal, or AI agent

Rivvr
MCP server
API
Python SDK
CLI
Deploy qwen35-9b with the support-adapter-v1-1-4 LoRA as qwen35-9b-support-prod, at most 30 USD/hour. Follow testing and deployment until it's ready.
Creating qwen35-9b-support-prod
qwen35-9b · LoRA support-adapter-v1-1-4 · ≤ $30/hour
Following testing and deployment
DEPLOY_READY

Inference inside your AWS VPC

Inference inside your AWS VPC

VPC deployment

Run inference inside your AWS account and VPC, within a network you control.

VPC deployment

Run inference inside your AWS account and VPC, within a network you control.

VPC deployment

Run inference inside your AWS account and VPC, within a network you control.

Data privacy

Your prompts, responses, logs, and model weights stay inside your AWS account.

Data privacy

Your prompts, responses, logs, and model weights stay inside your AWS account.

Data privacy

Your prompts, responses, logs, and model weights stay inside your AWS account.

RBAC and access control

Assign project roles to control who can deploy models and change serving settings.

RBAC and access control

Assign project roles to control who can deploy models and change serving settings.

RBAC and access control

Assign project roles to control who can deploy models and change serving settings.

Scoped API keys

Manage access to models and clusters through revocable API keys.

Scoped API keys

Manage access to models and clusters through revocable API keys.

Scoped API keys

Manage access to models and clusters through revocable API keys.

Pricing

Pricing

Startup

$2,200/month

Control plane and orchestrator

Deployment in 5 business days

2,200 units included monthly, then $1 per additional unit*

All inference and optimization features

Single AWS region

Standard support

Startup

$2,200/month

Control plane and orchestrator

Deployment in 5 business days

2,200 units included monthly, then $1 per additional unit*

All inference and optimization features

Single AWS region

Standard support

Enterprise

Custom

Custom contract terms

Multi-region deployments

Single Sign-On (SSO)

Role-based access control (RBAC)

Audit logs and governance

Dedicated engineering support

Enterprise

Custom

Custom contract terms

Multi-region deployments

Single Sign-On (SSO)

Role-based access control (RBAC)

Audit logs and governance

Dedicated engineering support

* 1 unit covers 1 hour of managing an 80 GB GPU. Usage scales with GPU memory: a 24 GB GPU uses 0.3 units/hour.

Built by AI infrastructure experts

Built by a team that shipped AI infrastructure and production software for research institutions, hospitals, big pharma, and insurers, including Fortune 500 enterprises.

Built by AI infrastructure experts

Built by a team that shipped AI infrastructure and production software for research institutions, hospitals, big pharma, and insurers, including Fortune 500 enterprises.

University of Toronto logo.
Xiaomi logo.
OneShield logo.
SickKids Research Institute logo.

Frequently asked questions

Frequently asked questions

How quickly can you roll out Rivvr?

We send you a rollout checklist. Once you complete it, we install Rivvr and run system checks within 5 business days.

What if we don’t have AWS GPU quota?

Your account needs an approved GPU quota before Rivvr can provision the corresponding instances.

AWS GPU instances are often unavailable. How does Rivvr handle that?

R1 checks availability across compatible GPU instance types and automatically reserves capacity. Your deployment can use different compatible GPUs instead of waiting for one specific instance type.

Do we need to change our application?

For an OpenAI-compatible client, update the endpoint URL, model name, and inference API key.

How is Rivvr priced?

We charge based on GPU capacity under management. Check the pricing page for more details.

Run models. Not infrastructure.

Run models. Not infrastructure.