Your Custom LLMs.
Your AWS.
Running 24/7.

AWS-Native Inference Platform
for Fine-Tuned LLMs

Your Custom LLMs.
Your AWS.
Running 24/7.

Bring your weights, set SLO and cost targets.
Rivvr runs, optimizes, scales, and recovers your inference inside your AWS account, so your ML team can ship.

Your custom models, running at scale inside your AWS. Never break, continuously optimized by R1, the most efficient inference orchestration engine.

Your custom models, running at scale inside your AWS. Never break, continuously optimized by R1, the most efficient inference orchestration engine.

Deploy any vLLM-compatible model

Auto load-testing
Your model or
LoRA adaptor
TTFT p50≤ 1,100 ms
TTFT p90≤ 1,800 ms
TPS p50≥ 50 tok/s
TPS p90≥ 50 tok/s
Cost≤ $30/hr
SLO compliance≥ 95%
OpenAI-compatible Endpoint
from openai import OpenAI
 
client = OpenAI(
base_url="https://rivvr.ai/v1",
)
 
client.chat.completions.create(
model="your-model",
messages=[{
"role": "user",
"content": "Hi",
}],
)

Rivvr keeps your custom SLOs

AWS VPC
Your SLOs and budget
TTFT p50
≤ 1,100 ms
840 ms
TTFT p90
≤ 1,800 ms
1,280 ms
TPS p50
≥ 50 tok/s
64 tok/s
TPS p90
≥ 50 tok/s
57 tok/s
Cost
≤ $30/hr
$14.19/hr
SLO compliance
≥ 95%
97.6%
Mixed VM Fleet
Live metrics
Requests
forecast 38/s
42 req/s
Queue depth
requests waiting
1
Tokens / request
prompt / output
2,048 / 180
GPU utilization
fleet average
62%
KV cache usage
fleet average
70%
KV hit rate
of 78% possible
72%
4×L40SOn-demand
g6e.12xlarge$10.49/hr
4×L4Spot
g6.12xlarge$1.84/hr
1×L40SOn-demand
g6e.xlarge$1.86/hr
03 Engine
Prompts grew to 2k tokens
Re-tuned onto a g6e.xlarge

Control performance and costs

Monitoringqwen35-9b-support-prodSLO compliance97.3%≥ 95%
TTFTtarget p90 ≤ 1,800 ms
p50 769·p90 1,328 ms
TPStarget p90 ≥ 50 tok/s
p50 64·p90 56 tok/s
Input costper 1M tokens
$0.056
Output costper 1M tokens
$0.41

Benchmarks

Benchmarks

10× lower cost during 10× bursts

Compared with SageMaker HyperPod for Qwen3 14B FP8, across both input and output tokens.

Rivvrllm-dSageMaker HyperPod
Qwen3 14B FP8 · Scaling through traffic bursts
Cost, $ / 1M tokens · Lower is better
Input
Output
No bursts
$0.040
$0.057
$0.076
$0.080
$0.114
$0.152
2x bursts
$0.050
$0.125
$0.156
$0.100
$0.250
$0.313
4x bursts
$0.060
$0.210
$0.263
$0.120
$0.420
$0.525
10x bursts
$0.075
$0.600
$0.750
$0.150
$1.200
$1.500

13.4× faster cold starts

Rivvr is ready in 39 seconds on a cold node, compared with 8 minutes 42 seconds for vLLM + EKS.

Scroll the plot to compare startup times →

Image pullModel loadingvLLM start
Time to ready · Lower is better
0m2m4m6m8m10m
0.450s39s8m 42s
Rivvr
warm node
Rivvr
cold node
vLLM + EKS

73% fewer reserved GPU-hours

Same capacity availability as an ODCR, with no commitment. Rivvr holds the node only while capacity is scarce, then releases it.

Reserved GPU-hours in a month · Lower is better
AWS ODCR
720 h
Rivvr
194 h
Day 151015202530
Reserved GPU capacityNo GPUs available in AWS

One platform to manage inference

One platform
to manage inference

RivvrRivvr
Overview
Inference
Model Storage
Monitoring
Project Settings
production
Runtime
Model Storage
Canonical Project Storage for model and LoRA artifacts.
5 artifacts·3 models·2 LoRA
Storage Overview
Project artifacts available for model and LoRA source selection.
ArtifactTypeFilesChecksIn UseUpdatedActions
qwen35-9b
ModelUnchanged
Checked 3 hours ago
ReadyIn useyesterday
support-adaptor-v1-1-4
LoRAUnchanged
Checked 3 hours ago
ReadyIn useyesterday
qwen35-9b-support-fused-v1-1-3
ModelUnchanged
Checked 4 days ago
1 warningAvailable4 days ago
support-adapter-v1-1-1
LoRAUnchanged
Checked 4 days ago
ReadyAvailable4 days ago
qwen3-14b
ModelUnchanged
Checked 9 days ago
ReadyIn use9 days ago

Manage inference from your code, terminal, or AI agent

Rivvr
MCP server
API
Python SDK
CLI
Deploy qwen35-9b with the support-adaptor-v1-1-4 LoRA as qwen35-9b-support-prod, at most 30 USD/hour. Follow testing and deployment until it's ready.
Creating qwen35-9b-support-prod
qwen35-9b · LoRA support-adaptor-v1-1-4 · ≤ $30/hour
Following testing and deployment
DEPLOY_READY

Inference inside
your AWS VPC

Inference inside
your AWS VPC

VPC deployment

Run inference inside your AWS account and VPC, within a network you control.

VPC deployment

Run inference inside your AWS account and VPC, within a network you control.

Data privacy

Your prompts, responses, logs, and model weights stay inside your AWS account.

Data privacy

Your prompts, responses, logs, and model weights stay inside your AWS account.

RBAC and access control

Assign project roles to control who can deploy models and change serving settings.

RBAC and access control

Assign project roles to control who can deploy models and change serving settings.

Scoped API keys

Manage access to models and clusters through revocable API keys.

Scoped API keys

Manage access to models and clusters through revocable API keys.

Pricing

Pricing

Startup

$2,200/month

Control plane and orchestrator

Deployment in 5 business days

2,200 units included monthly, then $1 per additional unit*

All inference and optimization features

Single AWS region

Standard support

Startup

$2,200/month

VPC deployment

Deployment in 5 business days

2,200 units included monthly, then $1 per additional unit*

All inference and optimization features

Single AWS region

Standard support

Startup

$2,200/month

VPC deployment

Deployment in 5 business days

2,200 units included monthly, then $1 per additional unit*

All inference and optimization features

Single AWS region

Standard support

Enterprise

Custom

Custom contract terms

Multi-region deployments

Single Sign-On (SSO)

Role-based access control (RBAC)

Audit logs and governance

Dedicated engineering support

Enterprise

Custom

Custom contract terms

Multi-region deployments

SSO

Role-based access control (RBAC)

Audit logs and governance

Dedicated engineering support

Enterprise

Custom

Custom contract terms

Multi-region deployments

SSO

Role-based access control (RBAC)

Audit logs and governance

Dedicated engineering support

* 1 unit covers 1 hour of managing an 80 GB GPU. Usage scales with GPU memory: a 24 GB GPU uses 0.3 units/hour.

Built by AI infrastructure experts

Built by a team that shipped AI infrastructure and production software for research institutions, hospitals, big pharma, and insurers, including Fortune 500 enterprises.

Built by AI infrastructure experts

Built by a team that shipped AI infrastructure and production software for research institutions, hospitals, big pharma, insurers, including Fortune 500 enterprises.

University of Toronto logo
Xiaomi logo
OneShield logo
SickKids Research Institute logo

Frequently asked questions

Frequently asked questions

How quickly can you roll out Rivvr?

We send you a rollout checklist. Once you complete it, we install Rivvr and run system checks within 5 business days.

What if we don’t have AWS GPU quota?

Your account needs an approved GPU quota before Rivvr can provision the corresponding instances.

AWS GPU instances are often unavailable. How does Rivvr handle that?

R1 checks availability across compatible GPU instance types and automatically reserves capacity. Your deployment can use different compatible GPUs instead of waiting for one specific instance type.

Do we need to change our application?

For an OpenAI-compatible client, update the endpoint URL, model name, and inference API key.

How is Rivvr priced?

A platform fee plus a charge based on GPU capacity under management.

Run models. Not infrastructure.

Run models.
Not infrastructure.