Cost Over Time
Cluster spend and selected model-attributed cost.
Cluster spendModel-attributed costInspect
Inference Platform for Fine-Tuned LLMs
Inference Platform
for Fine-Tuned LLMs
Inference Platform for Fine-Tuned LLMs
Your custom models, running at scale inside your AWS. Never break, continuously optimized by R1,
the most token-efficient inference orchestration engine.
Your custom models, running at scale inside your AWS. Never break, continuously optimized by R1, the most token-efficient inference orchestration engine.
Your custom models, running at scale inside your AWS. Never break, continuously optimized by R1, the most token-efficient inference orchestration engine.
AWS VPC
R1 Orchestrator
Qwen3.5-9B
LoRA support-adaptor-v1-1-4
TTFT p50
≤ 1,100 ms
TTFT p90
≤ 1,800 ms
TPS p50
≥ 50 tok/s
TPS p90
≥ 50 tok/s
Cost
≤ $30 / hr
H100
On-demand
vLLM
H100
On-demand
vLLM
L40S
On-demand
vLLM
A10G
Spot
vLLM
State-of-the-Art Token Efficiency
R1 continuously learns from live traffic and optimizes inference for speed and cost.
State-of-the-Art Token Efficiency
R1 continuously learns from live traffic and optimizes inference for speed and cost.
State-of-the-Art Token Efficiency
R1 continuously learns from live traffic and optimizes inference for speed and cost.
40%
higher token efficiency vs llm-d
70%
higher token efficiency vs SageMaker AI
70%
higher token efficiency vs SageMaker AI
Rivvrllm-dSageMaker HyperPod
Qwen3 14B FP8 · Scaling through traffic burstsCost, $ / 1M tokens · Lower is better
Input
Output
2x bursts
$0.050
$0.083
$0.104
$0.100
$0.167
$0.208
4x bursts
$0.060
$0.210
$0.263
$0.120
$0.420
$0.525
10x bursts
$0.075
$0.600
$0.750
$0.150
$1.200
$1.500
One platform to manage inference
One platform
to manage inference
Deployments
Model storage
Metrics
Costs
Access
Dev tools
Deploy any vLLM-compatible model as a dedicated OpenAI API endpoint with custom SLOs. Manage all deployments in one place.
RivvrOverview
Inference
Model Storage
Monitoring
Project Settings
production
PerformanceCosts
Monitoring
Costs
Infrastructure and serving cost for a cluster or one selected model.
1h24h7d90d
Refreshproduction-cluster
qwen35-9b-support-prod
Window: Last 60 minutesResolution: 1mUpdated: just now
Total Cluster Cost
$19.48
Cluster spend for selected range, not model-attributed cost
Allocated Model Cost
$15.39
Cost attributed to the selected model
$ / 1M Prefill
$0.57
Latest series point
$ / 1M Decode
$1.10
Latest series point
RivvrOverview
Inference
Model Storage
Monitoring
Project Settings
production
PerformanceCosts
Monitoring
Performance
Runtime performance and usage analytics for inference workloads.
1h24h7d90d
Refreshproduction-cluster
qwen35-9b-support-prod
Window: Last 60 minutesResolution: 1mUpdated: just now
Health
SLO Compliance
Meeting SLO97.6%Target 95%
Request-weighted across intervals in the selected range
Errors
13
+0.3% · prev period
Error Rate
0.1%
Across filtered workload
TTFT
Time to first token across live traffic.
p50 919msp90 1,421ms
Inspect2,500ms2,000ms1,500ms1,000ms500ms
Access group
alice · User
API key
Project production
qwen35-9b-support-prod
qwen3-14b
Project research
Access restricted
RivvrOverview
Inference
Model Storage
Monitoring
Project Settings
production
ClustersModelsTesting Runs
Runtime
Models
Operational control surface for model onboarding, testing, and automatic provisioning.
Search models...
All clusters
All statuses
2 models·2 ready
Models Overview
Model readiness, serving state, optimization posture, and quick entry into lifecycle operations.
Model
Cluster
Status
Optimization
Updated
qwen35-9b-support-prod
qwen35-9b · 1 LoRA
production-cluster
Ready
Ready
Up to $30.00 / hr
On
2 days ago
qwen3-14b
qwen3-14b
production-cluster
Ready
Ready
Up to $20.00 / hr
On
9 days ago
aider
Rivvr
MCP server
API
Python SDK
CLI
>Deploy qwen35-9b with the support-adaptor-v1-1-4 LoRA as qwen35-9b-support-prod, at most 30 USD/hour. Follow testing and deployment until it's ready.
●Creating qwen35-9b-support-prod
⎿qwen35-9b · LoRA support-adaptor-v1-1-4 · ≤ $30/hour
●Following testing and deployment
⎿DEPLOY_READY
1
from rivvr import Rivvr, ModelOptions
2
3
client = Rivvr(
4
api_url="https://rivvr.customer.example",
5
project="research",
6
)
7
model = client.models.create(
8
name="qwen35-9b-support-prod",
9
storage="qwen35-9b",
10
options=ModelOptions(
11
# adapter name: LoRA storage entry
12
lora={
13
"support-adaptor-v1-1-4":
14
"support-adaptor-v1-1-4",
15
},
16
),
17
max_cost_per_hour=30,
18
wait=True,
19
)
$ rivvr model create qwen35-9b-support-prod \
--storage qwen35-9b \
--lora support-adaptor-v1-1-4=support-adaptor-v1-1-4 \
--max-cost-per-hour 30 --wait
RivvrOverview
Inference
Model Storage
Monitoring
Project Settings
production
Runtime
Model Storage
Canonical Project Storage for model and LoRA artifacts.
5 artifacts·3 models·2 LoRA
Storage Overview
Project artifacts available for model and LoRA source selection.
| Artifact | Type | In Use | Integrity | Updated | Actions |
|---|---|---|---|---|---|
qwen35-9b | Model | In use | Verified Checked 3 hours ago | yesterday | |
support-adaptor-v1-1-4 | LoRA | In use | Verified Checked 3 hours ago | yesterday | |
qwen35-9b-support-fused-v1-1-3 | Model | Available | Verified Checked 4 days ago | 4 days ago | |
support-adapter-v1-1-1 | LoRA | Available | Verified Checked 4 days ago | 4 days ago | |
qwen3-14b | Model | In use | Verified Checked 9 days ago | 9 days ago |
Deployments
Model storage
Metrics
Costs
Access
Dev tools
Deploy any vLLM-compatible model as a dedicated OpenAI API endpoint with custom SLOs. Manage all deployments in one place.
RivvrOverview
Inference
Model Storage
Monitoring
Project Settings
production
PerformanceCosts
Monitoring
Costs
Infrastructure and serving cost for a cluster or one selected model.
1h24h7d90d
Refreshproduction-cluster
qwen35-9b-support-prod
Window: Last 60 minutesResolution: 1mUpdated: just now
Total Cluster Cost
$19.48
Cluster spend for selected range, not model-attributed cost
Allocated Model Cost
$15.39
Cost attributed to the selected model
$ / 1M Prefill
$0.57
Latest series point
$ / 1M Decode
$1.10
Latest series point
Cost Over Time
Cluster spend and selected model-attributed cost.
Cluster spendModel-attributed costInspect
RivvrOverview
Inference
Model Storage
Monitoring
Project Settings
production
PerformanceCosts
Monitoring
Performance
Runtime performance and usage analytics for inference workloads.
1h24h7d90d
Refreshproduction-cluster
qwen35-9b-support-prod
Window: Last 60 minutesResolution: 1mUpdated: just now
Health
SLO Compliance
Meeting SLO97.6%Target 95%
Request-weighted across intervals in the selected range
Errors
13
+0.3% · prev period
Error Rate
0.1%
Across filtered workload
TTFT
Time to first token across live traffic.
p50 919msp90 1,421ms
Inspect2,500ms2,000ms1,500ms1,000ms500ms
Access group
alice · User
API key
Project production
qwen35-9b-support-prod
qwen3-14b
Project research
Access restricted
RivvrOverview
Inference
Model Storage
Monitoring
Project Settings
production
ClustersModelsTesting Runs
Runtime
Models
Operational control surface for model onboarding, testing, and automatic provisioning.
Search models...
All clusters
All statuses
2 models·2 ready
Models Overview
Model readiness, serving state, optimization posture, and quick entry into lifecycle operations.
Model
Cluster
Status
Optimization
Updated
qwen35-9b-support-prod
qwen35-9b · 1 LoRA
production-cluster
Ready
Ready
Up to $30.00 / hr
On
2 days ago
qwen3-14b
qwen3-14b
production-cluster
Ready
Ready
Up to $20.00 / hr
On
9 days ago
aider
Rivvr
MCP server
API
Python SDK
CLI
>Deploy qwen35-9b with the support-adaptor-v1-1-4 LoRA as qwen35-9b-support-prod, at most 30 USD/hour. Follow testing and deployment until it's ready.
●Creating qwen35-9b-support-prod
⎿qwen35-9b · LoRA support-adaptor-v1-1-4 · ≤ $30/hour
●Following testing and deployment
⎿DEPLOY_READY
1
from rivvr import Rivvr, ModelOptions
2
3
client = Rivvr(
4
api_url="https://rivvr.customer.example",
5
project="research",
6
)
7
model = client.models.create(
8
name="qwen35-9b-support-prod",
9
storage="qwen35-9b",
10
options=ModelOptions(
11
# adapter name: LoRA storage entry
12
lora={
13
"support-adaptor-v1-1-4":
14
"support-adaptor-v1-1-4",
15
},
16
),
17
max_cost_per_hour=30,
18
wait=True,
19
)
$ rivvr model create qwen35-9b-support-prod \
--storage qwen35-9b \
--lora support-adaptor-v1-1-4=support-adaptor-v1-1-4 \
--max-cost-per-hour 30 --wait
RivvrOverview
Inference
Model Storage
Monitoring
Project Settings
production
Runtime
Model Storage
Canonical Project Storage for model and LoRA artifacts.
5 artifacts·3 models·2 LoRA
Storage Overview
Project artifacts available for model and LoRA source selection.
| Artifact | Type | In Use | Integrity | Updated | Actions |
|---|---|---|---|---|---|
qwen35-9b | Model | In use | Verified Checked 3 hours ago | yesterday | |
support-adaptor-v1-1-4 | LoRA | In use | Verified Checked 3 hours ago | yesterday | |
qwen35-9b-support-fused-v1-1-3 | Model | Available | Verified Checked 4 days ago | 4 days ago | |
support-adapter-v1-1-1 | LoRA | Available | Verified Checked 4 days ago | 4 days ago | |
qwen3-14b | Model | In use | Verified Checked 9 days ago | 9 days ago |
Deployments
Model storage
Metrics
Costs
Access
Dev tools
Deploy any vLLM-compatible model as a dedicated OpenAI API endpoint with custom SLOs. Manage all deployments in one place.
RivvrOverview
Inference
Model Storage
Monitoring
Project Settings
production
PerformanceCosts
Monitoring
Costs
Infrastructure and serving cost for a cluster or one selected model.
1h24h7d90d
Refreshproduction-cluster
qwen35-9b-support-prod
Window: Last 60 minutesResolution: 1mUpdated: just now
Total Cluster Cost
$19.48
Cluster spend for selected range, not model-attributed cost
Allocated Model Cost
$15.39
Cost attributed to the selected model
$ / 1M Prefill
$0.57
Latest series point
$ / 1M Decode
$1.10
Latest series point
Cost Over Time
Cluster spend and selected model-attributed cost.
Cluster spendModel-attributed costInspect
RivvrOverview
Inference
Model Storage
Monitoring
Project Settings
production
PerformanceCosts
Monitoring
Performance
Runtime performance and usage analytics for inference workloads.
1h24h7d90d
Refreshproduction-cluster
qwen35-9b-support-prod
Window: Last 60 minutesResolution: 1mUpdated: just now
Health
SLO Compliance
Meeting SLO97.6%Target 95%
Request-weighted across intervals in the selected range
Errors
13
+0.3% · prev period
Error Rate
0.1%
Across filtered workload
TTFT
Time to first token across live traffic.
p50 919msp90 1,421ms
Inspect2,500ms2,000ms1,500ms1,000ms500ms
Access group
alice · User
API key
Project production
qwen35-9b-support-prod
qwen3-14b
Project research
Access restricted
RivvrOverview
Inference
Model Storage
Monitoring
Project Settings
production
ClustersModelsTesting Runs
Runtime
Models
Operational control surface for model onboarding, testing, and automatic provisioning.
Search models...
All clusters
All statuses
2 models·2 ready
Models Overview
Model readiness, serving state, optimization posture, and quick entry into lifecycle operations.
Model
Cluster
Status
Optimization
Updated
qwen35-9b-support-prod
qwen35-9b · 1 LoRA
production-cluster
Ready
Ready
Up to $30.00 / hr
On
2 days ago
qwen3-14b
qwen3-14b
production-cluster
Ready
Ready
Up to $20.00 / hr
On
9 days ago
aider
Rivvr
MCP server
API
Python SDK
CLI
>Deploy qwen35-9b with the support-adaptor-v1-1-4 LoRA as qwen35-9b-support-prod, at most 30 USD/hour. Follow testing and deployment until it's ready.
●Creating qwen35-9b-support-prod
⎿qwen35-9b · LoRA support-adaptor-v1-1-4 · ≤ $30/hour
●Following testing and deployment
⎿DEPLOY_READY
1
from rivvr import Rivvr, ModelOptions
2
3
client = Rivvr(
4
api_url="https://rivvr.customer.example",
5
project="research",
6
)
7
model = client.models.create(
8
name="qwen35-9b-support-prod",
9
storage="qwen35-9b",
10
options=ModelOptions(
11
# adapter name: LoRA storage entry
12
lora={
13
"support-adaptor-v1-1-4":
14
"support-adaptor-v1-1-4",
15
},
16
),
17
max_cost_per_hour=30,
18
wait=True,
19
)
$ rivvr model create qwen35-9b-support-prod \
--storage qwen35-9b \
--lora support-adaptor-v1-1-4=support-adaptor-v1-1-4 \
--max-cost-per-hour 30 --wait
RivvrOverview
Inference
Model Storage
Monitoring
Project Settings
production
Runtime
Model Storage
Canonical Project Storage for model and LoRA artifacts.
5 artifacts·3 models·2 LoRA
Storage Overview
Project artifacts available for model and LoRA source selection.
| Artifact | Type | In Use | Integrity | Updated | Actions |
|---|---|---|---|---|---|
qwen35-9b | Model | In use | Verified Checked 3 hours ago | yesterday | |
support-adaptor-v1-1-4 | LoRA | In use | Verified Checked 3 hours ago | yesterday | |
qwen35-9b-support-fused-v1-1-3 | Model | Available | Verified Checked 4 days ago | 4 days ago | |
support-adapter-v1-1-1 | LoRA | Available | Verified Checked 4 days ago | 4 days ago | |
qwen3-14b | Model | In use | Verified Checked 9 days ago | 9 days ago |
Continuous improvement
Inference stack, tailored for each model. R1 continuously learns from traffic and optimizes inference and maintains custom SLOs and cost targets.
Continuous improvement
Inference stack, tailored for each model. R1 continuously learns from traffic and optimizes inference and maintains custom SLOs and cost targets.
Zero operations
Full production cycle, automated by Rivvr: load testing, rollouts, scaling, incident recovery and infrastructure tuning and monitoring.
Zero operations
Full production cycle, automated by Rivvr: load testing, rollouts, scaling, incident recovery and infrastructure tuning and monitoring.
Any vLLM-compatible model
Your weights, LoRA adaptors, quantization, custom SLOs and cost limits. Run any vLLM-compatible model without operational burden.
Any vLLM-compatible model
Your weights, LoRA adaptors, quantization, custom SLOs and cost limits. Run any vLLM-compatible model without operational burden.
R1 Orchestrator:
continuously optimizing
for speed and cost
R1 Orchestrator:
continuously optimizing
for speed and cost
Qwen3.5-9B
LoRA support-adaptor-v1-1-4
TTFT p50
≤ 1,100 ms
TTFT p90
≤ 1,800 ms
TPS p50
≥ 50 tok/s
TPS p90
≥ 50 tok/s
Cost
≤ $30/hr
01 · Routing
Routing, designed for LLM applications
Routes requests by cache locality and priority, probes replica health, and manages circuit breaking.
02 · KV Cache
Distributed, tiered KV cache
Coordinates cache across GPU memory, RAM, SSD, and persistent storage, preserving cached state as nodes enter and leave.
03 · Engine
vLLM auto-tuning
Profiles traffic patterns and continuously tunes vLLM to maximize throughput within your SLOs.
04 · GPU
Heterogeneous GPU bin packing
Places models across mixed GPU types and shares compute between them based on demand and workload type (batched or real-time).
05 · Scaling & recovery
Sub-30-second cold starts
and auto-recovery
Fast scaling and model swapping, self-healing dead pods within 1 minute.
06 · Cluster
Demand forecasting and SLO control
Forecasts traffic and adjusts serving capacity using latency, throughput, and cost feedback.
07 · AWS Cloud
Capacity discovery and
auto-reservation
Monitors regional availability across compatible VMs and automatically reserves AWS capacity.
Inference inside
your AWS VPC
Inference inside
your AWS VPC
VPC deployment
Run inference inside your AWS account and VPC, within a network you control.
VPC deployment
Run inference inside your AWS account and VPC, within a network you control.
Data privacy
Your prompts, responses, logs, and model weights stay inside your AWS account.
Data privacy
Your prompts, responses, logs, and model weights stay inside your AWS account.
RBAC and access control
Assign project roles to control who can deploy models and change serving settings.
RBAC and access control
Assign project roles to control who can deploy models and change serving settings.
Scoped API keys
Manage access to models and clusters through revocable API keys.
Scoped API keys
Manage access to models and clusters through revocable API keys.
Built by AI infrastructure experts
Built by a team that shipped AI infrastructure and production software for research institutions, hospitals, big pharma, insurers, including Fortune 500 enterprises.
Built by AI infrastructure experts
Built by a team that shipped AI infrastructure and production software for research institutions, hospitals, big pharma, insurers, including Fortune 500 enterprises.
Frequently asked questions
Frequently asked questions
How quickly can you roll out Rivvr?
We send you a rollout checklist. Once you complete it, we install Rivvr and run system checks within 1–3 business days.
What if we don’t have AWS GPU quota?
Your account needs an approved GPU quota before Rivvr can provision the corresponding instances.
AWS GPU instances are often unavailable. How does Rivvr handle that?
R1 checks availability across compatible GPU instance types and automatically reserves capacity. Your deployment can use different compatible GPUs instead of waiting for one specific instance type.
Do we need to change our application?
For an OpenAI-compatible client, update the endpoint URL, model name, and inference API key.
How is Rivvr priced?
A platform fee plus a charge based on GPU capacity under management.