ardelvo.

Size your workload.

Turn a text-inference workload into an initial hardware and energy budget. Adjust the assumptions to explore the trade-offs.

A starting point for sizing—not a performance guarantee.

The prefilled example uses sample throughput and budget values. A benchmark of your model is needed to validate GPU count and response time. Calculations stay in your browser.

01 Your inference workload

For text generation with a dense transformer model. Training, images, video and mixture-of-experts models need a separate assessment.

Use a peak-hour request rate. For memory safety, use your maximum planned per-request context, rather than a mean that hides long requests. Input tokens drive cache sizing; throughput must be benchmarked with the same input/output mix.

02 Hardware & throughput

We buy whole servers and round GPU groups to 1, 2, 4 or 8. Validate the selected model’s parallelism and GPU interconnect before ordering.

Presets change GPU memory and power only. Update the throughput benchmark and supplier prices for your selected GPU. Confirm OEM server layout, cooling and interconnect compatibility; SXM and OAM modules require compatible platforms.

Selected GPU manufacturer specifications
03 Storage & operating cost
Model architecture & infrastructure assumptions

Defaults describe an illustrative 8B grouped-query-attention model. Parameter count alone does not determine cache size. Update these values from the actual model configuration when changing models.

GPU TDP plus host allowance is a sizing approximation. PUE adds facility overhead, including cooling. Use an OEM whole-server specification and a qualified facility review for electrical design.

Advanced workload & resilience checks

Use values from the actual model configuration and workload benchmarks. These checks apply to conventional dense-transformer attention, not MLA, MoE, sliding-window or hybrid models.

A separate prefill benchmark adds prefill and decode service demand. Leave it at zero if the output benchmark already measures the complete prompt/output workload. Reload bandwidth assumes concurrent cold reads without shared caches; add protocol overhead and allow for deserialization. It does not estimate training checkpoint I/O.

Adjust sample hardware prices (USD)

These are illustrative budget inputs, not live prices or an Ardelvo quote. Replace every unit cost with supplier pricing. Server base excludes GPUs, RAM and storage to avoid double counting.

Excludes tax, software, support, installation, networking purchase, storage appliances, backups, financing and facility upgrades. Electricity is the only recurring cost calculated.

Estimates update automatically as you edit.

Every assumption is visible.

GPU memory & replicas

Weight memory = parameters × weight bits ÷ 8. KV cache = 2 × layers × KV heads × head dimension × cache bytes × total tokens × requests per replica. We add your overhead percentage, then fit each GPU within its usable memory allowance. Tensor parallelism divides weight memory but replicates KV heads when there are fewer KV heads than GPUs. Attention heads must divide across the selected GPU group.

The replica count is the larger of throughput demand and peak-concurrency demand. Optional standalone prefill demand is added to decode service demand. Additional spare servers are fully budgeted and powered. Throughput uses output tokens per second, with your benchmark and utilization target. Quantization metadata, activations and workspaces are approximated by overhead.

RAM & storage

Host RAM is a heuristic: at least your minimum per server, or twice the weight memory of hosted replicas, rounded up in 64 GB steps. This is not model-specific CPU memory profiling.

Storage includes your dataset, two weight copies per replica, retained logs and 0.5 TB per server for boot/software, plus 20% capacity headroom. It excludes backup copies and storage replication.

Power & electricity

Peak IT allowance = all purchased GPUs × GPU watts + servers × host watts + shared infrastructure watts. Facility peak = IT peak × PUE. Monthly kWh = facility peak × average draw fraction × powered-on hours.

Idle draw, temperature and workload dynamics are simplified. Results do not specify circuits, UPS capacity, cooling equipment or redundancy.

Budget

Equipment subtotal = GPUs + server bases + RAM + storage at your unit prices. Contingency is applied to that subtotal. Electricity uses the entered USD/kWh price.

All memory figures use decimal GB and storage uses decimal TB. Multi-GPU memory assumes supported tensor parallelism; topology, kernels, latency targets and failover operation still require validation.

Technical background: vLLM KV-cache replication · Hugging Face inference memory · KV cache strategies · NVIDIA H200 memory and power specifications