Tag

Infrastructure

7 articles tagged with “Infrastructure”

The Paralon capybara holding a pressure gauge up to an open GPU server that glows emerald green, looking at the dial with suspicion

Oct 11, 20267 min read

Our Benchmark Read an H200 at 48% of Spec. The Benchmark Was Wrong.

An NVIDIA H200 joined the network and our verification put its memory bandwidth at 2,319 GB/s, under half of the 4,800 on the spec sheet. The rent page warned renters about the card. We rented it ourselves, measured it another way, and found 4,110 GB/s. Here is how a bandwidth test that was right on every consumer card went wrong on a fast one, and the fix.

Read

The Paralon capybara holds a graphics card lit from within by a stack of glowing green blocks — a cached prompt prefix — while a second, empty card floats dark behind it

Engineering

Sep 16, 202615 min read

Route to the GPU That Already Has Your KV Cache

A follow-up question costs 0.54 seconds on the worker that answered the first one and 2.56 seconds on any other — the whole prompt prefilled again. We measured that across a live fleet, built cache-aware routing for two router replicas that must agree, and then found why our hybrid 27B showed a 0% hit rate: vLLM leaves prefix caching off on Mamba-style models unless you pass two arguments. With them, a repeated 13,000-token prompt goes from 7.2 s to 1.6 s — and a third flag brings the same win down to the 1,200-token chats where most traffic lives.

Read

Three GPU nodes on a black field, each receiving a different emerald configuration path, with capacity bars of unequal height beneath them

Engineering

Sep 6, 202610 min read

One Config Is Always the Weakest Card's Config

A mixed GPU fleet has one setting per model, and it has to be safe on the slowest machine in it. We measured what that costs: an RTX 5090 running at 8% of its memory bandwidth because a 24GB card three racks away needed a flag. Two arguments, applied only to the cards that could take them, made it fifteen times faster. Here is what the engine measures per node, what it changes, and the two things that surprised us.

Read

The Paralon capybara in its straw hat stacking emerald coins beside a graphics card plugged into a glowing emerald chart panel

News & Updates

Sep 2, 20267 min read

Two Ways Your GPU Earns Here

Connect a graphics card to ParalonCloud and it earns on two independent tracks: real USDC whenever someone rents it, and Season 0 points for every minute it stays online, whether anyone rents it or not. Here is how the split works, what each card is worth per hour, and why the cards we want most are the ones with the most VRAM.

Read

The Paralon capybara holding a glowing emerald globe with node points lit across Europe, a magnifying glass in its other paw

Engineering

Sep 1, 20266 min read

Where Does Your Inference Actually Run?

Ask any AI provider which machine processed your prompt and which jurisdiction it sat in, and you will not get an answer. A decentralized network is the only architecture that could answer it — so we went looking at what ours can actually prove today. The honest version: less than we would like, and more than anyone else offers.

Read

The Paralon capybara holding a glowing emerald cable that arcs from a home desktop tower up into a floating browser address bar

News & Updates

Sep 1, 20265 min read

A URL That Lives as Long as the Rental

Providers run GPUs on home connections behind NAT — no static IP, no port forwarding, no inbound anything. Renters still get an HTTPS address that works the moment the container starts and stops existing when the session ends. How we build and destroy one per rental, and the three constraints that shaped the design.

Read

The Paralon capybara studying two identical graphics cards, one dimmed behind frosted glass, an emerald throughput meter beside them

Benchmarks & Comparisons

Aug 31, 20268 min read

The Faster Card Was Half the Speed

Two RTX 4090s on our network, same container, same model, same arguments. One runs Windows with WSL2 and benchmarks better on every hardware axis we measure. It serves inference 48% slower — three times the penalty the published figures report. Here is the measurement, what we ruled out, and why we think large quantized models are the worst case for GPU virtualisation.

Read