Tag
17 articles tagged with “Engineering”
Engineering5 min read
Spot nodes promise their owners the machine back within two minutes. Doing that without cutting off inference requests mid-answer took a draining state, a fix to a heartbeat that kept undoing it, and one switch that the whole platform obeys.
Read
Engineering7 min read
An NVIDIA H200 joined the network and our verification put its memory bandwidth at 2,319 GB/s, under half of the 4,800 on the spec sheet. The rent page warned renters about the card. We rented it ourselves, measured it another way, and found 4,110 GB/s. Here is how a bandwidth test that was right on every consumer card went wrong on a fast one, and the fix.
Read
Engineering9 min read
A new RTX PRO 6000 joined our network and got a quarter of the traffic of a pair of RTX 4090s it outperforms 3x under load. The GPU-name table had no entry for it, and our measured correction could not see the difference, because per-request speed is nearly equal on the two. What we replaced both with, the cold-start mistake we made on the way, and two routing bugs the new weights exposed.
Read
Engineering6 min read
A customer's coding agent showed us a screenshot: Error: Stream ended without finish_reason. On our side the request row said completed. Both were wrong in their own way. Here is what happens to a streamed completion when the GPU worker's connection drops, why the common gateway behaviour turns it into a silent truncation, and the three rules we now follow: retry before the first token, an explicit error event after it, and never count a cut stream as a success.
Read
Engineering7 min read
On a marketplace built from other people's machines, uptime is two jobs, not one. The platform keeps itself always-on with rolling, zero-downtime deploys; the provider keeps the GPU alive for the length of a rental. Here's where the line sits — and how reputation holds each side to it.
Read
Engineering15 min read
A follow-up question costs 0.54 seconds on the worker that answered the first one and 2.56 seconds on any other — the whole prompt prefilled again. We measured that across a live fleet, built cache-aware routing for two router replicas that must agree, and then found why our hybrid 27B showed a 0% hit rate: vLLM leaves prefix caching off on Mamba-style models unless you pass two arguments. With them, a repeated 13,000-token prompt goes from 7.2 s to 1.6 s — and a third flag brings the same win down to the 1,200-token chats where most traffic lives.
Read
Engineering4 min read
Our OpenAI-compatible API answered curl, Node, python-requests and every SDK with 200, and Python's urllib with a 403 that never reached our servers. The cause was Cloudflare's Browser Integrity Check, which is right for web pages and wrong for an API. How to recognise error code 1010, how to prove it is the edge and not your app, and the Configuration Rule that turns the check off for /v1/ only.
Read
Engineering4 min read
A provider re-ran our Mac install command and every launch of the agent died instantly with "zsh: killed". The binary was fine, the same file ran on other Macs, and there was no crash log. The cause is how macOS caches code signatures per file: curl -o over an existing executable leaves one the kernel refuses to start. Here is how to recognise it, why it happens, and the fix for your install scripts.
Read
Engineering10 min read
The same model ID is now served by vLLM on Linux GPUs, vLLM on Windows/WSL, and llama.cpp on Apple Silicon Macs, and the caller cannot tell which one answered. Here is what had to be made equal for that to be true, the tokens-per-second we measured on an M1 Ultra and an M4 Max next to the GPUs, the 27B experiment that failed first and the rules it left behind, and a self-update bug that only exists when your inference server is a child process.
Read
Benchmarks & Comparisons8 min read
We had a rule that a model which fits on one GPU should run as one replica per GPU. On a two-RTX-4090 machine with a 27B model, that rule was wrong by a factor of two to four, and the reason was not the parallelism at all. Here are the measurements, the rule that replaced it, and how the router now sends a 40k-token prompt to the worker that can hold it.
Read
Engineering10 min read
A mixed GPU fleet has one setting per model, and it has to be safe on the slowest machine in it. We measured what that costs: an RTX 5090 running at 8% of its memory bandwidth because a 24GB card three racks away needed a flag. Two arguments, applied only to the cards that could take them, made it fifteen times faster. Here is what the engine measures per node, what it changes, and the two things that surprised us.
Read
Engineering6 min read
Ask any AI provider which machine processed your prompt and which jurisdiction it sat in, and you will not get an answer. A decentralized network is the only architecture that could answer it — so we went looking at what ours can actually prove today. The honest version: less than we would like, and more than anyone else offers.
Read
News & Updates5 min read
Providers run GPUs on home connections behind NAT — no static IP, no port forwarding, no inbound anything. Renters still get an HTTPS address that works the moment the container starts and stops existing when the session ends. How we build and destroy one per rental, and the three constraints that shaped the design.
Read
Benchmarks & Comparisons8 min read
Two RTX 4090s on our network, same container, same model, same arguments. One runs Windows with WSL2 and benchmarks better on every hardware axis we measure. It serves inference 48% slower — three times the penalty the published figures report. Here is the measurement, what we ruled out, and why we think large quantized models are the worst case for GPU virtualisation.
Read
Engineering9 min read
Every node has reported its connection speed since January. We published a 100 Mbps requirement in June. Nothing connected the two, so in August five of the nodes we were offering to renters were below it, one at 15 Mbps. What we found, why a single speed test can't be the gate, and why we kept those nodes on the network anyway.
Read
Engineering10 min read
Anyone can claim an H100. On a network built from other people's hardware, a claim is worth nothing — so we measure instead. How ParalonCloud verifies every GPU by benchmarking it, what the real numbers look like across 80 verified nodes, and what the method still can't catch.
Read
Engineering15 min read
How ParalonCloud runs LLMs, vision, image generation, speech, rendering and GPU rentals on one distributed network of consumer GPUs — with a single agent, a VRAM-aware scheduler, and OpenAI-compatible gateways. The architecture, the numbers, and what it makes possible for partners.
Read