Tag
3 articles tagged with “Reliability”
Engineering5 min read
Spot nodes promise their owners the machine back within two minutes. Doing that without cutting off inference requests mid-answer took a draining state, a fix to a heartbeat that kept undoing it, and one switch that the whole platform obeys.
Read
Engineering6 min read
A customer's coding agent showed us a screenshot: Error: Stream ended without finish_reason. On our side the request row said completed. Both were wrong in their own way. Here is what happens to a streamed completion when the GPU worker's connection drops, why the common gateway behaviour turns it into a silent truncation, and the three rules we now follow: retry before the first token, an explicit error event after it, and never count a cut stream as a success.
Read
Engineering7 min read
On a marketplace built from other people's machines, uptime is two jobs, not one. The platform keeps itself always-on with rolling, zero-downtime deploys; the provider keeps the GPU alive for the length of a rental. Here's where the line sits — and how reputation holds each side to it.
Read