Inside the Paralon Engine: One Agent, One Scheduler, Any AI Workload

In the last ten days we shipped a vision LLM, an image-generation API and a Discord bot on top of the same network — without changing the engine. Each one was a new row in a table, a container image, and a thin gateway. That is not an accident; it is the whole point of how ParalonCloud is built.

This post is the architecture tour we'd give a CTO, a partner or an investor: what the engine is, why it's shaped this way, what it runs today, the numbers behind it, and what it opens up. No hype — the parts that are hard are called hard.

The problem: a GPU network is not "a vLLM with a website in front"

ParalonCloud is a distributed compute network: independent providers connect machines — consumer GPUs like the RTX 3090/4090/5090, CPU-only boxes, Apple Silicon Macs — and builders consume them through APIs and rentals. That premise breaks most off-the-shelf designs:

• Hardware is heterogeneous and changes hourly. 42 distinct GPU models have registered so far, 179 nodes from 31 countries, on Linux, Windows/WSL2 and macOS. Nodes appear, disappear, and come back.
• Workloads are not one thing. Chat completions, vision, image generation, speech-to-text, text-to-speech, Blender rendering, interactive Jupyter rentals — different containers, ports, health semantics, and billing models.
• Trust has to be earned per machine. A node's claimed GPU has to be verified before it serves anyone, and a provider's machine must never be counted twice.
• Money moves in both directions. Every minute of compute is metered, split between provider and platform, and exposed to both sides in real time.

The engine solves this with three primitives — one agent, one scheduler, any workload — and one strong opinion: the network should not know or care what it is running.

The shape of the system

Three layers, each doing one job:

1. Gateways are deliberately thin. Each speaks a standard API shape (OpenAI chat/images/audio, a small REST for rentals), authenticates the key, enforces limits, meters usage, picks a worker and proxies. They hold no model logic — which is why adding the image API was a few hundred lines.
2. node-manager is the brain: it knows every node's hardware, decides what runs where, keeps the live worker registry, verifies GPUs, and tunnels requests to agents over a persistent WebSocket — providers never open inbound ports.
3. The agent is the hands: one small Go binary (or a native Mac agent) per provider machine that pulls images, starts containers with the right GPUs, health-checks them, reports back, and keeps itself up to date.

One agent: any container, any GPU, safely

The agent is the same binary on a 12-GB RTX 3060, a 2×5090 Threadripper workstation and a 4-GPU mining rig. On start it authenticates with a node token and its own image digest — only agent builds we've published can join — then reports hardware (GPUs, VRAM, driver, CPU, RAM, disk, network speed, location) and waits for commands.

What it does well, and what we'd highlight to anyone evaluating the platform:

• Any container is a workload. For LLMs it builds the vLLM command line itself — GPU selection by compute capability, homogeneous GPU grouping on mixed rigs, tensor/data-parallel planning, a persistent model cache so restarts don't re-download weights. For everything else (image, speech, render, rental) it runs a generic worker: the image, ports, environment, mounts and health endpoint come from the model definition. No agent release was needed to add image generation.
• Health before traffic. A container is only registered as a worker after its health endpoint answers; until then it receives nothing. Crashes are reported upstream with logs, so the scheduler can retry elsewhere and operators can see why.
• Self-updating, gated. Agents check for approved builds and update themselves when idle; rollout is controlled centrally, so a bad build can't sweep the fleet.
• Built for messy hardware. A large share of consumer GPUs live on Windows under WSL2/Docker Desktop. The agent handles the realities: driver hiccups, memory reporting quirks, images pre-pulled so a node is rent-ready the moment it comes online.

One scheduler: VRAM-aware, priority-based, pinnable

node-manager decides placement from three things: what each model needs (minimum VRAM, desired worker count, priority, optional node pinning), what each node has (verified GPUs, free VRAM after a safety derate, what's already running, whether it's currently rented), and what's live (the registry of ready workers, refreshed by heartbeats).

The allocation policy is simple enough to reason about and explain to a provider:

1. Pinned first — a model can be targeted at specific nodes (a customer's dedicated machines, a canary).
2. Fill minimums — models below their desired worker count get the next eligible free node, largest-VRAM models first.
3. Then share proportionally — remaining capacity is split by priority, so the flagship model gets more nodes without starving the rest.

Two properties matter commercially. Isolation: a node serves one workload at a time, and a rented node is never also an inference worker — the renter gets the whole GPU they pay for. Graceful degradation: when a node disappears, its workers expire from the registry within a minute and traffic flows to the others; when it returns, it's re-verified and re-used.

Trust is measured, not declared

A node's GPU claim is checked by running a verification container on it that actually allocates VRAM and measures memory bandwidth and compute — a 5090 reports ~1.5 TB/s, a 4090 ~1 TB/s, and a spoofed card reports nothing. Only verified nodes are scheduled. The same layer enforces one node per physical machine, so a provider can't earn twice for one box, and keeps an allowlist of approved container digests for the images we ship.

Any workload: a new model is a row

This is the part we're proudest of, and the part that's easiest to underestimate. Everything the network can run is described by one table. The columns that matter:

Column	What it lets us do
service_type	llm takes the vLLM path; image, stt, tts, … take the generic path and are routed by the matching gateway
docker_image, docker_args, docker_env, docker_binds, custom_command	run any container, with its own config and model files
service_port, health_endpoint	how the agent knows it's alive
min_vram_mb, min_workers, priority, target_nodes	how the scheduler places it
price_* columns	how it's metered — per token, per image, per minute

Proof by shipping, all in the last two weeks:

• Vision LLM — Qwen 3.8 27B went live as one row pointing at a newer vLLM image; the API gained image input without a code change.
• Image generation — a small diffusion container plus a ~300-line gateway gave the network POST /v1/images/generations (see the launch post). First image through the public endpoint: the same day.
• Discord /imagine — a bot that calls that gateway; images never touch a disk.

Earlier on the same chassis: speech-to-text and text-to-speech workers, a Blender render farm that turns a .blend upload into frames on 4090s, GPU rentals with a REST API and an MCP server, and Apple Silicon Macs serving small models through Ollama — proxied transparently, because the Mac is behind the same WebSocket as everything else.

Thin gateways, standard APIs

Every consumer-facing surface is OpenAI-compatible where an OpenAI shape exists (chat, images, audio) and plain REST where it doesn't (rentals). One prlc_ key works across all of them; limits are per capability (chat requests per minute, images per minute and per day) and editable per key. The chat gateway routes with weighted power-of-two-choices — faster GPUs get more traffic, the queue absorbs bursts, and streaming is passed through end to end so tokens arrive as they're generated.

The practical consequence for builders: switching to Paralon is a base-URL change. The practical consequence for us: any model we add is instantly usable by every SDK, coding agent and tool that already speaks these APIs.

Money, both ways

The economics are built into the engine, not bolted on. Builders pay in credits (USDC/USDT on Ethereum or Solana today; card checkout is built), spent per token, per image, or per minute of rental. Providers earn the majority of what their hardware generates — 80% on rentals, paid out in USDC — with the platform fee funding the network. Referrals pay both sides. Everything is visible live on both the provider and the builder side, and inference is free during the beta on purpose: the fastest way to find out what people build is to remove the meter for a while.

By the numbers

• 179 nodes registered from 31 countries; 42 distinct GPU models seen; 17 real nodes online as this is written.
• 11 Go services behind 25 containers in the control plane, plus the agents themselves (Linux/WSL and native Mac); 53 schema migrations since January.
• Workloads live today: LLM chat, vision, image generation, STT/TTS, Blender rendering, Jupyter rentals, Mac/Ollama — one agent, one scheduler.
• Time to add a new workload type: about a day, most of it testing. Time to add a new model of an existing type: minutes.
• Measured on the network: a 27B vision model answers in ~0.25 s to first token; a 1024² image in ~5 s; Blender frames on 4090s in tens of seconds.

What it doesn't do yet — and what's next

We'd rather you hear it from us:

• Multi-node training (DDP/DeepSpeed across providers) is designed, not shipped — the network is request/response and single-node-batch today.
• Rentals are HTTP-only (Jupyter); SSH and TCP access arrive with the next ingress layer, which also moves rental URLs onto our own domain.
• Files are moving to object storage: render inputs/outputs will go straight to R2 with presigned URLs, taking the control plane out of the data path.
• Image editing, video, embeddings on CPU nodes are the next workload types — each is, again, a row and a small gateway.

What this makes possible for partners

Because the engine is workload-agnostic, the interesting conversations aren't "do you serve model X" but:

• Bring your own model or container. A fine-tune, a private model, a custom inference server — we run it on dedicated nodes with your own endpoint, keys and billing. The scheduler's pinning and minimum-worker guarantees exist exactly for this.
• Self-hosted / sovereign deployments. The same engine runs inside a company's own fleet: agents on their machines, their control plane, their data staying put.
• Capacity partnerships. Data centers and GPU owners plug in with one binary and get scheduling, metering, verification and payouts for free.

If any of that describes you, talk to us — or just create a key and start with the APIs; the engine behind them is the one described here.

FAQ

What is the Paralon engine? The software that turns independent machines into one network: an agent per machine, a central scheduler and registry, and OpenAI-compatible gateways per capability. It runs LLMs, vision, image generation, speech, rendering and GPU rentals on the same infrastructure.

Why consumer GPUs? They exist in huge numbers, sit idle most of the day, and for a large class of workloads (27B-class LLMs, image generation, rendering) they're cost-efficient. The engine's job is to make a heterogeneous, intermittent fleet behave like a reliable service — scheduling, verification and isolation are what make that possible.

How do you add a new model or workload? A model of an existing type is a row: image, args, VRAM, priority. A new type of workload is a container plus a thin gateway; image generation took about a day end to end.

Is it secure for providers and users? Agents authenticate with per-node tokens and only approved builds can join, providers never open inbound ports, each node runs one workload at a time, GPUs are verified by measurement, and machines can't be registered twice. Users reach workers only through the gateways and the encrypted tunnel.

Can we run the engine on our own hardware? Yes — that's the enterprise / self-hosted offering: same agent, same scheduler, your machines and your data.

Where can I read more? The deep-dive on placing LLMs across mixed GPUs is here; the image API launch is here; the APIs are documented at /docs.