Batch API

The Batch API runs a file of requests for you. You upload a JSONL file, create a batch, and download a file of results when it finishes, at half the per-token price of the same requests sent live. No client code to pace requests, handle 429s or retry: that part is ours.

Batches run on the capacity live traffic is not using, so they never slow your real-time requests and they do not count against your key's per-minute or in-flight limits. Every batch completes, expires or is cancelled within 24 hours; most finish far sooner.

The request and response formats are OpenAI's, so the openai SDKs' files and batches methods work by changing the base URL and the key.

1. Prepare the input file

One request per line, each with a custom_id you choose, the endpoint in url, and the request body exactly as you would send it live:

{"custom_id": "ticket-1", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "qwen3.8-27b", "messages": [{"role": "user", "content": "Classify: my order never arrived"}], "max_tokens": 50}}
{"custom_id": "ticket-2", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "qwen3.8-27b", "messages": [{"role": "user", "content": "Classify: the zipper broke"}], "max_tokens": 50}}

supported url	body
/v1/chat/completions	chat completions, including response_format, tools and vision
/v1/embeddings	embeddings
/v1/responses	Responses API

All lines of a batch use the same url. Limits: 50,000 requests and 50 MB per file. custom_id must be unique in the file. stream is ignored.

2. Upload it

curl https://paraloncloud.com/v1/files \
  -H "Authorization: Bearer prlc_..." \
  -F purpose=batch \
  -F [email protected]

{"id": "file-7c1e...", "object": "file", "bytes": 412, "filename": "requests.jsonl", "purpose": "batch", "status": "processed", "created_at": 1789400000}

3. Create the batch

curl https://paraloncloud.com/v1/batches \
  -H "Authorization: Bearer prlc_..." \
  -H "Content-Type: application/json" \
  -d '{"input_file_id": "file-7c1e...", "endpoint": "/v1/chat/completions", "completion_window": "24h"}'

Every line is checked before anything runs. If one line is malformed, names a model that is not served, or repeats a custom_id, the batch is created with status: "failed" and errors lists the problems with their line numbers, and nothing is charged.

4. Poll, then download

curl https://paraloncloud.com/v1/batches/batch_3f9a... -H "Authorization: Bearer prlc_..."

{
  "id": "batch_3f9a...", "object": "batch", "endpoint": "/v1/chat/completions",
  "status": "completed", "input_file_id": "file-7c1e...",
  "output_file_id": "file-a41d...", "error_file_id": null,
  "request_counts": {"total": 2, "completed": 2, "failed": 0},
  "created_at": 1789400010, "in_progress_at": 1789400010, "completed_at": 1789400031, "expires_at": 1789486410
}

status moves through in_progress and finalizing to completed, or ends as failed (validation), expired (24 hours passed) or cancelled. request_counts updates while it runs.

curl https://paraloncloud.com/v1/files/file-a41d.../content -H "Authorization: Bearer prlc_..." > results.jsonl

Each output line carries your custom_id and the response as the live endpoint would have returned it. Lines are not guaranteed to be in input order; match on custom_id.

{"id": "batch_req_...", "custom_id": "ticket-1", "response": {"status_code": 200, "request_id": "batch_req_...", "body": {"id": "chatcmpl-...", "choices": [...], "usage": {...}}}, "error": null}

Requests that failed are in the file named by error_file_id, in the same shape, with the status code and error body, or an error object for requests that never ran (cancelled or expired).

Cancel, list, clean up

call
POST /v1/batches/{id}/cancel	stops pending requests; the batch ends as cancelled with the results so far
GET /v1/batches?limit=20&after=batch_...	your batches, newest first
GET /v1/files, GET /v1/files/{id}	your files
DELETE /v1/files/{id}	deletes a file

Files and batches belong to the API key that created them: another key, even on the same account, does not see them. Use the same key to create a batch, poll it and download its results.

Files are kept for 30 days after they are uploaded or written, input and output files alike, and then deleted; download your results before then. The batch itself stays in your list.

With the OpenAI Python SDK

from openai import OpenAI
import time

client = OpenAI(base_url="https://paraloncloud.com/v1", api_key="prlc_...")

f = client.files.create(file=open("requests.jsonl", "rb"), purpose="batch")
batch = client.batches.create(input_file_id=f.id, endpoint="/v1/chat/completions", completion_window="24h")

while batch.status not in ("completed", "failed", "expired", "cancelled"):
    time.sleep(30)
    batch = client.batches.retrieve(batch.id)

if batch.output_file_id:
    results = client.files.content(batch.output_file_id).text

Pricing and limits

• Price: every completed request is billed at 50% of the model's per-token price, from the same credits as live requests (billing). Requests that fail are not billed.
• Keys: a free key's 250,000-token trial covers batches too; when it runs out, or a premium account's balance reaches zero, the remaining requests of the batch fail with that reason.
• Capacity: batches use capacity that live requests leave free, so they run fastest at quiet hours and never take a slot a live request is waiting for.
• Not supported: /v1/completions, file purposes other than batch, completion windows other than 24h.