How Tasks Work

Inference on the AIOZ AI API is asynchronous. When you submit work, the API accepts it, returns a task ID immediately, and runs the actual compute on the AIOZ network, which may take anywhere from a couple of seconds to several minutes depending on the model and current network load. You read the result back by polling the task until it reaches a terminal status.

This page walks through the lifecycle, when each status applies, and the patterns you'll use to work with tasks day-to-day.

The lifecycle

A task progresses through these statuses:

A typical task starts in in_queue, transitions to computing once a node picks it up, then ends in success or failed. success, failed, and cancelled are all terminal: once you see one of them, the task is done and its status won't change again. Stop polling as soon as you observe any of them.

End-to-end duration depends on three things: the model itself (a small classifier finishes in a second; an image generator takes much longer), the inputs you provide (larger files take longer to process), and how busy the network is at the time (more queued tasks ahead of yours means more time in in_queue).

Creating and reading a task

Every inference follows the same shape: create a task, then poll it until it's terminal.

1. tasks.create(model_id, input_params, files)   ->  task_id
2. loop:
     task = tasks.get(task_id)
     if task.status is terminal:
         break
     sleep a couple of seconds
3. read task.result

The Quickstart has runnable code for this pattern in curl, Python, Node, and Go.

A reasonable polling interval is 2 to 5 seconds. Inference duration depends on the model: short tasks complete in a single poll, while larger ones take several. Polling more aggressively won't make a task complete any faster; it just adds traffic. For long-running models, consider increasing the interval over time (for example, double it after every few polls up to a cap) so quick tasks stay snappy and slow ones don't generate excessive requests.

When a task fails

If a task ends in failed, the message field on the returned task describes what went wrong. Common reasons are malformed inputs, files the model couldn't read, or errors raised inside the model itself. Failed tasks are not retried automatically. If you want to retry, call tasks.create again with corrected inputs.

You aren't charged for failed tasks; only successful inferences draw from your balance. See How Billing Works.

Cancelling a task

If you submit a task you no longer need, call tasks.cancel(task_id). Cancellation is best-effort: a task that's still in_queue can be cancelled, but once it has moved to computing it can no longer be stopped; it will run to completion (and you'll be charged for it if it succeeds). For that reason, cancelling is most useful as a quick correction right after creation, not as a way to abort work that's already running.

What's in a task

Each task object carries:

• id / task_id: the identifier you use when polling and cancelling.
• status: the current lifecycle status.
• cost: the inference cost, as a decimal string. Meaningful only on success; you aren't charged for failed or cancelled tasks.
• message: present on failed, describing the error.
• input_data, input_format, output_format: what you sent and what the model declared as its input and output schemas.
• result: the model's output, present on success. Its shape depends on the model; see the model's own page for its output schema.
• created_at / updated_at: ISO 8601 timestamps for when the task was created and when its status last changed.

See the API Reference for the full field types and the responses of each task endpoint.