Léxico / CONCEPTO

Inference

The process of running input data through a trained AI model to generate predictions or outputs, as opposed to training the model.

Inference is when a trained model does its job — you send it a prompt, and it generates a response. Every time you chat with an AI, you’re running inference.

Inference vs. Training

Training: The model learns from data (expensive, done once)
Inference: The model applies what it learned (fast, done per request)

What Affects Inference Cost

Input tokens — how much text you send
Output tokens — how much text the model generates
Model size — larger models cost more per token
Streaming — delivers tokens as they’re generated instead of waiting for the full response