When you need a GPU for AI (and when you do not)

A GPU massively accelerates AI tasks, but it costs more. Here is when you really need one.

Runs fine on CPU

• Small language models (1-3 billion parameters) for simple tasks.
• Low volumes, where a few seconds of waiting are acceptable.
• Occasional audio transcription with small Whisper models.

You need a GPU

• Large language models (over 7 billion parameters) with fast responses.
• Image generation with Stable Diffusion, see image generation.
• Training or fine-tuning models.
• Many concurrent requests (multiple users).

The practical difference

A model that responds in 30 seconds on CPU can respond in under a second on a GPU. For an app used by several people, the GPU becomes necessary.

Hourly billing for AI

With hourly billing you can start a GPU server only for as long as you need (for example for a batch of generations or a fine-tuning run) and delete it after, paying only for the hours used. See the GPU plans.