How a deployment runs

A dedicated deployment is set up together with you. This page describes the steps and what you need to bring to each.

1. Define the workload

   In a first call we agree on the model, the region, and your targets: expected requests per second, typical prompt and response length, and the latency you need. If you already use the model through the gateway, your logs give us these numbers.

2. Receive an offer

   You receive an offer with the GPU type and count, the region, the monthly price for the reserved capacity, and the terms for scaling up or down.

3. Deployment and load test

   We start the model on the reserved GPUs, tune it for your targets, and measure latency and throughput under your expected load. You receive the measured numbers before go-live.

4. Handover

   You receive the base URL, the model id and an API key. Requests use the same OpenAI-compatible format as the gateway, so in most code only those three values change.

Changing the model later

A newer version of the model, or a model trained for you, can be put behind the same endpoint. Your code stays as it is.

Start a project

Book a call and tell us which model you want to run and what volume you expect.