Skip to main content

Deployment Quickstart

Use this tutorial after cluster activation to deploy one available model and confirm that its service reaches Ready.

HUMAN QUICKSTARTGo from activated GPU capacity to a running modelConfirm your cluster, select a platform or custom model, then configure deployment resources.

Prerequisites​

  • At least one activation shows Activated.
  • GPU Observability shows the nodes and GPU capacity you expect to allocate.
  • A model is available in Model Gallery or My Models.

Choose a Model Source​

Start from a model that is already available to your account.

Model Gallery

Browse platform models, review model details, and choose Deploy.

Learn More
My Models

Upload or select a custom or fine-tuned model, then choose Deploy.

Learn More

Quick Deployment Steps​

  1. Confirm cluster capacity. Open GPU Observability from Activate Cluster and confirm that the expected nodes and GPU capacity are healthy.
  2. Select a model. Choose a model from Model Gallery or My Models.
  3. Configure resources. Select your cluster and configure the service topology, replicas, and optional KV cache settings.
  4. Deploy and verify. Create the deployment, wait for Ready, then verify its serving workload in GPU Observability.

Verify the Result​

The quickstart is complete when the deployment shows Ready, its details show the intended cluster and model, and GPU Observability lists the service under Model Serving. If the deployment shows Failed, open its error message and continue with Manage Deployments.

Troubleshooting​

No activated cluster is available​

Return to Activate Cluster, finish the activation workflow, and confirm the expected capacity in GPU Observability.

The deployment reaches Failed​

Open the status error in Deployments. Compare the selected model and GPU requirements, then follow Manage Deployments for the supported restart and recovery actions.

Best Practices​

Resource Selection
  • Match the GPU type to your model's size.
  • Start with a single replica for testing.
  • Use P/D Disaggregated for high-concurrency workloads.
Service Topology
  • Choose Standard mode for simple, low-traffic use cases.
  • Enable Distributed KV Cache when replicas serve overlapping context.

Next Steps​

GPU Observability

Validate cluster capacity before and after deployment.

Learn More
Call Models via API

Call your deployed models through the unified API endpoint.

Manage Deployments

Monitor state and perform supported lifecycle operations.

Learn More