# Qwen2.5-72B

A 72B dense chat model (AWQ INT4) on a single 80GB GPU, on AKS or Nebius.

Source: /examples/qwen2.5-72b/

<!-- vale write-good.Passive = NO -->
A 72B dense chat model served from an AWQ INT4 quantization on a single 80GB
GPU: one `Standalone` engine fed by a `ModelCache`. The platform side comes in
two shapes - an A100 on AKS and an H100 on Nebius - and the ML side is the same
manifest for both. The deployment carries no `clusterSelector`, so device
capacity alone matches it to either platform.

These manifests mirror the repository's AKS and Nebius demos. Apply the
platform side first (pick a tab), then the ML side.

## Platform

{{< tabs >}}
{{< tab "AKS" >}}
{{< manifests "examples/qwen2.5-72b/inference-class-aks.yaml" >}}

{{< manifests "examples/qwen2.5-72b/inference-cluster-aks.yaml" >}}
{{< /tab >}}
{{< tab "Nebius" >}}
{{< manifests "examples/qwen2.5-72b/inference-class-nebius.yaml" >}}

{{< manifests "examples/qwen2.5-72b/inference-cluster-nebius.yaml" >}}
{{< /tab >}}
{{< /tabs >}}

## Deployment

{{< manifests "examples/qwen2.5-72b/model-cache.yaml" >}}

{{< manifests "examples/qwen2.5-72b/model-deployment.yaml" >}}

{{< manifests "examples/qwen2.5-72b/model-service.yaml" >}}
<!-- vale write-good.Passive = YES -->