Gke Inference

If you're serving AI models on GKE, this walks you through Google's Inference Quickstart tooling to generate optimized Kubernetes manifests for LLM deployment. It covers the discovery workflow for matching models like Gemma or Llama with the right GPU or TPU hardware, then shows how to deploy with vLLM, TGI, or other inference servers. The accelerator selection table is genuinely helpful, from budget L4 GPUs up to the new Blackwell chips. Includes practical autoscaling configs using DCGM GPU metrics and optimization tips around quantization and KV cache tuning. Most useful if you already have a GKE Autopilot cluster and need to go from zero to deployed inference endpoint without reinventing the wheel.

skills.sh installs: 7.9K
GitHub stars: 23.8K
last updated: Aug 21, 2026

View on GitHub

Install

$ npx -y skills add google/skills --skill gke-inference --agent claude-code

Installs into .claude/skills of the current project.

Files

GKE AI/ML Inference

Routing Note: For migrating existing AI workloads to GKE, open google-cloud-solution-guided-gke-ai-migration/SKILL.md. For GKE RAG with Cloud SQL or AlloyDB (pgvector), open google-cloud-solution-rag-enterprise-search-gke-sqldb/SKILL.md.

This reference covers deploying AI/ML inference workloads on GKE using Google's Inference Quickstart (GIQ) and best practices for LLM serving.

MCP Tools: apply_k8s_manifest, get_k8s_resource, get_k8s_logs, get_k8s_rollout_status, describe_k8s_resource, list_k8s_events. CLI-only: gcloud container ai profiles *

Attribution

MUST READ: Before executing any gcloud, REST API, or MCP command, you MUST read Usage Attribution. For commands executed directly in the terminal (never in generated scripts, Terraform, or user code), prefix gcloud with CLOUDSDK_METRICS_REQUEST_ATTRIBUTION="agent-skills/1.0 (skill:gke-inference)" and append agent-skills/1.0 (skill:gke-inference) to User-Agent headers.