azure-aigateway
microsoft/azure-skills
Configure Azure API Management as an AI Gateway for model governance, cost control, and safety policies.
What is azure-aigateway?
This skill sets up Azure API Management (APIM) as a gateway to govern AI models, MCP tools, and agents. Use it when you need semantic caching, token limits, content safety filtering, load balancing, jailbreak detection, or cost tracking for AI workloads.
- Apply semantic caching to reduce AI costs by 60-80%
- Enforce token limits and rate limiting for cost control and MCP tool protection
- Implement content safety and jailbreak detection policies
- Add Azure OpenAI and AI Foundry model backends with managed identity authentication
- Track token usage metrics and load balance across AI models
- Convert APIs to MCP and import OpenAPI specifications into the gateway
How to install azure-aigateway
npx skills add https://github.com/microsoft/azure-skills --skill azure-aigateway- Azure subscription with API Management service deployed (use azure-prepare skill)
- Azure CLI (az) installed and authenticated
- Access to Azure OpenAI or AI Foundry resources to add as backends
- Appropriate RBAC roles to create backends and assign permissions
How to use azure-aigateway
- 1.Deploy or access an existing Azure API Management instance
- 2.Discover and add AI backends (Azure OpenAI, AI Foundry) using az apim backend create
- 3.Configure managed identity authentication between APIM and your AI resources
- 4.Apply governance policies in order: authentication, semantic cache lookup, token limits, content safety, backend selection, metrics
- 5.Test the gateway endpoint using curl or your client with the subscription key
- 6.Monitor token usage and adjust policy thresholds based on cost and performance metrics
Use cases
- Protect expensive AI model endpoints with token limits and semantic caching to reduce costs
- Govern multi-tenant access to AI models with subscription keys and rate limiting
- Filter harmful content and detect jailbreak attempts before requests reach the model
- Load balance requests across multiple Azure OpenAI deployments or AI Foundry models
- Monitor token consumption and enforce usage quotas per subscription or API consumer
- Platform engineers managing shared AI infrastructure
- DevOps teams implementing AI governance and cost controls
- Security teams enforcing content safety and compliance policies
- AI application developers needing multi-model load balancing and caching
- Teams converting internal APIs to MCP tool specifications
azure-aigateway FAQ
Apply in this order: authentication (managed identity), semantic cache lookup, token limits, content safety, backend selection, and metrics tracking. See references/policies.md for the complete example.
Semantic caching typically saves 60-80% on AI costs by caching semantically similar requests and reusing responses without calling the model.
Use az apim backend create with your Azure OpenAI endpoint URL, then grant APIM's managed identity the 'Cognitive Services User' role on the resource.
Increase the tokens-per-minute threshold in the token-limit policy, or add load balancing across multiple backends to distribute the load.
Yes. You can rate-limit MCP tool access using rate-limit-by-key policy, and convert APIs to MCP specifications for exposure through the gateway.
Full instructions (SKILL.md)
Source of truth, from microsoft/azure-skills.
name: azure-aigateway description: "Configure Azure API Management as an AI Gateway for AI models, MCP tools, and agents. WHEN: semantic caching, token limit, content safety, load balancing, AI model governance, MCP rate limiting, jailbreak detection, add Azure OpenAI backend, add AI Foundry model, test AI gateway, LLM policies, configure AI backend, token metrics, AI cost control, convert API to MCP, import OpenAPI to gateway." license: MIT metadata: author: Microsoft version: "3.2.1" compatibility: Requires Azure CLI (az) for configuration and testing
Azure AI Gateway
Configure Azure API Management (APIM) as an AI Gateway for governing AI models, MCP tools, and agents.
To deploy APIM, use the azure-prepare skill. See APIM deployment guide.
When to Use This Skill
| Category | Triggers |
|---|---|
| Model Governance | "semantic caching", "token limits", "load balance AI", "track token usage" |
| Tool Governance | "rate limit MCP", "protect my tools", "configure my tool", "convert API to MCP" |
| Agent Governance | "content safety", "jailbreak detection", "filter harmful content" |
| Configuration | "add Azure OpenAI backend", "configure my model", "add AI Foundry model" |
| Testing | "test AI gateway", "call OpenAI through gateway" |
Quick Reference
| Policy | Purpose | Details |
|---|---|---|
azure-openai-token-limit | Cost control | Model Policies |
azure-openai-semantic-cache-lookup/store | 60-80% cost savings | Model Policies |
azure-openai-emit-token-metric | Observability | Model Policies |
llm-content-safety | Safety & compliance | Agent Policies |
rate-limit-by-key | MCP/tool protection | Tool Policies |
Get Gateway Details
# Get gateway URL
az apim show --name <apim-name> --resource-group <rg> --query "gatewayUrl" -o tsv
# List backends (AI models)
az apim backend list --service-name <apim-name> --resource-group <rg> \
--query "[].{id:name, url:url}" -o table
# Get subscription key
az apim subscription keys list \
--service-name <apim-name> --resource-group <rg> --subscription-id <sub-id>
Test AI Endpoint
GATEWAY_URL=$(az apim show --name <apim-name> --resource-group <rg> --query "gatewayUrl" -o tsv)
curl -X POST "${GATEWAY_URL}/openai/deployments/<deployment>/chat/completions?api-version=2024-02-01" \
-H "Content-Type: application/json" \
-H "Ocp-Apim-Subscription-Key: <key>" \
-d '{"messages": [{"role": "user", "content": "Hello"}], "max_tokens": 100}'
Common Tasks
Add AI Backend
See references/patterns.md for full steps.
# Discover AI resources
az cognitiveservices account list --query "[?kind=='OpenAI']" -o table
# Create backend
az apim backend create --service-name <apim> --resource-group <rg> \
--backend-id openai-backend --protocol http --url "https://<aoai>.openai.azure.com/openai"
# Grant access (managed identity)
az role assignment create --assignee <apim-principal-id> \
--role "Cognitive Services User" --scope <aoai-resource-id>
Apply AI Governance Policy
Recommended policy order in <inbound>:
- Authentication - Managed identity to backend
- Semantic Cache Lookup - Check cache before calling AI
- Token Limits - Cost control
- Content Safety - Filter harmful content
- Backend Selection - Load balancing
- Metrics - Token usage tracking
See references/policies.md for complete example.
Troubleshooting
| Issue | Solution |
|---|---|
| Token limit 429 | Increase tokens-per-minute or add load balancing |
| No cache hits | Lower score-threshold to 0.7 |
| Content false positives | Increase category thresholds (5-6) |
| Backend auth 401 | Grant APIM "Cognitive Services User" role |
See references/troubleshooting.md for details.
References
- Detailed Policies - Full policy examples
- Configuration Patterns - Step-by-step patterns
- Troubleshooting - Common issues
- AI-Gateway Samples
- GenAI Gateway Docs
SDK Quick References
- Content Safety: Python | TypeScript
- API Management: Python | .NET
Related skills
More from microsoft/azure-skills and the wider catalog.
