PluginBench
Skill
Official
Review
Audit score 70

azure-aigateway

microsoft/azure-skills

Configure Azure API Management as an AI Gateway for model governance, cost control, and safety policies.

SourceView on skills.sh

What is azure-aigateway?

This skill sets up Azure API Management (APIM) as a gateway to govern AI models, MCP tools, and agents. Use it when you need semantic caching, token limits, content safety filtering, load balancing, jailbreak detection, or cost tracking for AI workloads.

  • Apply semantic caching to reduce AI costs by 60-80%
  • Enforce token limits and rate limiting for cost control and MCP tool protection
  • Implement content safety and jailbreak detection policies
  • Add Azure OpenAI and AI Foundry model backends with managed identity authentication
  • Track token usage metrics and load balance across AI models
  • Convert APIs to MCP and import OpenAPI specifications into the gateway

How to install azure-aigateway

npx skills add https://github.com/microsoft/azure-skills --skill azure-aigateway
Prerequisites
  • Azure subscription with API Management service deployed (use azure-prepare skill)
  • Azure CLI (az) installed and authenticated
  • Access to Azure OpenAI or AI Foundry resources to add as backends
  • Appropriate RBAC roles to create backends and assign permissions
Claude Code
Cursor
Windsurf
Cline

How to use azure-aigateway

  1. 1.Deploy or access an existing Azure API Management instance
  2. 2.Discover and add AI backends (Azure OpenAI, AI Foundry) using az apim backend create
  3. 3.Configure managed identity authentication between APIM and your AI resources
  4. 4.Apply governance policies in order: authentication, semantic cache lookup, token limits, content safety, backend selection, metrics
  5. 5.Test the gateway endpoint using curl or your client with the subscription key
  6. 6.Monitor token usage and adjust policy thresholds based on cost and performance metrics

Use cases

Good for
  • Protect expensive AI model endpoints with token limits and semantic caching to reduce costs
  • Govern multi-tenant access to AI models with subscription keys and rate limiting
  • Filter harmful content and detect jailbreak attempts before requests reach the model
  • Load balance requests across multiple Azure OpenAI deployments or AI Foundry models
  • Monitor token consumption and enforce usage quotas per subscription or API consumer
Who it's for
  • Platform engineers managing shared AI infrastructure
  • DevOps teams implementing AI governance and cost controls
  • Security teams enforcing content safety and compliance policies
  • AI application developers needing multi-model load balancing and caching
  • Teams converting internal APIs to MCP tool specifications

azure-aigateway FAQ

What policies should I apply first?

Apply in this order: authentication (managed identity), semantic cache lookup, token limits, content safety, backend selection, and metrics tracking. See references/policies.md for the complete example.

How much can semantic caching save?

Semantic caching typically saves 60-80% on AI costs by caching semantically similar requests and reusing responses without calling the model.

How do I add an Azure OpenAI backend?

Use az apim backend create with your Azure OpenAI endpoint URL, then grant APIM's managed identity the 'Cognitive Services User' role on the resource.

What should I do if I'm hitting token limit 429 errors?

Increase the tokens-per-minute threshold in the token-limit policy, or add load balancing across multiple backends to distribute the load.

Can I use this with MCP tools?

Yes. You can rate-limit MCP tool access using rate-limit-by-key policy, and convert APIs to MCP specifications for exposure through the gateway.

Full instructions (SKILL.md)

Source of truth, from microsoft/azure-skills.


name: azure-aigateway description: "Configure Azure API Management as an AI Gateway for AI models, MCP tools, and agents. WHEN: semantic caching, token limit, content safety, load balancing, AI model governance, MCP rate limiting, jailbreak detection, add Azure OpenAI backend, add AI Foundry model, test AI gateway, LLM policies, configure AI backend, token metrics, AI cost control, convert API to MCP, import OpenAPI to gateway." license: MIT metadata: author: Microsoft version: "3.2.1" compatibility: Requires Azure CLI (az) for configuration and testing

Azure AI Gateway

Configure Azure API Management (APIM) as an AI Gateway for governing AI models, MCP tools, and agents.

To deploy APIM, use the azure-prepare skill. See APIM deployment guide.

When to Use This Skill

CategoryTriggers
Model Governance"semantic caching", "token limits", "load balance AI", "track token usage"
Tool Governance"rate limit MCP", "protect my tools", "configure my tool", "convert API to MCP"
Agent Governance"content safety", "jailbreak detection", "filter harmful content"
Configuration"add Azure OpenAI backend", "configure my model", "add AI Foundry model"
Testing"test AI gateway", "call OpenAI through gateway"

Quick Reference

PolicyPurposeDetails
azure-openai-token-limitCost controlModel Policies
azure-openai-semantic-cache-lookup/store60-80% cost savingsModel Policies
azure-openai-emit-token-metricObservabilityModel Policies
llm-content-safetySafety & complianceAgent Policies
rate-limit-by-keyMCP/tool protectionTool Policies

Get Gateway Details

# Get gateway URL
az apim show --name <apim-name> --resource-group <rg> --query "gatewayUrl" -o tsv

# List backends (AI models)
az apim backend list --service-name <apim-name> --resource-group <rg> \
  --query "[].{id:name, url:url}" -o table

# Get subscription key
az apim subscription keys list \
  --service-name <apim-name> --resource-group <rg> --subscription-id <sub-id>

Test AI Endpoint

GATEWAY_URL=$(az apim show --name <apim-name> --resource-group <rg> --query "gatewayUrl" -o tsv)

curl -X POST "${GATEWAY_URL}/openai/deployments/<deployment>/chat/completions?api-version=2024-02-01" \
  -H "Content-Type: application/json" \
  -H "Ocp-Apim-Subscription-Key: <key>" \
  -d '{"messages": [{"role": "user", "content": "Hello"}], "max_tokens": 100}'

Common Tasks

Add AI Backend

See references/patterns.md for full steps.

# Discover AI resources
az cognitiveservices account list --query "[?kind=='OpenAI']" -o table

# Create backend
az apim backend create --service-name <apim> --resource-group <rg> \
  --backend-id openai-backend --protocol http --url "https://<aoai>.openai.azure.com/openai"

# Grant access (managed identity)
az role assignment create --assignee <apim-principal-id> \
  --role "Cognitive Services User" --scope <aoai-resource-id>

Apply AI Governance Policy

Recommended policy order in <inbound>:

  1. Authentication - Managed identity to backend
  2. Semantic Cache Lookup - Check cache before calling AI
  3. Token Limits - Cost control
  4. Content Safety - Filter harmful content
  5. Backend Selection - Load balancing
  6. Metrics - Token usage tracking

See references/policies.md for complete example.


Troubleshooting

IssueSolution
Token limit 429Increase tokens-per-minute or add load balancing
No cache hitsLower score-threshold to 0.7
Content false positivesIncrease category thresholds (5-6)
Backend auth 401Grant APIM "Cognitive Services User" role

See references/troubleshooting.md for details.


References

SDK Quick References