PluginBench
Skill
Official
Review
Audit score 70

finetuning

microsoft/azure-skills

Fine-tune models on Microsoft Foundry using SFT, DPO, or RFT with dataset prep, training, and evaluation.

SourceView on skills.sh

What is finetuning?

Fine-tune language and vision models on Microsoft Foundry using supervised (SFT), preference (DPO), or reinforcement (RFT) training methods. Use this skill when preparing training data, submitting and monitoring training jobs, calibrating graders, deploying fine-tuned models, or evaluating results.

  • Submit and monitor SFT, DPO, and RFT training jobs on Microsoft Foundry
  • Prepare, validate, and convert training datasets between formats
  • Calibrate graders and pass thresholds for reinforcement fine-tuning (RFT)
  • Deploy fine-tuned models via ARM REST API and evaluate against test sets
  • Generate synthetic training data via distillation and score dataset quality
  • Analyze training curves, checkpoints, and diagnose poor training results

How to install finetuning

npx skills add https://github.com/microsoft/azure-skills --skill microsoft-foundry
Prerequisites
  • Microsoft Foundry account with API access
  • OpenAI SDK version 1.0 or later
  • Training data in JSONL format (SFT, DPO, or RFT schema)
  • For RFT: a grader function or endpoint to evaluate model outputs
Claude Code
Cursor
Windsurf
Cline

How to use finetuning

  1. 1.Prepare training data in the appropriate format (SFT, DPO, or RFT) and validate using scripts/validate/
  2. 2.Choose training type (SFT for supervised, DPO for preferences, RFT for reinforcement) based on your use case
  3. 3.Submit a training job using scripts/submit_training.py with your model, training file, and hyperparameters
  4. 4.Monitor job progress with scripts/monitor_training.py and analyze training curves with scripts/check_training.py
  5. 5.Evaluate candidate checkpoints using scripts/evaluate_model.py before deploying the final model
  6. 6.Deploy the selected fine-tuned model using scripts/deploy_model.py and monitor for deployment readiness

Use cases

Good for
  • Fine-tune GPT-4 mini on domain-specific instruction data using SFT to improve task accuracy
  • Train a preference model using DPO on human-ranked response pairs to align outputs with user preferences
  • Set up RFT with a custom grader to optimize model behavior on structured tasks like code generation or reasoning
  • Validate and clean training data before submission to catch format errors and quality issues early
  • Deploy a fine-tuned checkpoint and evaluate it against a held-out test set using LLM-based judging
Who it's for
  • ML engineers and researchers training custom models on proprietary data
  • Product teams optimizing model behavior for specific use cases without retraining from scratch
  • Data scientists preparing and validating training datasets at scale
  • Teams using reinforcement fine-tuning with custom graders or tool-use evaluation

finetuning FAQ

What is the difference between SFT, DPO, and RFT?

SFT (supervised fine-tuning) trains on input-output pairs; DPO (direct preference optimization) trains on ranked response pairs to align with preferences; RFT (reinforcement fine-tuning) uses a grader function to optimize for specific criteria. Choose SFT for basic task adaptation, DPO for preference alignment, and RFT for complex evaluation criteria.

How do I validate my training data before submitting a job?

Run scripts/validate/validate_sft.py (or validate_dpo.py, validate_rft.py) on your JSONL file to check format, token counts, and data statistics. Fix any validation errors before submission.

What should my RFT grader pass rate be?

Target a 25–50% failure rate on the base model. Use scripts/calibrate_grader.py to find the optimal pass_threshold for your grader function.

Why is my training job stuck in post-training evaluation?

For RFT, the tool endpoint may be under-provisioned. Scale to S2+ and enable Always On. For other issues, check platform-gotchas.md or delete and recreate the deployment after waiting 5 minutes.

Can I fine-tune vision models?

Yes. Use the same SFT/DPO/RFT workflows with vision-compatible data formats. See references/vision-fine-tuning.md for format details and examples.

Full instructions (SKILL.md)

Source of truth, from microsoft/azure-skills.


name: finetuning description: "Fine-tune models on Microsoft Foundry using SFT (supervised), DPO (preference), or RFT (reinforcement with graders). Covers dataset preparation, training job submission, deployment, and evaluation. USE FOR: fine-tune, SFT, DPO, RFT, training data, grader, distillation, fine-tuned model, training job, large file upload, calibrate grader, deploy fine-tuned model, evaluate fine-tuned model. DO NOT USE FOR: general model deployment without fine-tuning (use deploy-model), agent creation (use agents), prompt optimization without training (use prompt-optimizer)." license: MIT metadata: author: Microsoft version: "0.0.0-placeholder"

Fine-Tuning on Microsoft Foundry

Fine-tune models using SFT (supervised), DPO (preference), or RFT (reinforcement with graders). Covers dataset prep, training, deployment, and evaluation.

When to Use

Use this sub-skill when the user asks about:

  • Fine-tuning a model (SFT, DPO, or RFT)
  • Preparing, validating, or formatting training data
  • Submitting, monitoring, or diagnosing training jobs
  • Calibrating graders or pass thresholds for RFT
  • Deploying or evaluating a fine-tuned model
  • Choosing between training types (SFT vs DPO vs RFT)
  • Distillation, synthetic data generation, or dataset quality scoring
  • Large file uploads for training data
  • Cleaning up fine-tuning resources (files, deployments)

Do NOT use for: General model deployment without fine-tuning (use deploy-model), agent creation (use agents), prompt optimization without training (use prompt-optimizer).

Workflows

StageGuide
Quick startworkflows/quickstart.md
Full pipelineworkflows/full-pipeline.md
Create dataworkflows/dataset-creation.md
Iterateworkflows/iterative-training.md
Diagnoseworkflows/diagnose-poor-results.md

References

TopicFile
SFT vs DPO vs RFTreferences/training-types.md
Hyperparametersreferences/hyperparameters.md
Data formatsreferences/dataset-formats.md
Grader design (RFT)references/grader-design.md
Reward hackingreferences/reward-hacking.md
Agentic RFT (tools)references/agentic-rft.md
Deploymentreferences/deployment.md
Training curvesreferences/training-curves.md
Evaluationreferences/evaluation.md
Vision fine-tuningreferences/vision-fine-tuning.md
Large file uploadsreferences/large-file-uploads.md
Platform gotchasreferences/platform-gotchas.md

Scripts

ScriptPurpose
scripts/submit_training.pySubmit SFT/DPO/RFT jobs
scripts/monitor_training.pyPoll job until completion
scripts/calibrate_grader.pyFind optimal RFT pass_threshold
scripts/check_training.pyAnalyze curves, list checkpoints
scripts/deploy_model.pyDeploy via ARM REST API
scripts/evaluate_model.pyLLM judge evaluation
scripts/convert_dataset.pyConvert between SFT/DPO/RFT formats
scripts/generate_distillation_data.pyGenerate synthetic training data
scripts/score_dataset.pyQuality scoring on training data
scripts/cleanup.pyDelete old files and deployments
scripts/validate/Data validators (SFT, DPO, RFT) + stats

Rules

  1. Always baseline first — evaluate the base model before fine-tuning
  2. Validate data before submitting — run scripts/validate/validate_sft.py
  3. Calibrate RFT graders — target 25-50% failure rate on the base model
  4. Evaluate checkpoints — don't blindly deploy the final one
  5. Measure token cost alongside accuracy when comparing models

Quick Reference

TaskCommand
Validate SFT datapython scripts/validate/validate_sft.py data.jsonl
Submit SFT jobpython scripts/submit_training.py --model gpt-4.1-mini --training-file train.jsonl --validation-file val.jsonl --type sft
Monitor jobpython scripts/monitor_training.py --job-id ftjob-xxx
Analyze curvespython scripts/check_training.py --job-id ftjob-xxx
Deploy modelpython scripts/deploy_model.py --model-id ft:gpt-4.1-mini:... --name my-eval
Evaluate modelpython scripts/evaluate_model.py --deployment-name my-eval --test-file test.jsonl

Error Handling

ErrorCauseFix
"API version not supported"Older openai SDK on /v1/ endpointUpgrade to openai>=1.0
"does not support fine-tuning with Standard TrainingType"OSS model needs globalStandardUse --use-rest flag or script auto-falls back
Job stuck in post-training evalUnder-provisioned tool endpoint (RFT)Scale to S2+, enable Always On
"DeploymentNotReady" after ARM succeedsARM/data-plane race conditionDelete and recreate deployment, wait 5 min
Content safety block at deploymentPII-dense training dataRemove problematic document types