Skill

Fail

Audit score 45

experiment-code

lingzhi227/agent-research-skills

Generate and iteratively improve ML experiment code for research papers with training pipelines, debugging, and result optimization.

Source View on skills.sh

What is experiment-code?

Write self-contained ML experiment code with proper logging, visualization, and iterative refinement. Use this skill when implementing experiments for a research paper—it handles code generation, error debugging, result analysis, and publication-quality plotting.

Generate initial experiment code with mandatory structure (experiment.py, plot.py, results logging)
Improve existing code by reflecting on results and applying targeted edits with re-runs
Debug experiment errors with minimal fixes and up to 4 retry attempts
Generate publication-quality plots from experiment results across multiple runs
Plan experiments first (hyperparameter sweeps, ablations, baselines) before writing code
Enforce reproducibility with seeds and proper documentation in notes.txt

How to install experiment-code

npx skills add https://github.com/lingzhi227/agent-research-skills --skill experiment-code

Prerequisites

PyTorch or scikit-learn installed (TensorFlow/Keras not preferred)
Access to actual datasets (not toy data)
Project directory structure for organizing runs and results

Claude Code

Cursor

Windsurf

Cline

How to use experiment-code

1.Specify the task (generate, improve, debug, or plot) and provide research plan or error message
2.For generate: code will create experiment.py, plot.py, notes.txt, and run_N/ directories with results
3.For improve: provide current code and results; skill will reflect on performance and apply targeted edits
4.For debug: paste error message; skill will identify root cause and apply minimal fix with retry attempts
5.For plot: skill reads all run_*/final_info.json files and generates comparison figures

Use cases

Good for

Implementing experiments described in a research paper with multiple runs and hyperparameter sweeps
Debugging training pipeline failures and fixing accuracy calculation errors
Comparing baseline vs. ablation results and generating comparison plots
Iteratively improving model performance by analyzing what worked in previous runs
Creating reproducible experiment code with proper logging and result tracking

Who it's for

ML researchers implementing experiments for papers
PhD students running ablation studies and baseline comparisons
Data scientists debugging training pipelines
Anyone needing reproducible, documented ML experiment code

experiment-code FAQ

What datasets should I use?

Use actual datasets, not toy data, unless explicitly requested. The skill enforces this to ensure realistic results.

Can I use TensorFlow or Keras?

PyTorch or scikit-learn are preferred. TensorFlow/Keras are not recommended for this skill.

How many retry attempts does the debug action allow?

Up to 4 retry attempts before changing approach. If errors persist, the skill will suggest a different strategy.

What output structure is required?

Mandatory structure includes experiment.py, plot.py, notes.txt, and run_N/ directories with final_info.json files for each run.

How do I ensure reproducibility?

Use seeds for reproducibility and document all experiments in notes.txt with print statements explaining what results show.

Full instructions (SKILL.md)

Source of truth, from lingzhi227/agent-research-skills.

name: experiment-code description: Write ML experiment code with iterative improvement. Generate training/evaluation pipelines, debug errors, and optimize results through code reflection. Use when implementing experiments for a research paper. argument-hint: [plan-or-idea]

Experiment Code

Generate and iteratively improve ML experiment code for research papers.

Input

$0 — Task: generate, improve, debug, plot
$1 — Research plan, idea description, or error message

References

Experiment prompts and patterns: ~/.claude/skills/experiment-code/references/experiment-prompts.md
Code patterns (error handling, repair, hill-climbing): ~/.claude/skills/experiment-code/references/code-patterns.md

Action: `generate`

Generate initial experiment code following this structure:

Plan experiments first — List all runs needed (hyperparameter sweeps, ablations, baselines)
Write self-contained code — All code in project directory, no external imports from reference repos
Include proper logging — Save results to JSON, print intermediate metrics
Generate figures — At minimum Figure_1.png and Figure_2.png

Mandatory Structure

project/
├── experiment.py      # Main experiment script
├── plot.py            # Visualization script
├── notes.txt          # Experiment descriptions and results
├── run_1/             # Results from run 1
│   └── final_info.json
├── run_2/
└── ...

Constraints

No placeholder code (pass, ..., raise NotImplementedError)
Must use actual datasets (not toy data unless explicitly requested)
PyTorch or scikit-learn preferred (no TensorFlow/Keras)
Each run uses: python experiment.py --out_dir=run_i

Action: `improve`

Improve existing experiment code:

Read current code and results
Reflect on what worked and what didn't
Apply targeted edits (prefer small edits over full rewrites)
Re-run and compare scores
Keep the best-performing code variant

Action: `debug`

Fix experiment code errors:

Read the error message (truncate to last 1500 chars if very long)
Identify the root cause
Apply minimal fix
Up to 4 retry attempts before changing approach

Action: `plot`

Generate publication-quality plots from experiment results:

Read all run_*/final_info.json files
Generate comparison plots with proper labels
Use the figure-generation skill for styling

Rules

Always plan experiments before writing code
After each run, document results in notes.txt
Include print statements explaining what results show
Method MUST not get 0% accuracy — verify accuracy calculations
Use seeds for reproducibility
Before each experiment include a print statement explaining exactly what the results are meant to show