PluginBench
Skill
Review
Audit score 70

idea-generation

lingzhi227/agent-research-skills

Generate and refine novel research ideas with literature-backed novelty scoring.

What is idea-generation?

This skill helps researchers brainstorm and validate research ideas through iterative refinement and automated novelty checking against academic literature. Use it when exploring new research directions, validating idea originality, or scoring ideas on Interestingness, Feasibility, and Novelty dimensions.

  • Generate 3-5 diverse research ideas from a given research area or codebase context
  • Iteratively refine ideas through critical evaluation over up to 5 rounds
  • Perform automated novelty assessment by searching Semantic Scholar and arXiv
  • Score ideas on three dimensions: Interestingness (1-10), Feasibility (1-10), and Novelty (1-10)
  • Identify and rank the most similar existing papers for each idea
  • Output structured JSON with idea details, ratings, and novelty verdict

How to install idea-generation

npx skills add https://github.com/lingzhi227/agent-research-skills --skill idea-generation
Prerequisites
  • Python environment with access to Semantic Scholar and arXiv APIs
  • Research area or task description to seed idea generation
  • Optional: existing codebase or paper context for more targeted ideas
Claude Code
Cursor
Windsurf
Cline

How to use idea-generation

  1. 1.Provide a research area or task description as input (e.g., 'attention mechanism optimization')
  2. 2.Optionally add constraints or context (e.g., 'for NeurIPS', resource limitations)
  3. 3.Review the generated 3-5 ideas with their initial ratings and experiment plans
  4. 4.Select ideas for iterative refinement and run up to 5 rounds of critical evaluation
  5. 5.Run novelty_check.py on promising ideas to assess literature novelty
  6. 6.Score final ideas on Interestingness, Feasibility, and Novelty (1-10 each)
  7. 7.Review the ranked output and select top idea(s) for development

Use cases

Good for
  • Brainstorming novel research directions within a specific ML or AI subfield
  • Validating whether a research idea is sufficiently novel for conference submission
  • Refining rough research hypotheses into concrete, feasible experiments
  • Assessing the feasibility and interestingness of ideas before committing resources
  • Comparing multiple research directions to select the most promising one
Who it's for
  • Academic researchers and PhD students
  • Machine learning and AI practitioners planning new projects
  • Research teams evaluating multiple project proposals
  • Conference paper authors validating idea novelty before submission

idea-generation FAQ

How does the novelty check work?

The skill performs iterative literature search using Semantic Scholar and arXiv. It generates multiple search queries, reviews results, and makes a binary novel/not-novel decision with justification based on the most similar papers found.

What makes an idea 'feasible' in this context?

Feasible ideas must be implementable with available resources (standard compute, existing datasets) and should not require massive new datasets or specialized infrastructure. The skill rates feasibility 1-10 based on resource requirements and implementation complexity.

Can I refine ideas multiple times?

Yes, the skill supports iterative refinement for up to 5 rounds per idea. Each round critically evaluates quality, novelty, and feasibility while preserving the core idea spirit. You can stop early if the idea converges.

What output do I get from this skill?

For each idea, you receive a JSON object with: Name, Title, detailed Experiment plan, three numerical ratings (Interestingness, Feasibility, Novelty), a novelty verdict (true/false), and a list of the most similar papers from literature.

How should I interpret the three scoring dimensions?

Interestingness measures research impact and significance; Feasibility assesses whether you can realistically implement it; Novelty rates how original the contribution is relative to existing work. Ratings are 1-10 and should be realistic rather than optimistic.

Full instructions (SKILL.md)

Source of truth, from lingzhi227/agent-research-skills.


name: idea-generation description: Generate novel research ideas with iterative refinement and novelty checking against literature. Score ideas on Interestingness, Feasibility, and Novelty. Use when brainstorming research directions or validating idea novelty. argument-hint: [research-area]

Idea Generation

Generate and refine novel research ideas with literature-backed novelty assessment.

Input

  • $0 — Research area, task description, or existing codebase context
  • $1 — Optional: additional context (e.g., "for NeurIPS", constraints)

Scripts

Novelty check against Semantic Scholar

python ~/.claude/skills/idea-generation/scripts/novelty_check.py \
  --idea "Adaptive attention head pruning via gradient-guided importance" \
  --max-rounds 5

Performs iterative literature search to assess if an idea is novel.

References

  • Ideation prompts (generation, reflection, novelty): ~/.claude/skills/idea-generation/references/ideation-prompts.md

Workflow

Step 1: Generate Ideas

Given a research area and optional code/paper context:

  1. Generate 3-5 diverse research ideas
  2. For each idea, provide: Name, Title, Experiment plan, and ratings
  3. Use the ideation prompt templates from references

Step 2: Iterative Refinement (up to 5 rounds per idea)

For each idea:

  1. Critically evaluate quality, novelty, and feasibility
  2. Refine the idea while preserving its core spirit
  3. Stop when converged ("I am done") or max rounds reached

Step 3: Novelty Assessment

For each promising idea:

  1. Run novelty_check.py or manually search Semantic Scholar / arXiv
  2. Use the novelty checking prompts from references
  3. Multi-round search: generate queries, review results, decide
  4. Binary decision: Novel / Not Novel with justification

Step 4: Rank and Select

  • Score each idea on three dimensions (1-10): Interestingness, Feasibility, Novelty
  • Be cautious and realistic on ratings
  • Select the top idea(s) for development

Output Format

{
  "Name": "adaptive_attention_pruning",
  "Title": "Adaptive Attention Head Pruning via Gradient-Guided Importance Scoring",
  "Experiment": "Detailed implementation plan...",
  "Interestingness": 8,
  "Feasibility": 7,
  "Novelty": 9,
  "novel": true,
  "most_similar_papers": ["paper1", "paper2"]
}

Rules

  • Ideas must be feasible with available resources (no requiring new datasets or massive compute)
  • Do not overfit ideas to a specific dataset or model — aim for wider significance
  • Be a harsh critic for novelty — ensure sufficient contribution for a conference paper
  • Each idea should stem from a simple, elegant question or hypothesis
  • Always check novelty before committing to an idea

Related Skills

  • Upstream: literature-search, deep-research
  • Downstream: research-planning, experiment-design
  • See also: novelty-assessment