self-review
lingzhi227/agent-research-skills
Automatically review academic papers using NeurIPS form with three reviewer personas and ensemble scoring.
What is self-review?
Self-review generates structured peer reviews of academic papers by extracting text from PDF or LaTeX, running three independent reviews with different personas, and synthesizing actionable feedback. Use this when you need to evaluate a draft paper before submission or get detailed reviewer-style critique.
- Extracts text from PDF or LaTeX files using multiple fallback methods
- Generates three independent reviews using different reviewer personas (harsh-but-fair, harsh-critical, open-minded)
- Applies reflection refinement up to 3 rounds per reviewer to improve accuracy
- Aggregates scores with weighted averaging based on NeurIPS scoring dimensions
- Produces structured report with strengths, weaknesses, questions, and specific improvement suggestions
- Verifies presence of required paper sections (Abstract, Introduction, Methods, Experiments, Discussion)
How to install self-review
npx skills add https://github.com/lingzhi227/agent-research-skills --skill self-review- Python with pymupdf4llm, pymupdf, or pypdf installed (pip install pymupdf4llm pymupdf pypdf)
- PDF or LaTeX source file of the paper to review
- Access to references/review-form.md in the skill installation
How to use self-review
- 1.Provide the path to your PDF or .tex file as the argument
- 2.The skill extracts paper text using the best available PDF parser
- 3.Three independent reviews are generated using different reviewer personas
- 4.Each review undergoes reflection refinement to improve soundness
- 5.All three reviews are aggregated with weighted scoring
- 6.A final report is output with consensus strengths, weaknesses, questions, and specific suggestions
Use cases
- Get detailed peer-style feedback on a research paper before conference submission
- Identify major weaknesses and missing sections in a draft manuscript
- Generate consensus review scores across multiple reviewer perspectives
- Prepare for potential reviewer feedback by seeing common critique patterns
- Refine paper structure and arguments based on actionable reviewer suggestions
- Academic researchers preparing papers for peer-reviewed venues
- PhD students seeking feedback on dissertation chapters
- Conference paper authors revising before resubmission
- Research teams conducting internal paper quality checks
self-review FAQ
PDF files and LaTeX (.tex) files. For PDFs, the skill tries pymupdf4llm first (best quality), then falls back to pymupdf or pypdf.
The harsh-but-fair reviewer expects good experiments leading to insights; the harsh-critical reviewer looks for impactful ideas in the field; the open-minded reviewer seeks novel ideas not previously proposed.
The skill checks for Abstract, Introduction, Methods/Approach, Experiments/Results, and Discussion/Conclusion. Scores are reduced if any required sections are missing.
Numerical scores from all three reviewers are averaged and rounded to the nearest integer. Weighted averaging is applied using AgentLaboratory weights: Overall (1.0), Contribution (0.4), Presentation (0.2), and others (0.1 each).
The skill uses the NeurIPS review form and scoring structure, so it works best for papers following standard academic conference formats. It may be less effective for papers with significantly different structures.
Full instructions (SKILL.md)
Source of truth, from lingzhi227/agent-research-skills.
name: self-review description: Automatically review an academic paper using the NeurIPS review form with three reviewer personas, ensemble scoring, and reflection refinement. Extracts text from PDF, runs structured review, and outputs actionable feedback. Use when the user wants to review a paper before submission or get feedback on a draft. argument-hint: [pdf-or-tex-file]
Self-Review
Review an academic paper using a structured review form with multiple reviewer personas.
Input
$ARGUMENTS— Path to PDF file or.texfile
Scripts
Extract text from PDF
python ~/.claude/skills/self-review/scripts/extract_pdf_text.py paper.pdf --output paper_text.txt
python ~/.claude/skills/self-review/scripts/extract_pdf_text.py paper.pdf --format markdown
Tries pymupdf4llm (best) → pymupdf → pypdf. Install: pip install pymupdf4llm pymupdf pypdf
Parse PDF into structured sections
python ~/.claude/skills/self-review/scripts/parse_pdf_sections.py \
--pdf paper.pdf --output sections.json
Extracts title (via font size), section headings, and section text. Requires: pip install pymupdf
Key flags: --format text, --verbose
Workflow
Step 1: Load Paper
- If PDF: use
extract_pdf_text.pyto extract text - If
.tex: read the LaTeX source directly
Step 2: Three-Persona Review
Run three independent reviews using different personas (from references/review-form.md):
- Harsh but fair reviewer: Expects good experiments that lead to insights
- Harsh and critical reviewer: Looking for impactful ideas in the field
- Open-minded reviewer: Looking for novel ideas not proposed before
For each persona, generate a review following the NeurIPS review JSON format in references/review-form.md.
Step 3: Reflection Refinement (up to 3 rounds per reviewer)
After each review, apply the reflection prompt: re-evaluate accuracy and soundness, refine if needed. Stop when "I am done".
Step 4: Aggregate
- Combine all three reviews
- Average numerical scores (round to nearest integer)
- Synthesize a meta-review finding consensus
- Weight scores using AgentLaboratory weights: Overall (1.0), Contribution (0.4), Presentation (0.2), others (0.1 each)
Step 5: Actionable Report
Output format:
## Review Summary
- **Overall Score**: X/10 (Weighted: Y/10)
- **Decision**: Accept / Reject
- **Confidence**: Z/5
## Strengths (consensus across reviewers)
1. ...
2. ...
## Weaknesses (consensus across reviewers)
1. ...
2. ...
## Questions for Authors
1. ...
## Specific Suggestions for Improvement
1. [Section X, Page Y]: ...
2. [Section Z, Page W]: ...
## Score Breakdown
| Dimension | R1 | R2 | R3 | Avg |
|-----------|----|----|-----|-----|
| Overall | ... | ... | ... | ... |
| Contribution | ... | ... | ... | ... |
| ... | ... | ... | ... | ... |
References
- NeurIPS review form, scoring weights, personas, reflection prompts:
~/.claude/skills/self-review/references/review-form.md - PDF text extraction:
~/.claude/skills/self-review/scripts/extract_pdf_text.py
Missing Sections Check
You MUST verify that all required sections are present: Abstract, Introduction, Methods/Approach, Experiments/Results, Discussion/Conclusion. Reduce scores if any are missing.
Related Skills
- Upstream: paper-compilation
- Downstream: paper-revision, rebuttal-writing
- See also: slide-generation
Related skills
More from lingzhi227/agent-research-skills and the wider catalog.
slide-generation
Convert research papers to presentation slides or posters with automated content extraction and Beamer LaTeX generation.
survey-generation
Generate comprehensive academic survey papers with structured outlines, RAG-based writing, and validated citations.
symbolic-equation
Discover interpretable scientific equations from data using LLM-guided evolutionary search.
table-generation
Generate publication-quality LaTeX tables from experimental results with booktabs styling and bold best results.
linkfox-multimodal-recognize-image
基于多模态AI的图片识别与分析。当用户想分析、描述、从图片URL中提取信息、image recognition, image analysis, image description, image content understanding, OCR text recognition, visual Q&A时触发此技能。当用户提到图片识别、图片分析、图片描述、识别图片内容、分析产品图、从图片中读取文字、描述图片、提取视觉内容或理解照片内容时触发。当用户提供图片URL并就其视觉内容提问时，即使未明确说"图片识别"，也应触发此技能。
anti-render
Generate ideal-vs-reality visual comparisons by analyzing images and creating side-by-side contrasts.