text-optimizer
kochetkov-ma/claude-brewcode
Reduces token count in prompts and docs by 20–40% using 52 research-backed optimization rules.
What is text-optimizer?
Text Optimizer applies 52 research-backed rules across 8 categories (Claude behavior, token efficiency, structure, deduplication, reference integrity, perception, LLM comprehension, and aggressive lossy) to compress prompts, documentation, and agent instructions without losing meaning. Use it when you need to reduce API costs, speed up model responses, clarify LLM instructions, or minimize hallucinations.
- Applies 52 research-backed optimization rules across 8 categories
- Reduces token count by 20–40% while preserving semantic meaning
- Offers three modes: light (text cleanup only), medium (balanced restructuring), and deep (maximum compression with aggressive lossy rules)
- Deduplicates exact, near, and cross-format duplicates automatically
- Verifies file paths, URLs, and reference integrity
- Restructures content for better LLM comprehension and perception
How to install text-optimizer
npx skills add https://github.com/kochetkov-ma/claude-brewcode --skill text-optimizer- File or folder path to optimize (single .md file or directory of .md files)
- Access to references/rules-review.md (required before optimization begins)
How to use text-optimizer
- 1.Run the skill with a file or folder path: `/text-optimize prompt.md` for default medium mode
- 2.Choose a mode: use `-l` flag for light mode (text cleanup only), `-d` flag for deep mode (maximum compression), or no flag for medium mode (balanced)
- 3.The skill reads the rules reference, analyzes your file, deduplicates content, applies rules by mode, and self-verifies for semantic preservation
- 4.Review the optimization report generated after processing
- 5.For directories, use `-d agents/` to process all .md files recursively
Use cases
- Compress verbose system prompts to reduce API costs and latency
- Optimize agent instruction sets for clarity and token efficiency
- Reduce documentation size while maintaining accuracy and completeness
- Prepare training or reference materials for LLM consumption
- Merge and deduplicate multi-file documentation into single sources of truth
- Prompt engineers optimizing system prompts and instructions
- AI agent developers reducing token consumption in agent configs
- Technical writers compressing documentation for LLM ingestion
- Teams managing large prompt libraries or knowledge bases
- Anyone building cost-sensitive LLM applications
text-optimizer FAQ
Light mode performs text cleanup only (wording, structure, lists) with zero semantic loss. Medium mode applies balanced restructuring across all standard rules with zero loss. Deep mode enables aggressive lossy compression (line fusion, word drop, paraphrase, elision) and requires >=95% semantic preservation with a loss ledger.
Light and medium modes guarantee 100% semantic preservation. Deep mode targets >=95% preservation and explicitly reports any low-value details removed. Deduplication merges facts rather than discarding them, so repeated information is preserved in the most specific form.
Yes. Pass a folder path (e.g., `agents/`) to process all .md files in that directory, or pass comma-separated file paths to process them sequentially.
The skill verifies file paths and URLs as part of reference integrity checks (rules R.1–R.3) and linearizes circular references to prevent broken links.
Typical reductions are 20–40% depending on input verbosity and mode. Light mode focuses on wording; medium and deep modes restructure content more aggressively for greater compression.
Full instructions (SKILL.md)
Source of truth, from kochetkov-ma/claude-brewcode.
name: text-optimizer description: "Optimizes text, prompts, and documentation for LLM token efficiency. Applies 52 research-backed rules across 8 categories: Claude behavior, token efficiency, structure, deduplication, reference integrity, perception, LLM comprehension, and aggressive lossy (deep only). Use when optimizing prompts, reducing tokens, compressing verbose docs, or improving LLM instruction quality." license: MIT metadata: author: "kochetkov-ma" version: "2.16.0" source: "claude-brewcode" allowed-tools: Read Write Edit Grep Glob
Plugin: kochetkov-ma/claude-brewcode
Text Optimizer
Reduces token count in prompts, docs, and agent instructions by 20–40% without losing meaning. Applies 52 research-backed rules across 8 categories: Claude behavior, token efficiency, structure, deduplication, reference integrity, perception, LLM comprehension, aggressive lossy (deep only).
Benefits: cheaper API calls · faster model responses · clearer LLM instructions · fewer hallucinations
Examples:
/text-optimize prompt.md # single file, medium mode (default)
/text-optimize -d agents/ # deep mode — all .md files in directory
Skill text is written for LLM consumption and optimized for token efficiency.
Text & File Optimizer
Step 0: Load Rules
REQUIRED: Read
references/rules-review.mdbefore ANY optimization. If file not found -> ERROR + STOP. Do not proceed without rules reference.
Modes
Parse $ARGUMENTS: -l/--light | -d/--deep | no flag -> medium (default).
| Mode | Flag | Scope |
|---|---|---|
| Light | -l, --light | Text cleanup only — structure, lists, flow untouched |
| Medium | (default) | Balanced restructuring — all standard transformations |
| Deep | -d, --deep | Max density — rephrase, merge, compress aggressively |
Rule ID Quick Reference
| Category | Rule IDs | Scope |
|---|---|---|
| Claude behavior | C.1-C.8 | Literal following, avoid "think", positive framing, match style, descriptive instructions, overengineering, avoid ALL-CAPS, prompt format |
| Token efficiency | T.1-T.8, T.10 | Tables, bullets, one-liners, inline code, abbreviations, filler, comma lists, arrows, strip whitespace |
| Structure | S.1-S.8 | XML tags, imperative, single source, context/motivation, blockquotes, progressive disclosure, consistent terminology, ref depth |
| Deduplication | D.1-D.6 | Exact/near/cross-format merge, emphasis cap <=2, cross-file SSOT, wrong-merge guard |
| Reference integrity | R.1-R.3 | Verify file paths, check URLs, linearize circular refs |
| Perception | P.1-P.6 | Examples near rules, hierarchy, bold keywords, standard symbols, instruction order, default over options |
| LLM Comprehension | L.1-L.8 | Critical info position, documents-first, conciseness, quote-first, add WHY, reiterate constraint, prompt repetition, preserve scope qualifiers |
| Aggressive lossy (deep only) | A.1-A.4 | Line fusion, low-value word drop, aggressive paraphrase, common-knowledge elision |
ID-to-Rule Mapping
| ID | Rule | ID | Rule |
|---|---|---|---|
| C.1 | Literal instruction following | C.2 | Avoid "think" word |
| C.3 | Positive framing (do Y not don't X) | C.4 | Match prompt style to output |
| C.5 | Descriptive over emphatic instructions | C.6 | Overengineering prevention |
| T.1 | Tables over prose (multi-column) | T.2 | Bullets over numbered (~5-10%) |
| T.3 | One-liners for rules | T.4 | Inline code over blocks |
| T.5 | Standard abbreviations (tables only) | T.6 | Remove filler words |
| T.7 | Comma-separated inline lists | T.8 | Arrows for flow notation |
| S.1 | XML tags for sections | S.2 | Imperative form |
| S.3 | Single source of truth | S.4 | Add context/motivation |
| S.5 | Blockquotes for critical | S.6 | Progressive disclosure |
| R.1 | Verify file paths | R.2 | Check URLs |
| R.3 | Linearize circular refs | P.1 | Examples near rules |
| P.2 | Hierarchy via headers (max 3-4) | P.3 | Bold for keywords (max 2-3/100 lines) |
| P.4 | Standard symbols (→ + / ✅❌⚠️) | ||
| S.7 | Consistent terminology | S.8 | One-level reference depth |
| P.5 | Instruction order (anchoring) | P.6 | Default over options |
| C.7 | Avoid ALL-CAPS emphasis (4.x) | C.8 | Prompt format → output format |
| T.10 | Strip whitespace from code | ||
| L.1 | Critical info at START or END | L.2 | Documents first, query last |
| L.3 | Explicitly request conciseness | L.4 | Quote-first grounding |
| L.5 | Add WHY to instructions | L.6 | Reiterate constraint at END |
| L.7 | Prompt repetition (non-reasoning) | L.8 | Preserve scope qualifiers |
| D.1 | Exact-duplicate merge | D.2 | Near-duplicate merge (keep specific) |
| D.3 | Cross-format duplicate | D.4 | Emphasis cap (<=2/doc, echo @ END) |
| D.5 | Cross-file dedup (SSOT + pointer) | D.6 | Wrong-merge guard |
| A.1 | Line fusion (loss-free merge) | A.2 | Low-value word drop (gate-neutral) |
| A.3 | Aggressive paraphrase (preserved) | A.4 | Common-knowledge elision (ledger) |
Mode-to-Rules Mapping
| Mode | Applies | Notes |
|---|---|---|
| Light | C.1-C.8, T.6, D.1, R.1-R.3, P.1-P.4, L.1-L.8 | Text cleanup only — no restructuring |
| Medium | All rules (C + T + S + D + R + P + L) | Balanced transformations |
| Deep | All rules (C + T + S + D + R + P + L) + A.1-A.4 (aggressive lossy) | Merge sections, max compression |
Loss Budget per Mode
Content essence is untouchable at light/medium; small deliberate loss only at deep — explicitly reported. Dedup-merged facts count as preserved, never as loss.
| Mode | Semantic match target | Allowed loss |
|---|---|---|
| Light | 100% | None — wording cleanup only |
| Medium | 100% | None — self-check fact inventory, zero loss |
| Deep | >= 95% | Low-value connective detail; self-verify round required, warn with loss list if < 95% |
Deduplication Pass (All Modes)
Runs during analysis, BEFORE compression:
- Build fact inventory: one atomic fact per line, numbered
- Flag facts appearing 2+ times (exact, reworded, or cross-format)
- Classify each repeat: intentional emphasis (marked critical/blockquote, or start+end sandwich) vs accidental
- Accidental -> merge to single MOST SPECIFIC statement (D.1-D.3), best position wins
- Intentional -> cap at 2: full form early + <=1-line echo at END (D.4)
- Wrong-merge guard (D.6): differing scope/numbers/conditions = NOT duplicates — keep both
Usage
| Input | Action |
|---|---|
| No args | Prompt user for file or folder path |
| Single path | Process file directly |
path1, path2 | Process files sequentially |
-l file.md | Light mode — text cleanup only |
-d file.md | Deep mode — max compression |
folder/ | All .md files in directory |
File Processing
Input Parsing
| Input | Action |
|---|---|
| No args | Prompt user for file or folder path |
| Single path | Process directly |
path1, path2 | Process files sequentially |
Execution Flow
- Read
references/rules-review.md— load all optimization rules - Read target file(s)
- Analyze: identify type (prompt, docs, agent, skill), note critical info and cross-references
3a. Dedup pass (D.1-D.6): fact inventory -> merge accidental dups -> cap intentional emphasis at 2/doc
3b. Deep only: aggressive lossy pass (A.1 fusion -> A.3 paraphrase -> A.2 word drop -> A.4 elision); A.2/A.4 drops -> loss ledger; A.4 counts as
elided-knownagainst the >=95% gate, A.2 is gate-neutral - Apply rules by mode (see Mode-to-Rules Mapping)
- Edit file with optimized content 5a. Medium: self-check — re-check fact inventory against output, zero loss required 5b. Deep mode: self-verify — fact inventory original vs compressed, (kept + merged)/total >= 95%; merged = preserved; warn with loss list if below
- Generate optimization report
Quality Checklist
Before
- Read entire text
- Identify type (prompt, docs, agent, skill)
- Note critical info and cross-references
During — Apply by Mode
| Check | Light | Med | Deep |
|---|---|---|---|
| C.1-C.8 (Claude behavior) | Yes | Yes | Yes |
| T.6 (filler removal) | Yes | Yes | Yes |
| T.1-T.5, T.7-T.8 (token compression) | - | Yes | Yes |
| T.10 (strip code whitespace) | - | Yes | Yes |
| S.1-S.8 (structure/clarity) | - | Yes | Yes |
| D.1 (exact dedup) | Yes | Yes | Yes |
| D.2-D.4, D.6 (smart dedup + emphasis cap) | - | Yes | Yes |
| D.5 (cross-file dedup, folder runs) | Yes | Yes | Yes |
| R.1-R.3 (reference integrity) | Yes | Yes | Yes |
| P.1-P.4 (LLM perception) | Yes | Yes | Yes |
| P.5-P.6 (anchoring, default-over-options) | - | Yes | Yes |
| L.1-L.8 (LLM comprehension) | Yes | Yes | Yes |
| A.1-A.4 (aggressive lossy) | - | - | Yes |
| Loss within mode budget | 100% | 100% | >=95% |
After
- All facts preserved
- Logic consistent
- References valid (R.1-R.3)
- Tokens reduced
Output Format
## Optimization Report: [filename]
| Metric | Before | After | Change |
|--------|--------|-------|--------|
| Lines | X | Y | -Z% |
| Tokens | ~X | ~Y | -Z% |
### Rules Applied
- [Rule IDs]: [Description of changes]
### Issues Found & Fixed
- [Issue]: [Resolution]
### Cross-Reference Verification
- [x] All file refs valid (R.1)
- [x] All URLs checked (R.2)
- [x] No circular refs (R.3)
---
**Full development platform:** [brewcode](https://github.com/kochetkov-ma/claude-brewcode) — infinite focus tasks, 16 agents, quorum reviews, knowledge persistence.
Install: `claude plugin marketplace add https://github.com/kochetkov-ma/claude-brewcode && claude plugin install brewcode@claude-brewcode`
Anti-Patterns
| Avoid | Why |
|---|---|
| Remove all examples | Hurts generalization (P.1) |
| Over-abbreviate | Reduces readability (T.5 caveat) |
| Generic compression | Domain terms matter |
| Over-aggressive language | Opus 4.5 overtriggers (C.5) |
| Flatten hierarchy | Loses structure (P.2) |
| "Don't do X" framing | Less effective than "Do Y" (C.3) |
| Overengineer prompts | Opus 4.5 follows literally (C.6) |
| Overload single prompts | Divided attention, hallucinations (S.3) |
| Over-focus on wording | Structure > word choice (T.1) |
| Merge similar-looking facts blindly | Different scope/numbers/conditions = different facts (D.6) |
Related skills
More from kochetkov-ma/claude-brewcode and the wider catalog.
memory-optimize
Optimize Claude Code memory files in 4 interactive steps, reducing token count by 30-50%.
finlab
Comprehensive quantitative trading package for global stock markets with backtesting, factor analysis, and strategy development.
404-page-generator
When the user wants to create, optimize, or audit 404 error pages. Also use when the user mentions "404 page," "404 error," "error page," "page not found," "broken link page," "404 design," "custom 404," "404 redirect," "404 page UX," or "404 recovery." For sitewide page planning, use website-structure.
about-page-generator
When the user wants to create, optimize, or audit About page content. Also use when the user mentions "about page," "about us," "company story," "our team," "about section," "company overview," "brand story," "team page," or "who we are." For sitewide page planning, use website-structure.
affiliate-marketing
When the user wants to plan, implement, or optimize affiliate marketing strategy. Also use when the user mentions "affiliate marketing," "affiliate program strategy," "CPS model," "affiliate recruitment," "commission structure," "affiliate partners," "affiliate network," "affiliate tracking," "affiliate commission," or "partner marketing." For affiliate page, use affiliate-page-generator.
affiliate-page-generator
When the user wants to create, optimize, or audit affiliate program page content. Also use when the user mentions "affiliate program," "affiliate page," "partner program," "referral program page," "affiliate landing," "partner landing," "commission page," or "affiliate signup." For affiliate program strategy, use affiliate-marketing.