Semantic Compressor

A fail-closed Claude Code skill for reducing skill and agent Markdown without silently changing behavior.

English | 繁體中文

[!IMPORTANT] v2.1.0 aligns discovery and frontmatter checks with Claude Code 2.1.305 and fixes several v2.0.0 bugs. Backups made by v2.0.0 remain restorable. If you are upgrading from v1, review CHANGELOG.md first.

What’s new in v2.1.0

• Scan covers what Claude Code loads today: custom commands in .claude/commands/, plugin skills, commands, and agents (including plugins kept in .claude/skills//), and only those agent files Claude Code loads as agents.
• Scan skips, with a reason, names Claude Code refuses, agent files it ignores, frontmatter outside the supported YAML subset, and non-UTF-8 files.
• Validation fails a skill or command candidate whose description and when_to_use grow past Claude Code’s 1,536-character listing cap, and warns when a candidate is still over the cap or a SKILL.md has 500 or more lines.
• Fixed: apply no longer puts the marker where Claude Code reads the listing text of a skill or command without a description.
• Fixed: agents are no longer dropped when the project path contains a folder named like build or dist, and Claude Code worktrees in .claude/worktrees/ no longer list every entrypoint twice.
• Fixed: restore accepts backups written by any v2 release, so upgrading does not strand v2.0.0 backups.

What v2 changes

Semantic Compressor now separates drafting from mutation:

• scan, compress, verify, benchmark, and restore preview are read-only for source files.
• Only explicit apply and restore ... --apply can replace a source file.
• A dependency-free Python control script validates paths, frontmatter, preservation inventories, ratios, hashes, backups, manifests, and rollback state.
• A fresh read-only reviewer judges conceptual equivalence after deterministic checks pass.
• Semantic review is reported as model judgment with evidence, never as an absolute guarantee.

The skill is user-invocable only. Its frontmatter sets disable-model-invocation: true, so Claude does not trigger it automatically. Use scan, compress, verify, benchmark, and restore preview as source-unmodified dry runs.

Safety model

Requirements

• Claude Code 2.1.196 or newer (frontmatter checked with Claude Code 2.1.305)
• Python 3.10 or newer (CI tests 3.10 through 3.15)
• Git for installation from this repository
• An authenticated Claude Code session using any supported subscription, API, or cloud provider configuration

A direct Anthropic API key is not required if Claude Code is already authenticated through another supported provider. If the first session reports Not logged in, run /login inside Claude Code or complete your provider-specific authentication before invoking the skill.

Check your local versions in a shell:

claude --version
python3 --version

The shell examples below use macOS, Linux, or WSL syntax.

Installation

The commands below install v2.1.0 from the public default branch.

Personal installation

Use a personal installation to make the skill available in every local project:

git clone https://github.com/claude-world/skills-optimizer.git skills-optimizer
test -f skills-optimizer/.claude/skills/semantic-compressor/scripts/semantic_compressor.py
mkdir -p ~/.claude/skills
cp -R skills-optimizer/.claude/skills/semantic-compressor ~/.claude/skills/

Installed path:

~/.claude/skills/semantic-compressor/SKILL.md

Project installation

Use a project installation when the repository should pin and share the skill with collaborators:

test -f /path/to/skills-optimizer/.claude/skills/semantic-compressor/scripts/semantic_compressor.py
mkdir -p .claude/skills
cp -R /path/to/skills-optimizer/.claude/skills/semantic-compressor .claude/skills/

Review project skills before accepting Claude Code workspace trust, then commit the installed directory if the team should share it.

Verify the installation and launch Claude Code

For a personal installation, verify the bundled control script in a shell:

python3 ~/.claude/skills/semantic-compressor/scripts/semantic_compressor.py --version

For a project installation, use:

python3 .claude/skills/semantic-compressor/scripts/semantic_compressor.py --version

The expected script version is 2.1.0. Start a fresh Claude Code session after installation or update:

claude

Then invoke the skill inside Claude Code, not in your shell:

/semantic-compressor scan

Update

Update the checkout first:

git -C skills-optimizer pull --ff-only

Keep the previous personal installation recoverable, then copy the new one:

mkdir -p "$HOME/.claude/disabled-skills"
SEMANTIC_COMPRESSOR_PREVIOUS="$(mktemp -d "$HOME/.claude/disabled-skills/semantic-compressor-XXXXXXXX")"
mv "$HOME/.claude/skills/semantic-compressor" "$SEMANTIC_COMPRESSOR_PREVIOUS/"
cp -R skills-optimizer/.claude/skills/semantic-compressor "$HOME/.claude/skills/"

For a project installation:

mkdir -p .claude/disabled-skills
SEMANTIC_COMPRESSOR_PREVIOUS="$(mktemp -d ".claude/disabled-skills/semantic-compressor-XXXXXXXX")"
mv .claude/skills/semantic-compressor "$SEMANTIC_COMPRESSOR_PREVIOUS/"
cp -R /path/to/skills-optimizer/.claude/skills/semantic-compressor .claude/skills/

Re-run the version check and start a fresh Claude Code session.

Uninstall

Move the skill outside the discovery directory so the uninstall remains recoverable:

mkdir -p "$HOME/.claude/disabled-skills"
SEMANTIC_COMPRESSOR_DISABLED="$(mktemp -d "$HOME/.claude/disabled-skills/semantic-compressor-XXXXXXXX")"
mv "$HOME/.claude/skills/semantic-compressor" "$SEMANTIC_COMPRESSOR_DISABLED/"

For a project installation:

mkdir -p .claude/disabled-skills
SEMANTIC_COMPRESSOR_DISABLED="$(mktemp -d ".claude/disabled-skills/semantic-compressor-XXXXXXXX")"
mv .claude/skills/semantic-compressor "$SEMANTIC_COMPRESSOR_DISABLED/"

Start a fresh Claude Code session after uninstalling. Delete the moved directory later only if you no longer need it.

Usage

Run all commands in an interactive Claude Code session:

Paths may be absolute or project-relative, but they must resolve inside the active project. Put -- before a path beginning with a dash.

Scan

/semantic-compressor scan

Scan reports candidates and skipped. Each candidate has a kind: skill, command, or agent. It excludes the compressor itself, supporting references, samples, backups, workspaces, Claude Code worktrees in .claude/worktrees/, symlinks, and files already carrying the v2 marker. Each skipped file has a reason:

Compress without changing the source

/semantic-compressor compress .claude/skills/my-skill/SKILL.md

The source remains byte-for-byte unchanged. The skill creates a source-bound inventory and candidate below:

.claude/semantic-compressor-work//

It then runs deterministic validation and a fresh read-only semantic review. For skills and commands, validation also reports the listing length: Claude Code lists description, plus - and when_to_use, and truncates entries above 1,536 characters. A candidate that grows past the cap fails. A candidate that stays over the cap, or a SKILL.md with 500 or more lines, gets a warning that does not change the result. Inspect the reported candidate, hashes, evidence, ratio, warnings, and final state before applying.

Apply an accepted candidate

/semantic-compressor apply .claude/skills/my-skill/SKILL.md

apply snapshots the source, candidate, completed inventory, and independent review; validates their bound hashes and per-concept evidence; creates and verifies a backup; writes a manifest; inserts the v2 idempotence marker without changing candidate body indentation; checks for concurrent source edits; and atomically replaces the source. If any gate fails, this run does not replace the source.

The marker goes right after the frontmatter. A skill or command without a description gets the marker at the end instead, because Claude Code lists its first non-empty body line in place of a description.

Verify an original/candidate pair

/semantic-compressor verify .claude/skills/my-skill/SKILL.md .claude/semantic-compressor-work//candidate.md --inventory .claude/semantic-compressor-work//inventory.json

Verify is read-only. It reports deterministic preservation separately from semantic judgment and ratio status. A semantically acceptable candidate below the configured ratio is not mislabeled as a successful compression.

Benchmark bundled samples

/semantic-compressor benchmark

Benchmark locates samples relative to the installed skill, not the current project. It reports reproducible line, word, character, byte, hash, and pairing data. The ratio gate uses whitespace-delimited words; these metrics are not Claude tokenizer counts and do not prove semantic equivalence.

Preview and apply a restore

Preview first:

/semantic-compressor restore .claude/backups//manifest.json

Apply only after reviewing the operations:

/semantic-compressor restore .claude/backups//manifest.json --apply

Restore accepts only .claude/backups//manifest.json from the same project root, written by a v2 release, with the current one-file schema and applied state. It refuses a target changed after compression, a changed file mode, or a backup whose SHA-256 no longer matches the manifest.

Backups and manifests

Successful apply creates:

.claude/backups//
├── manifest.json
└── files/
    └── 

The manifest records the source path, backup path, file mode, project root, compressor version, completed-inventory path/hash, semantic-review path/hash/reviewer, and SHA-256 values before compression, for the validated candidate, and after marker insertion. Backup bytes and the source/candidate/inventory-bound review artifact are verified before source replacement. Reported manifest paths are the authoritative rollback handles; do not move a manifest between project roots or guess a timestamped directory.

Result states

Only PASS, APPLIED, and RESTORED are success states for their respective operations.

Limitations, cost, and privacy

• Semantic Compressor handles Claude Code skill, command, and agent Markdown, not arbitrary documents or source code.
• Symlinks, directories, non-UTF-8 files, and paths outside the active project are rejected.
• Scan covers the default plugin layout. Components at custom paths declared in plugin.json are not scanned, but you can compress them by path.
• The listing check uses Claude Code’s default cap of 1,536 characters. A skillListingMaxDescChars setting does not change the gate.
• The default minimum compression ratio is 2.0:1 by whitespace-delimited words. Already-concise files may correctly return NOT_COMPRESSIBLE.
• Candidate drafting and independent review use model calls. Cost and latency grow with file size and retries, with at most three candidate drafts per file.
• Source, candidate, and inventory content enters the configured Claude Code model context and is processed by your configured provider. Do not use the skill on data that provider is not authorized to receive.
• The local Python control script performs deterministic checks, but conceptual equivalence remains a model judgment. Review evidence before apply.
• The script can verify reviewer attestation and artifact bindings, not the reviewer’s actual identity. Claude Code orchestration must assign a genuinely fresh read-only reviewer; a user-fabricated review JSON is outside this trust boundary.
• If a fresh independent reviewer is unavailable, the skill returns REVIEW_REQUIRED and must not apply.
• Workspaces and backups intentionally remain on disk for inspection and rollback. Check git status and your repository ignore policy before committing.

Deterministic control script

The bundled script is useful for diagnostics and CI-style checks:

python3 .claude/skills/semantic-compressor/scripts/semantic_compressor.py --help
python3 .claude/skills/semantic-compressor/scripts/semantic_compressor.py benchmark \
  --samples .claude/skills/semantic-compressor/samples \
  --format markdown

Direct script execution provides deterministic controls only. It does not replace the skill’s independent semantic review requirement.

Tests

From the repository root:

python3 -m unittest discover -s tests -v
python3 scripts/validate_project.py

The suite covers discovery scope across skills, commands, agents, and plugins; the Claude Code name, agent, and listing rules; unique project-local workspaces; idempotence markers and their placement; symlink-component and backup-path safety; conservative frontmatter validation; completed source-bound inventories; literal failures; source/candidate/inventory-bound reviews; apply races and unknown-byte rollback refusal; strict manifest restore across v2 releases; file modes; and deterministic benchmark edge cases. The project validator checks the shipped skill against the current Claude Code frontmatter rules, the sample evidence, the benchmark output, and documentation drift.

Project structure

skills-optimizer/
├── README.md
├── README.zh-TW.md
├── CONTRIBUTING.md
├── CHANGELOG.md
├── BENCHMARK-REPORT.md
├── benchmark-results.json
├── scripts/
│   ├── validate_project.py
│   └── validate-format.sh
├── tests/
│   ├── test_project_validator.py
│   └── test_semantic_compressor.py
└── .claude/skills/semantic-compressor/
    ├── SKILL.md
    ├── scripts/
    │   └── semantic_compressor.py
    ├── references/
    │   ├── inventory-schema.md
    │   ├── strategies.md
    │   └── examples.md
    ├── evals/
    └── samples/
        ├── skills/
        ├── agents/
        └── verification/

Contributing

See CONTRIBUTING.md. Behavior changes must include tests; documentation changes affecting commands or safety must update both READMEs.

License

MIT — see LICENSE.