skill-creator Complete Guide - Claude Skills Development Framework¶

For / Key Points

For: Developers creating or improving Skills for Claude Code, Claude.ai, or the Claude API

Key Points:

skill-creator now covers drafting, testing, comparison, iteration, and trigger optimization
The core loop is draft → compare with and without the Skill → human review → revise
Keep SKILL.md lean; put deterministic work in scripts/ and detailed material in references/

A generated SKILL.md is not proof that a Skill works. The current skill-creator asks teams to test two or three realistic tasks, compare runs with and without the Skill, review the artifacts, and iterate using both qualitative and quantitative evidence.¹²

The practical question is: how do you use skill-creator to build a Skill whose benefit can be demonstrated, not merely described?

What skill-creator Is Now¶

skill-creator is a Skill for creating, improving, and evaluating Agent Skills. Its scope includes new Skills, existing-Skill revisions, benchmark comparisons, and description optimization for more accurate triggering.²

The workflow is iterative.

flowchart LR
    A[Capture intent] --> B[Draft SKILL.md]
    B --> C[Create 2–3 tests]
    C --> D[Compare with and without Skill]
    D --> E[Human reviews artifacts]
    E --> F[Revise instructions and scripts]
    F --> C
    E --> G[Optimize description]
    G --> H[Validate and package]

The older summary—answer a few questions and receive a finished Skill—misses the main change. The current implementation treats measured iteration as the center of Skill development.

Install It in Claude Code¶

Register Anthropic's official repository as a Claude Code plugin marketplace, then install example-skills, which includes skill-creator.¹

/plugin marketplace add anthropics/skills
/plugin install example-skills@anthropic-agent-skills

Install the separate document plugin only if you also need document-processing Skills.

/plugin install document-skills@anthropic-agent-skills

skill-creator is published under Apache 2.0. The docx, pdf, pptx, and xlsx Skills are source-available under different terms.¹

After installation, invoke skill-creator explicitly in natural language.

Use skill-creator to build a Skill that reviews Markdown articles.

It can also start from an existing Skill.

Use skill-creator to evaluate this Skill's trigger accuracy and output quality, then improve it.

Decide Four Things First¶

Define success before writing files. skill-creator reuses answers already present in the conversation and asks only for missing information.²

Settle these four questions:

What should the Skill enable Claude to do?
When should it trigger?
What output format should it produce?
Should the Skill have test cases?

File transforms, extraction, code generation, and fixed workflows benefit from objective tests. Subjective work such as writing voice or visual art may be better evaluated primarily by people rather than forced into weak assertions.

Standard Skill Structure¶

A directory containing SKILL.md is the minimum valid Skill. The public Agent Skills specification defines this basic structure.³

my-skill/
├── SKILL.md          # Required: metadata and instructions
├── scripts/          # Optional: executable code
├── references/       # Optional: reference material
└── assets/           # Optional: templates and resources

SKILL.md Frontmatter¶

---
name: article-review
description: Review Markdown articles for factual, structural, and style problems. Use for article review, proofreading, and pre-publication checks.
compatibility: Requires Python 3.12 and network access for source verification
---

The public specification defines these main constraints.³

Field	Required	Main constraint
`name`	Yes	Up to 64 characters; lowercase letters, digits, and hyphens; matches parent directory
`description`	Yes	Up to 1,024 characters; states both purpose and trigger context
`license`	No	License name or bundled license reference
`compatibility`	No	Up to 500 characters; product, package, or network requirements
`metadata`	No	Additional key-value metadata
`allowed-tools`	No	Experimental; support varies by implementation

Claude Platform documentation also reserves anthropic and claude in the name field.⁴ The open specification permits descriptions up to 1,024 characters, while the Claude Help Center documents a 200-character product limit. For portability across Claude surfaces, staying within 200 characters is the conservative choice.³⁵

The Description Drives Triggering¶

Claude does not keep every Skill body in context. At startup, it sees the name and description; it loads the matching SKILL.md only after deciding the Skill applies.⁴

Putting a “when to use” section only in the body is therefore too late. The description needs both:

What the Skill does
Which requests, files, or contexts should trigger it

“Helps with PDFs” is weak. “Extracts PDF text, fills forms, and merges files; use for PDF, form, and document-extraction tasks” gives the agent an actionable boundary.

Design for Progressive Disclosure¶

Do not pack every detail into SKILL.md. The public specification and Anthropic's implementation use three loading levels.²³

Level	Loaded	Content
1. Metadata	Always	`name` and `description`, about 100 tokens
2. Instructions	On trigger	SKILL.md body, ideally under 500 lines
3. Resources	As needed	scripts, references, and assets

Move detailed API specifications and domain procedures into references/. Move repeated deterministic checks into scripts/, and put output templates or media in assets/.

cloud-deploy/
├── SKILL.md
├── scripts/
│   └── validate_manifest.py
└── references/
    ├── aws.md
    ├── azure.md
    └── gcp.md

Avoid deep reference chains. Link required material directly from SKILL.md and add a table of contents to large reference files.

The Current Evaluation Loop¶

1. Create Two or Three Realistic Tests¶

Use prompts that resemble real user requests. Store them in evals/evals.json.²

{
  "skill_name": "article-review",
  "evals": [
    {
      "id": 1,
      "prompt": "Review docs/blog/example.md before publication",
      "expected_output": "A prioritized review report",
      "files": []
    }
  ]
}

A prompt such as “review this” is too thin to exercise real routing decisions. Mix file names, goals, constraints, and plausible ambiguity into the test set.

2. Run the Skill and a Baseline Together¶

For a new Skill, compare the same task under two conditions:

with_skill: uses the draft Skill
without_skill: uses no Skill

When improving an existing Skill, snapshot the old version and use that as the baseline. Run both conditions in the same batch to reduce timing and environment differences.

3. Separate Assertions from Human Judgment¶

Do not confuse machine-checkable requirements with subjective quality. “Produced a CSV” and “contains five required columns” can be assertions. “The explanation is easy to read” and “the design feels coherent” usually need human review.

The current pipeline records pass rate, wall-clock time, and token usage. A Skill can preserve output quality yet add enough latency or context overhead to make adoption unattractive.

4. Review Artifacts in the Eval Viewer¶

eval-viewer/generate_review.py combines each test's artifact, formal grades, previous iteration, and feedback field.² In a headless environment, --static writes a standalone HTML review file.

python eval-viewer/generate_review.py \
  ../article-review-workspace/iteration-1 \
  --skill-name "article-review" \
  --benchmark ../article-review-workspace/iteration-1/benchmark.json \
  --static ../article-review-workspace/iteration-1/review.html

A higher score is not success if the artifact is unusable. That is why skill-creator keeps human review inside the loop.

5. Revise and Rerun¶

Adding exceptions for one failing prompt produces an overfit Skill. When several tests recreate the same helper, bundle it once as a script; when an instruction adds no value, remove it.

Write the next run to iteration-2/ and compare it with the previous workspace. Stop when the user is satisfied, feedback is empty, or revisions stop producing meaningful improvement.

Optimize Trigger Accuracy Separately¶

Output quality and trigger accuracy are different evaluations. A strong Skill is useless if it does not activate when needed, while over-triggering wastes context and time.

The current skill-creator improves descriptions through this process.²

Create 8–10 realistic should-trigger and 8–10 should-not-trigger queries
Include hard negative examples from adjacent domains
Split 60% into training and 40% into a held-out set
Run each query three times and iterate up to five times
Select the description by held-out, not training, performance

Example:

cd /path/to/skill-creator
python -m scripts.run_loop \
  --eval-set /path/to/eval_set.json \
  --skill-path /path/to/article-review \
  --model <current-model-id> \
  --max-iterations 5 \
  --verbose

This loop uses claude -p, so it is not available in Claude.ai alone. On Claude.ai, manually try several positive and negative trigger prompts, then revise the description iteratively.

Validate and Package¶

skill-creator bundles structure validation and .skill packaging scripts.¹²

cd /path/to/skill-creator

# Validate frontmatter and naming
python scripts/quick_validate.py /path/to/article-review

# Create a .skill file, which uses ZIP format
python -m scripts.package_skill /path/to/article-review ./dist

Packaging stops if validation fails and excludes the root evals/ directory from the distributable. For specification-only validation, the Agent Skills documentation also provides skills-ref validate ./my-skill.³

Claude Code, Claude.ai, and API Differences¶

The Skill format is portable, but deployment and runtime behavior differ.

Area	Claude Code	Claude.ai	Claude API
Install	Plugin, `.claude/skills/`, or `~/.claude/skills/`	Upload ZIP in Customize > Skills	Upload through Skills API
Trigger	Description-based auto-selection or explicit request	Auto-selected after enabling	Reference `skill_id` in container
Scripts	Local execution environment	Requires Code Execution	Code Execution container
Network	Follows Claude Code permissions	Follows execution environment	Unavailable in Skill container
Dependencies	Depends on local environment and permissions	Standard repositories may be available	No runtime installation

The API uses pptx, xlsx, docx, or pdf as pre-built Skill IDs, while custom Skills are uploaded through /v1/skills.⁴ API Skills run with Code Execution; they are not equivalent to pasting SKILL.md into a system prompt.

Agent Skills are also not eligible for Zero Data Retention. Systems with strict retention requirements should review the current policy before adoption.⁴

Security Checks That Matter¶

Audit third-party Skills as executable dependencies, not prose documents. Anthropic recommends using Skills you created or obtained from Anthropic and thoroughly reviewing unknown Skills.⁴

Check that:

SKILL.md does not request actions outside the stated purpose
scripts do not access unnecessary files or credentials
fetched external content is not treated as trusted instruction
API keys and passwords are not hardcoded
package and script dependencies can be pinned and reviewed

A harmless description does not make bundled code or remote references safe. Use the Principle of Lack of Surprise: a Skill should not perform behavior the user could not reasonably infer from its purpose.

When a Skill Fits¶

Decision	Good fit	Prefer another mechanism
Reuse	Repeated multi-step task	One-off simple request
Knowledge	Organization-specific procedure	Fast-changing temporary facts
Execution	Deterministic checks or transforms	External service operation is the main purpose
Evaluation	Artifacts or steps can be compared	Success cannot be defined

MCP is usually the better core mechanism for connecting to an external service. A Skill can still define the organization's procedure for using that MCP; the two mechanisms are complementary.

The Practical Next Step¶

skill-creator's main value is not faster SKILL.md authoring. It is the ability to show whether the Skill improves on a no-Skill baseline and preserve the evidence.

Start with a narrow Skill and two or three tests. Review output quality, script only the repeated deterministic work, and optimize triggering last. This sequence avoids investing in a large instruction package before proving that the workflow helps.

skill-creator Eval Pipeline Deep Dive - Architecture and rationale behind the evaluation loop
Skill Obsolescence and Recovery Design - Continuous evaluation and maintenance
Claude Skills Complete Guide - Broader Skills overview
Claude Skills API Implementation Guide - API integration