Claude Agent Skills in Production: Packaging, Permissions, and Cache Traps

Why We Brought This Tool Into Our Lab#

We brought Claude Agent Skills into our lab because agent behavior was becoming harder to reuse than application code. We had spread working prompts, scripts, templates, and operating procedures across system prompts, repository documentation, and code that coordinated the agent’s steps. Moving one capability into another service meant copying all four layers and hoping nobody omitted a constraint.

A Skill packages instructions and resources so teams can reuse them together. Instead of placing every instruction in the initial prompt, we package a concise description in SKILL.md and keep detailed procedures, scripts, and assets beside it. Claude first sees basic information about the Skill, such as its name and description. This metadata lets it discover the capability before loading detailed instructions when the task matches. Progressive disclosure means Claude loads a short description first and detailed instructions when needed. Skills use a standard file layout, not a new protocol for calling remote tools.

The Model Context Protocol, or MCP, lets an agent call tools provided by another program, locally or over a network. A Claude Agent Skill gives the agent instructions for a task and can direct it to local scripts and files. We use MCP to connect agents to other software. We use Skills to share task instructions across applications.

Our test converted a brief in JSON, a text format for storing structured data, into a PowerPoint file. The same brief had to produce the same slide content. The model had to discover the Skill, read its instructions, invoke a bundled Python generator, and return the output path. This gave us observable failure points without depending on an external slide-generation service.

We treated these references as our implementation contract:

• The Agent Skills quickstart for the package structure and activation flow.
• The Agent Skills best-practices guide for concise instructions, discovery metadata, and progressive disclosure.
• We used the Agent SDK Skills reference to configure project Skill loading through Anthropic’s software development kit, a library for running agents from application code.
• The Agent SDK hosting guidance for isolation, filesystem persistence, permissions, and hosted runtime boundaries.

The marketing interpretation is that dropping in a Markdown file gives an agent a reliable new capability. Before deployment, we need to control how Claude finds the Skill, the software and supporting files it needs, and the tools it can use. We also need to check its outputs and control how outdated stored information is cleared or replaced.

This review therefore emphasizes packaging mechanics over prose. Teams standardizing agent capabilities can also review our tools collection or contact us about an agent deployment. Deployment also requires an environment that restricts the agent’s access and tests that check its behavior.

Hands-On Walkthrough: Setup, Execution & Output#

We created a clean Python project and installed the Claude Agent SDK and python-pptx, a library for creating PowerPoint files. We also installed a loader that reads settings from a .env file: bash mkdir claude-skill-lab cd claude-skill-lab

uv init --python 3.12 uv add claude-agent-sdk python-pptx python-dotenv

mkdir -p .claude/skills/pptx/scripts mkdir -p .claude/skills/pptx/assets mkdir -p briefs output

printf 'ANTHROPIC_API_KEY=replac...y\n' > .env chmod 600 .env Our working package used this exact repository layout: text claude-skill-lab/ ├── .claude/ │ └── skills/ │ └── pptx/ │ ├── SKILL.md │ ├── assets/ │ │ └── theme.json │ └── scripts/ │ └── create_deck.py ├── briefs/ │ └── launch.json ├── output/ ├── .env ├── pyproject.toml └── run_agent.py The critical point is that SKILL.md sits directly inside the individual Skill directory. Placing it at .claude/skills/SKILL.md, naming it skill.md, or adding another unnecessary directory level prevented the package from being discovered in our negative tests.

Our .claude/skills/pptx/SKILL.md was intentionally short: markdown#

name: pptx description: Create a PowerPoint presentation from a local JSON brief. Use this when the user asks for a slide deck, presentation, or PPTX file.#

Create decks only from an explicit JSON brief.

1. Read the brief before invoking the generator.
2. Run scripts/create_deck.py from this Skill directory.
3. Write the result beneath the repository output/ directory.
4. Never overwrite an existing file unless the user explicitly requests it.
5. Report the final path and the slide count.

The generator command is:

python .claude/skills/pptx/scripts/create_deck.py --brief <brief.json> --output <deck.pptx>

The brief schema is:

• title: non-empty string
• subtitle: optional string
• slides: array of objects with title and bullets We kept implementation detail out of the discovery description. The description tells Claude when to load the Skill; the body tells it what to do after selection. A vague description such as “Helps with documents” consistently created ambiguity once we added neighboring document Skills.

The bundled generator was deterministic: the same brief produced the same slide content. python import argparse import json from pathlib import Path

from pptx import Presentation

def load_brief(path: Path) -> dict: data = json.loads(path.read_text(encoding="utf-8")) if not isinstance(data.get("title"), str) or not data["title"].strip(): raise ValueError("brief.title must be a non-empty string") if not isinstance(data.get("slides"), list): raise ValueError("brief.slides must be an array") return data

def create_deck(brief: dict, output: Path) -> int: if output.exists(): raise FileExistsError(f"refusing to overwrite {output}")

output.parent.mkdir(parents=True, exist_ok=True)
presentation = Presentation()

title_slide = presentation.slides.add_slide(
    presentation.slide_layouts[0]
)
title_slide.shapes.title.text = brief["title"]
title_slide.placeholders[1].text = brief.get("subtitle", "")

for item in brief["slides"]:
    slide = presentation.slides.add_slide(
        presentation.slide_layouts[1]
    )
    slide.shapes.title.text = item["title"]
    frame = slide.placeholders[1].text_frame
    frame.clear()

    for index, bullet in enumerate(item.get("bullets", [])):
        paragraph = frame.paragraphs[0] if index == 0 else frame.add_paragraph()
        paragraph.text = str(bullet)
        paragraph.level = 0

presentation.save(output)
return len(presentation.slides)

def main() -> None: parser = argparse.ArgumentParser() parser.add_argument("--brief", type=Path, required=True) parser.add_argument("--output", type=Path, required=True) args = parser.parse_args()

brief = load_brief(args.brief.resolve())
slide_count = create_deck(brief, args.output.resolve())
print(json.dumps({
    "status": "created",
    "output": str(args.output.resolve()),
    "slides": slide_count,
}))

if name == "main": main() We launched the SDK with project settings explicitly enabled and with only the tools required by this flow: python import asyncio from pathlib import Path

from dotenv import load_dotenv from claude_agent_sdk import ClaudeAgentOptions, query

async def main() -> None: load_dotenv() root = Path(file).resolve().parent

options = ClaudeAgentOptions(
    cwd=str(root),
    setting_sources=["project"],
    allowed_tools=["Skill", "Read", "Bash"],
)

prompt = (
    "Use the pptx skill to create output/launch-review.pptx "
    "from briefs/launch.json. Do not overwrite an existing file."
)

async for message in query(prompt=prompt, options=options):
    print(message)

asyncio.run(main()) This simulated transcript follows the format of our test program’s output. We have left out secrets and extra message fields: text $ uv run python run_agent.py AssistantMessage: I will inspect the brief and use the pptx skill. ToolUse: Skill {"skill":"pptx"} ToolUse: Read {"file_path":"briefs/launch.json"} ToolUse: Bash { "command":"python .claude/skills/pptx/scripts/create_deck.py --brief briefs/launch.json --output output/launch-review.pptx" } ToolResult: {"status": "created","output":"/workspace/claude-skill-lab/output/launch-review.pptx","slides":4} AssistantMessage: Created output/launch-review.pptx with 4 slides.

$ test -s output/launch-review.pptx $ printf '%s\n' $? 0 We also ran the generator directly before involving Claude. This caught problems with the brief’s structure and required software without a model call. If the script fails on a fixed test input, adding Claude will not fix the underlying problem.

What Broke: The Gotchas and Limitations We Hit#

The SDK did not discover the Skill or report an error. The SDK session could answer normally, read files, and run shell commands, yet it never selected the Skill. The package itself was valid. Our missing line was: python setting_sources=["project"] An Agent SDK application may not read the same settings files as an interactive Claude Code session. We explicitly configure it to load project settings. We also include Skill in allowed_tools so the model can activate the Skill during the test.

The second failure was layout-related. These variants did not represent the package we meant to ship: text .claude/skills/SKILL.md .claude/skills/pptx/pptx/SKILL.md .claude/skills/pptx/skill.md skills/pptx/SKILL.md We encountered filename case differences when we moved the repository from a default macOS filesystem to Linux. Our Linux runner executes automated continuous integration checks whenever code changes. The SDK appeared to recognize Skill.md on our local case-insensitive filesystem but did not recognize it on Linux. Our packaging check now requires the literal filename SKILL.md and verifies that each Skill has one direct package root.

The third failure was weak discovery language. Claude rarely selected the Skill when its description explained implementation rather than the user’s goal. “Runs a Python script using python-pptx” describes how it works. “Create a PowerPoint presentation from a local JSON brief” describes when to use it. We rewrote descriptions around user intent and added distinguishing terms such as “slide deck,” “presentation,” and “PPTX.”

Putting every instruction in SKILL.md consumed more of the model’s limited input space and made it harder to understand why Claude selected a Skill. We kept decision rules in the main file and moved data formats, examples, scripts, and themes into separate files. The main file should tell Claude what to do and which supporting files to read when it needs details.

Permissions were the more serious production issue. This configuration is convenient: python allowed_tools=["Skill", "Read", "Write", "Bash"] This configuration also grants more access than the PowerPoint task needs. The SDK grants these tools to the whole agent session, not just to the command in SKILL.md. Activating the Skill does not restrict Bash to the files or folders needed for the presentation.

We gave each run a disposable working directory and made inputs read-only. We allowed writes only to the output directory, used an account without administrator privileges, and excluded unrelated credentials. We also ran SDK permission checks on commands when hosting requirements were stricter. Frontmatter is the metadata at the start of SKILL.md; we did not rely on it to restrict access. Instructions guide the agent, but operating-system controls enforce which files and resources it can use.

Asset paths caused another reproducible failure. A script normally interprets relative file paths from the folder where the process runs, unless its code explicitly chooses another base folder. A reference such as assets/theme.json worked when we happened to execute inside the Skill directory and failed when the SDK launched from the repository root. We corrected the script pattern to derive resources from Path(__file__).resolve().parent.parent / "assets".

Cache invalidation means stopping the reuse of stored information once it is outdated. This was less obvious because there are several caches to distinguish. We could not rely on edits to SKILL.md taking effect during an active SDK conversation. We restarted the session after changing discovery metadata or instructions. In hosted environments, we also versioned the complete Skill bundle and replaced the worker or container rather than mutating a shared long-lived directory.

Prompt caching lets the model service reuse work already done on the opening part of a prompt. Changing a Skill file does not automatically update every running process, conversation, saved filesystem copy, or cached prompt. Our release identifier includes a fingerprint calculated from SKILL.md, scripts, and assets. We record it with each run and reject execution environments that have a different Skill version from the requested release.

Finally, we needed to check more than whether the generated .pptx file existed. We opened the ZIP-based package through python-pptx, checked the slide count, and rejected zero-byte or unreadable outputs. A success message from the model does not verify the output file.

Scale, Latency & Cost vs. Alternatives#

Skills themselves add little coordination work. Most runtime costs come from generating model responses, calling tools and waiting for results, starting processes, and running the bundled script.

What this article could not verify#

We could not establish general figures for response time, requests handled per second, or memory use from the available evidence. These values depend on the model, hosting environment, prompt, and rules for running containers.

How the alternatives compare#

We could directly compare how the alternatives work in production:

The Skill beat a large system prompt once multiple agents needed our slide-generation procedure. It reduced the instructions sent at the start and let us package the generator, brief format, and examples together. It did not beat a direct Python call when the application already knew that every request was a slide-generation request. In that case, model-driven discovery would add cost without adding useful judgment.

Our break-even model is straightforward: text Break-even runs = packaging and evaluation effort / (maintenance effort per copied implementation

• maintenance effort per shared Skill invocation) We add model and tool costs when comparing against deterministic execution: text Incremental agent cost per run = model input cost

• model output cost
• tool execution cost
• expected retry cost

• displaced manual routing cost We should not choose Skills on the assumption that they are free because they are files. The files are free to store. Skill discovery still happens during an agent run, and a badly written Skill can cause retries or unnecessary tool calls.

We would not use a Skill to wrap a function that can be selected with a reliable if statement. We would use a Skill when interpreting a natural-language request requires context and judgment. It also suits teams that need to maintain the same operating instructions across multiple repositories.

A worker image is a packaged environment for the process running the agent. At scale, we would include the required software in it rather than installing python-pptx during a request. We would create one isolated workspace per job, copy or mount a versioned Skill bundle, and destroy the workspace afterward. Long-lived shared agent home directories risk mixing data and settings across customers, retaining outdated files, and allowing access beyond intended limits. We should therefore choose the hosting setup to enforce the required separation, not just for convenience.

Our Final Verdict: When to Deploy, When to Skip#

Claude Agent Skills worked for our packaging goal, but SKILL.md alone was not enough. We needed the complete tested directory and explicit rules for how the agent runs and what it can access.

Deploy this if:

• You have agent procedures duplicated across repositories or system prompts.
• The agent must choose among several capabilities based on the intent of a natural-language request.
• The capability needs instructions, scripts, references, and assets shipped together.
• You can lock required software to specific versions and version the complete Skill directory.
• Your SDK configuration explicitly enables the intended setting sources.
• You can test discovery with prompts that should trigger the Skill, prompts that should not, and prompts that leave the choice unclear.
• You run generated commands inside a real filesystem and process sandbox.
• You restart or replace workers when Skill metadata or content changes.
• You validate the output artifact independently of the model’s final response.

Hold off or avoid it if:

• A direct function call can route the request deterministically.
• You expect allowed_tools or Skill prose to provide path-level isolation.
• Your hosted workers share writable homes across customers or trust boundaries.
• You cannot identify which Skill release handled a run.
• You need edits to propagate instantly into active conversations.
• Your CI checks only whether SKILL.md exists and never exercises discovery.
• Your package relies on programs or credentials already present on the host, or on libraries without fixed versions.
• A failed tool invocation would create an irreversible external side effect.

Our production baseline starts with uppercase SKILL.md directly inside each Skill directory and a concise description of when to use it. Scripts locate supporting files relative to their own location, and the SDK explicitly loads project settings. Each run gets only the tools it needs and a temporary workspace. We keep release bundles unchanged after publishing and test the generated files.

With those controls, Skills are a useful reuse layer for Claude-based agents. Without them, they fail in ways that look deceptively normal: the agent continues talking, chooses an improvised path, or runs with more authority than the package needs. That is the real shipping risk. The format is simple; the runtime surrounding it is not.