Research
How Detonator performs
What we have measured so far, how we measured it, and what it does not prove yet. Every number comes from the benchmark and test suite in our repository.
Benchmark · agent skills
Dormant skills: what a single run misses
Malicious skills often stay quiet until conditions match: a CI pipeline, cloud credentials, a deploy argument, a date, a long wait, or an environment variable with one specific value. We wrote 14 malicious and 4 benign skills, one per technique, and compared a single plain run with Detonator's explorer.
The explorer re-runs a skill under CI, cloud, production and argument variants, reads the values its code compares environment variables against (statically and by hooking reads at runtime), and moves the clock 45 days ahead with sleeps running a thousand times faster. Catches that only exploration made: CI-only and cloud-only theft, a deploy-argument trigger, an obfuscated variable name built at runtime, a date-gated payload and a six-hour sleep.
Limits. We wrote these samples ourselves, so they prove the mechanism, not real-world coverage. That is what the in-the-wild study below is for.
| Single run | Detonator explorer | |
|---|---|---|
| Malicious skills caught | 6 of 14 | 14 of 14 |
| False alarms on benign skills | 0 of 4 | 0 of 4 |
Benchmark · MCP servers
MCP servers: install hooks, rug pulls, time bombs
Nine MCP servers, two benign and seven malicious, each speaking the real protocol. Detonator installs them with scripts disabled, runs the scripts inside the sandbox offline, calls every tool four times with schema-valid arguments, re-lists tools between rounds and repeats under time travel and CI conditions.
The poisoned description only becomes harmful when an agent obeys it. With a model attached, Detonator gives it the server's tools next to decoy filesystem, Slack and fetch servers. The agent read a planted SSH key through the decoy and passed it into the tool; a control run with neutral descriptions did not, which attributes the behavior to the server.
Limits. The agent result above uses a scripted test model; runs with production models are part of the next study.
| Attack | Single run | Explorer |
|---|---|---|
| Credential theft in an install script | caught | caught |
| Theft at startup, before any tool call | caught | caught |
| Theft when a tool is used | caught | caught |
| Tool definitions swapped after four calls (rug pull) | caught | caught |
| Theft only in CI | missed | caught |
| Theft one hour after startup | missed | caught |
| Poisoned tool description, harmless code | suspicious | malicious with an agent |
Field test · false positives
False alarms on 117 real public skills
A detector that cries wolf gets switched off. We ran every skill in Anthropic's and Trail of Bits' public skill repositories, all expected to be benign, including a real AI model following each skill.
The one real finding: a reputable setup skill switches the coding agent to skip all permission prompts. It is documented and meant for throwaway containers, but risky on a developer's own machine, so Detonator reports it as Suspicious with that explanation rather than as malware.
Limits. 117 skills from two careful publishers is a clean baseline, not a random sample of the ecosystem.
| Pass | Flagged | Cause |
|---|---|---|
| First pass, no model | 10 | temp files, git's sample hooks, gh reading its own login |
| With the model | 20 | gaps in our own sandbox: a crash reporter writing to the host, the engine's folder reachable |
| After fixes | 3 | two "can't run here" replies read as refusals; one real finding (below) |
| Now | 0 false alarms | every cause fixed and covered by a test |
In progress
Next: in the wild
- Real malware. 703 malicious skills collected from the wild, detonated on isolated infrastructure.
- The marketplaces. Skills and MCP servers from public registries and marketplaces, scanned and published to the database.
- Head-to-head. The same samples through popular static scanners and through Detonator, with the misses on both sides published.
Malicious packages found in public registries are reported to the registry before we publish details. We share hashes and indicators, never live samples.