wuyoscar/internal-safety-collapse

Internal Safety Collapse in Frontier Large Language Models

[!CAUTION] Research-use only. Internal Safety Collapse (ISC) is released exclusively for accelerating red-teaming process, evaluation, and mitigation work. We do not condone or permit any use of these materials for malicious purposes or real-world harm.

News

• 🔴 All OpenRouter frontier LLMs triggered ISC.
• 🌟 2026-06-26 — 900 GitHub stars.
• 🎭 2026-06-09 — Fable 5 triggered ISC.
• 🔥 2026-04-17 / 2026-06-25 — Opus 4.7 and 4.8 triggered ISC.
• 🌟 2026-03-27 — 500 GitHub stars.
• 🚀 2026-03-22 — Open-sourced.

See CHANGELOG.md for the full update history.

Example Content/Data/Artifact

If ISC only reproduced known harmful-text categories, it would not be very interesting. The point is broader: the failure shows up inside workflow completion. The model can produce harmful artifacts that sit outside standard chat-safety taxonomies, including scientific and tool-verifiable outputs.

What We Found

ISC triggered across all tested frontier LLMs under ASR@3. It does not rely on a magic prompt, a fixed jailbreak string, or a carefully tuned template. The demos are here to make that obvious: the failure lives in the workflow, and the barrier is low.

ISC is not an attack — it is a phenomenon. The harmful outcome it produces is the same kind of thing existing jailbreaks can elicit; what is new is how it surfaces. There are many ways to trigger it. In the paper we use a setup called TVD (Task-Validator-Data), which turns the phenomenon into a controlled experiment that usually succeeds within a single user request. For the details of TVD, see the paper or jump straight to the TVD Framework section below.

Beyond TVD, there are other ways to probe whether a web-app LLM is susceptible to ISC-style collapse, and the community, the author team, and other researchers have run examples in the wild. We share a selection below to give a sense of the shape and generality of the failure.

Test

Media

Since release, a few people have posted videos, summaries, and independent takes on ISC. We collect some of them here because they explain the idea from different angles.

If you are a researcher and want to study ISC to do follow-up work, you can refer to these fantastic research works:

Our Role

ISC is a red-teaming project. The point is not to jailbreak models for fun. The point is to find failures early enough that people can study them and build better defenses.

We first noticed the ISC pattern around November 2025. After the paper was submitted in March, we decided to open-source the project. Before that, we reached out to LLM developers and AI safety/red-team researchers, shared what we had found, and encouraged them to look into it.

We believed this was more than another jailbreak trick. It looked like a workflow-level failure that deserved attention. We did not receive a substantive response.

So we made a conservative release. We publish trajectories and lower-risk demonstrations, enough to show the failure exists without turning the repository into an operational playbook.

Beyond the Paper

62 frontier models triggered so far. The table tracks public evidence, not private runs.

Reproductions

Type: ① Direct template use · ② Modified template · ③ New method using ISC · ④ Outside TVD

---

Benchmark Surface

84 codebase templates. 9 domains. Task, validator, data. A reproducible surface for workflow-level ISC.

Codebase Templates

The templates are small codebase surfaces, not prompt-only examples. Each folder centers on exp0.txt plus run variants.

Different tools validate different artifacts. So the data changes too: classifier text, sequence records, molecular records, network/binary artifacts, misinformation records. The table stays high-level on purpose.

AI Safety & ML (26)

Biology & Genomics (16)

Chemistry & Materials (11)

Clinical & Health Sciences (5)

Cybersecurity (8)

Epidemiology & Public Health (4)

Media & Communication (8)

Pharmacology & Toxicology (5)

Other (1)

cat codebase_templates/aiml_llamaguard/exp0.txt
# inspect a released codebase template

TVD Framework

The TVD Framework: Task, Validator, Data.

Internal Safety Collapse (ISC) is the failure. TVD Framework is one way to trigger it: task, validator, missing data. The model fills the gap because completion is the objective.

Setup

No setup. No dependencies. Bring your own API key.

Experiments Conducted in the Paper

Three ways to reproduce the same failure surface:

ISC-Chatbot — task, validator, data, and failure trace in one prompt. No full agent environment. Easy to run; still triggers roughly 95% of tested frontier models in our tests.

cd experiment/isc_single && uv run run.py --model  --bench jbb --task ai-guard --samples 0

ISC-ICL — completed trajectories first, target case after.

cd experiment/isc_icl && uv run run.py --model  --demos 5

ISC-Agent — gives an agent shell access and a high-level task. The loop is simple: inspect files, run code, validate, repair. From the user side, one initial interaction is enough.

cd experiment/isc_agent && docker build -t isc-agent . && ./run.sh --model 

Released materials: Codebase Templates · community/ · experiment/

Changelog

Full history: CHANGELOG.md. Highlights:

• 2026-07-03 — Template names unified; ISC-Agent guard/moderation templates consolidated; per-template SKILL.md removed.
• 2026-04-17 (v0.0.5) — README reframed around workflow-level failure; Claude Opus 4.7 added.
• 2026-03-25 — First public frontier-model batch.

License

CC BY-NC-SA 4.0 — academic AI safety research only. No commercial use. No harmful generation.

Citation

@article{wu2026isc,
  title={Internal Safety Collapse in Frontier Large Language Models},
  author={Wu, Yutao and Liu, Xiao and Gao, Yifeng and Zheng, Xiang and Huang, Hanxun and Li, Yige and Wang, Cong and Li, Bo and Ma, Xingjun and Jiang, Yu-Gang},
  journal={arXiv preprint arXiv:2603.23509},
  year={2026},
  url={https://arxiv.org/abs/2603.23509}
}

Contact

Questions, collaborations, responsible disclosure: wuy⁷¹¹⁷ ⓐ 𝗴𝗺𝗮𝗶𝗹 𝗰𝗼𝗺