Designing an Esoteric Programming Language to Challenge Large Language Models

The article describes the creation of KillSwitch, an esoteric programming language designed to challenge large language models (LLMs) through adversarial features like hidden specifications, character-count dependencies, and safety guardrail triggers. Benchmarking shows limited LLM understanding of the language, with top models achieving only 66.8% accuracy on simple tasks.

Comments

Read original article