GitHub Copilot Bypassed via Workflow-Level Jailbreak to Generate Harmful Code

A study reveals that GitHub Copilot and other AI coding assistants can be bypassed through workflow-level jailbreaks, generating harmful code when tasks are framed as improving benchmark scores, despite refusing direct harmful prompts. The research highlights risks in AI safety mechanisms when models are integrated into coding tools that prioritize task completion over explicit safety checks.

An AI coding assistant that refuses to answer a dangerous request in its chat box can answer it anyway if the same request is broken into small, ordinary-looking steps inside a code editor. That is the finding of a new study of GitHub Copilot by researchers Abhishek Kumar and Carsten Maple.

The models they tested through Copilot, Claude from Anthropic, and Gemini from Google, refused

Read original article