Cybersecurity researchers have uncovered a sophisticated attack technique that allows malicious actors to bypass safety guardrails in popular AI-powered chat interfaces, potentially exposing users’ sensitive data. The method, dubbed “Cryptographic Context Injection,” exploits the way these systems handle encrypted prompts.
The discovery was made by Adversa AI, which reported its findings to xAI on June 3 but has yet to receive a response. Despite being unable to disclose their research to Google due to the company’s vulnerability disclosure program limitations, the researchers have chosen to go public with their findings and potential defensive solutions.
The attack works by injecting encrypted prompts into the AI model’s code execution sandbox. These prompts are then decrypted within the trusted execution context, allowing them to bypass safety guardrails that would normally flag them as malicious. The resulting plaintext prompt can instruct the model to perform various unwanted actions, such as leaking user data or producing restricted content.
One of the most concerning aspects of this attack is its potential for social engineering. In an example targeting xAI’s Grok web chat framework, a user is persuaded to examine a weaponized web page containing an encrypted JSON object and decryption instructions. The resulting plaintext prompt then instructs the agent to resolve private session metadata and conversation history, transmitting sensitive data to an attacker-controlled URL.
Similarly, in another example targeting Google’s Gemini public chat interface, a single prompt instructs Gemini to run a Python script that decrypts supplied ciphertext. The decrypted prompt can then produce restricted content, “framed as something it will encrypt ‘for safety,'” thereby evading safety filters.
The researchers warn that this attack technique is particularly insidious because the attacker payload inherits credibility from the encrypted context. If pasted directly into the prompt, the same text would likely be flagged as malicious, but when delivered as a ciphertext, it gains trust within the model’s execution sandbox.
While the success rate of this attack against Gemini has decreased since its discovery, Adversa AI notes that it remains potentially possible. The researchers’ report emphasizes the need for defenders to take preventive measures, including implementing additional security checks and regularly updating their systems with the latest filter updates and model versions.
Ultimately, this vulnerability highlights the ongoing cat-and-mouse game between AI-powered chat interfaces and malicious actors seeking to exploit their weaknesses. As these systems continue to evolve and become increasingly integrated into our daily lives, it is essential for developers and users alike to prioritize security and vigilance in the face of emerging threats like Cryptographic Context Injection.
For those concerned about protecting themselves from this type of attack, Adversa AI’s report provides practical advice on how to implement additional security measures. Specifically, defenders are advised to monitor their systems closely for any suspicious activity, regularly update their filter updates and model versions, and consider implementing additional security checks to detect and prevent Cryptographic Context Injection attacks. By staying informed and proactive, users can help mitigate the risks associated with these sophisticated threats.
Source: SecurityWeek — 2026-08-21