**A New Approach to AI Safety: Peeking Inside the ‘Black Box’**
Researchers have long acknowledged that relying solely on analyzing the inputs and outputs of large language models (LLMs) can be insufficient for detecting malicious activity. Despite the growing number of LLMs being used in various applications, their “black box” nature has made it challenging to identify when an AI system may take an unwanted action. A group of researchers from Ben-Gurion University of The Negev is proposing a novel approach that treats the LLM as an open system, focusing on identifying specific cognitive elements within the model’s internal workings.
This new framework, known as Governance via Activation-based Verification and Extensible Logic (GAVEL), aims to create a standardized set of rules for detecting safety events. Unlike traditional approaches, which often rely on labeling certain activation distributions as “cybercrime” or “hate speech,” GAVEL uses a more granular scheme of cognitive elements that can be combined in logical statements. By doing so, the system can detect specific types of safety events, similar to how Snort or YARA rulesets work.
The idea is to create an open system where identified cognitive elements and rules can be combined to detect potential threats. This would allow practitioners to focus on high-level decisions about what a model should or shouldn’t do, rather than dealing with the complex math and processes that occur at the low level of the neural network. “What we really tried to do here is make that separation,” explains Yisroel Mirsky, an assistant professor and head of the Offensive AI Research Lab at Ben-Gurion University. “The practitioner wants [just to deal with] high-level stuff — what the model should and shouldn’t do.”
Current attempts to secure AI systems have been largely ineffective against sophisticated attackers. By using different language or finding vulnerabilities, malicious users can evade traditional content-based guardrails. The GAVEL system addresses this limitation by instrumenting the model and detecting activation patterns that may indicate unwanted behavior. This approach is more akin to neurobiological research, which illuminates the mechanistic workings of the brain by recording activations across neural networks.
While the current efforts have mapped misuse through broad datasets, resulting in a high number of false positives and reduced nuance in detection rules, GAVEL’s activation analysis aims to associate specific patterns with more granular objects and predicates. This would enable more precise detection and reduce the risk of misclassification.
The development of GAVEL is a significant step towards creating a more robust AI safety framework. By treating LLMs as open systems and focusing on internal workings, researchers can create a more effective defense against malicious activity. As AI continues to play an increasingly prominent role in various applications, it’s essential to develop more sophisticated security measures that can keep pace with emerging threats.
**Practical Takeaway**
The GAVEL system offers a promising approach to AI safety by focusing on internal workings of LLMs rather than just analyzing inputs and outputs. To implement this approach effectively, organizations should consider the following:
* Instrumenting models to detect activation patterns and associate them with specific cognitive elements
* Developing standardized rules for detecting safety events using a granular scheme of cognitive elements
* Focusing on high-level decisions about what a model should or shouldn’t do, rather than dealing with complex math and processes
By adopting this approach, organizations can create more effective defenses against malicious activity and ensure the safe deployment of AI systems in various applications.
Source: Dark Reading — 2026-07-28