The Hidden Flaw in AI Security Frameworks
In recent years, the world of artificial intelligence (AI) has been swept up in a flurry of regulatory activity, with governments and industry leaders scrambling to establish frameworks for ensuring the security and accountability of these powerful systems. The EU AI Act, NIST’s AI Risk Management Framework, and ISO/IEC 42001 are just a few examples of the many guidelines now being used to evaluate an organization’s approach to AI. But despite their best efforts, it seems that something fundamental is missing from these frameworks: a clear and practical way to assess the security of AI systems.
The problem lies not with the frameworks themselves, but rather in how they are applied downstream. When organizations attempt to translate the broad strokes of these guidelines into concrete questions for vendors or internal teams, the results are often disastrous. Hundreds of open-ended questions, many of which ask for vague descriptions or attestations, dominate questionnaires and audits. These types of questions not only fail to provide meaningful insights but also create a culture of creative writing and compliance by proxy.
The issue is twofold. Firstly, these questions rely on prose rather than evidence, making it impossible to separate genuine security efforts from those that are merely cosmetic. A vendor with a well-documented program can produce answers that sound just as convincing as one with real substance, simply because they have the technical writers and resources to spin a good story. This approach rewards clever wordplay over actual security measures.
Secondly, these questions often ignore the fundamental nature of AI systems themselves. Large Language Models (LLMs), for example, are stochastic systems that defy easy explanation or prediction. Asking if an LLM produces biased outputs is akin to asking if a car’s engine runs smoothly – it’s far too simplistic and doesn’t account for the complexity of these systems.
Furthermore, current questionnaires rarely scale with risk. The same set of 300 questions is often used regardless of the system’s actual impact or sensitivity level. This approach is not only inefficient but also fundamentally flawed, as it suggests that all AI systems carry equal levels of risk and consequence.
Instead of relying on these flawed questionnaires, we should adopt a more practical approach to assessing AI security. Drawing from the success of checklists in fields like medicine and aviation, we can establish a clear bar for what constitutes effective AI compliance. Every question used in an assessment or questionnaire should be designed with five key criteria in mind:
Firstly, it should be answerable with evidence – concrete artifacts that demonstrate real practices, not just words on paper.
Secondly, it should be scoped to the actual risk tier of the system being assessed. This means asking fewer questions for lower-risk systems and more detailed ones for higher-risk applications.
Thirdly, it should avoid open-ended prompts in favor of specific requirements or evidence-based inquiries.
By adopting this checklist approach, we can create a more effective and efficient system for evaluating AI security. It’s time to move beyond vague attestations and creative writing exercises and towards a more practical, risk-driven methodology for ensuring the safety and accountability of these powerful technologies.
Source: SecurityWeek — 2026-07-30