Skip to main content

Pro audit program · v1.0

AI Model Risk & Governance

Whether you train, fine-tune or just consume models, you need governance — inventory, risk classification, human oversight, evaluation.

  • General target area
  • NIST AI RMF / ISO 42001 framework
  • 8 controls in this program
  • Cyentrix Cyentrix Trusted Author
Sign in to run → Back to library

About this program

Whether you train, fine-tune or just consume models, you need governance — inventory, risk classification, human oversight, evaluation.

Risks addressed

  • High Shadow AI: models used by teams with no oversight
  • High Bias / unfair outcomes harming customers + the brand
  • Medium Model drift goes unnoticed in production
  • Critical High-risk use-case (e.g. EU AI Act) without conformity

Controls (8)

  1. AI use-case inventory

    High

    This control requires organizations to maintain a comprehensive, current inventory of all artificial intelligence and machine learning use cases deployed across the enterprise, including both internally developed and third-party AI systems. The inventory captures key metadata such as system purpose,…

    How to test + evidence

    Testing procedure: Register every AI use-case (internal + customer-facing) with owner + risk tier.

    Evidence to collect: Inventory document.

  2. Risk classification per use-case

    High

    Risk classification per use-case requires organizations to categorize each AI system, application, or data-processing activity based on its potential impact to confidentiality, integrity, availability, and compliance obligations. Classifications drive tailored security controls, testing rigor, access restrictions, and monitoring thresholds appropriate…

    How to test + evidence

    Testing procedure: Each use-case classified (low / limited / high / unacceptable per EU AI Act + your taxonomy).

    Evidence to collect: Classification register.

  3. Approved-model list + procurement gate

    High

    This control establishes and enforces a centrally maintained list of pre-approved AI models (including large language models, machine learning frameworks, and generative AI tools) that have undergone security, privacy, and compliance vetting. Organizations implement a procurement gate requiring all AI…

    How to test + evidence

    Testing procedure: Models / providers approved by Security + Legal before use; vendor DPA reviewed.

    Evidence to collect: Approved list + procurement workflow.

  4. Human oversight for high-risk decisions

    Critical

    This control mandates that automated systems, including artificial intelligence and machine learning models, cannot independently execute decisions classified as high-risk without human review and approval. High-risk decisions typically include actions affecting safety, legal compliance, financial liability exceeding thresholds, access to…

    How to test + evidence

    Testing procedure: Decisions affecting customers (credit, hiring, etc.) reviewable by a human; not fully automated.

    Evidence to collect: Process flow + sample reviews.

  5. Bias + fairness testing pre-launch

    High

    Bias and fairness testing pre-launch requires organizations to evaluate AI/ML models for discriminatory outcomes, unintended bias, and fairness issues across protected characteristics (race, gender, age, disability, etc.) before deploying systems into production. Testing involves statistical analysis of model outputs across…

    How to test + evidence

    Testing procedure: Bias evaluation performed before launch + at every material model update.

    Evidence to collect: Eval reports.

  6. Drift + accuracy monitoring

    Medium

    Drift and accuracy monitoring detects unintended changes to deployed configurations, code, infrastructure, or AI/ML model behavior over time. This control establishes automated comparison mechanisms that measure deviations from approved baselines—such as configuration files, security policies, model prediction accuracy, or system…

    How to test + evidence

    Testing procedure: Production metrics tracked + alerted on degradation.

    Evidence to collect: Monitoring dashboard.

  7. Documented model cards / system cards

    Medium

    Model cards and system cards are structured documentation artifacts that describe the capabilities, limitations, training data, performance characteristics, intended use cases, and potential biases of AI/ML systems. Model cards focus on individual models, while system cards encompass broader AI-enabled systems…

    How to test + evidence

    Testing procedure: Each deployed model has a card: intended use, limits, data, evals, owner.

    Evidence to collect: Card repository.

  8. User notification + opt-out where required

    Medium

    This control requires organizations to provide clear, timely notice to users (customers, employees, or data subjects) about data collection, processing, and monitoring activities, and to offer an opt-out mechanism where legally or contractually mandated. Notifications must be presented before or…

    How to test + evidence

    Testing procedure: Users informed when interacting with AI; opt-out path for synthetic content.

    Evidence to collect: Notice + opt-out flow.