PII / secret redaction before sending to model
Demonstrate that PII and secrets are systematically detected and redacted from all data sent to AI models, preventing sensitive information leakage to third-party services.
Description
What this control does
This control ensures that Personally Identifiable Information (PII), secrets (API keys, passwords, tokens), and other sensitive data are identified and removed or masked from user inputs, prompts, and data payloads before transmission to large language models or third-party AI services. Redaction typically occurs through automated scanning using pattern matching, regular expressions, named entity recognition, or data loss prevention (DLP) libraries integrated into the application layer or API gateway. This control is critical because AI model providers may log, retain, or use submitted data for training purposes, potentially exposing confidential information to unauthorized parties or violating privacy regulations.
Control objective
What auditing this proves
Demonstrate that PII and secrets are systematically detected and redacted from all data sent to AI models, preventing sensitive information leakage to third-party services.
Associated risks
Risks this control addresses
- Unauthorized disclosure of customer PII to AI model providers who may retain or share training data
- Exposure of API keys, OAuth tokens, or cryptographic secrets embedded in prompts leading to account compromise
- Non-compliance with GDPR, CCPA, HIPAA, or other privacy regulations due to uncontrolled PII transmission to third parties
- Inadvertent inclusion of proprietary business data, trade secrets, or confidential financial information in model requests
- Social engineering attacks leveraging PII extracted from model provider logs or data breaches at the provider
- Credential stuffing or lateral movement attacks using secrets harvested from improperly redacted inputs
- Reputational damage and regulatory penalties following public disclosure of sensitive data leakage via AI service logs
Testing procedure
How an auditor verifies this control
- Obtain and review the organization's data classification policy and inventory of PII types and secret formats subject to redaction.
- Review architectural diagrams and data flow documentation showing where redaction occurs in the data pipeline before model invocation.
- Examine configuration files, code repositories, or DLP rule sets defining redaction patterns for PII (email, SSN, phone numbers, names) and secrets (API keys, passwords, tokens).
- Select a representative sample of 20-30 recent production API calls or model requests from application logs or observability platforms.
- Inspect sampled payloads for presence of unredacted PII or secrets by searching for known patterns (e.g., email regex, credit card formats, AWS key prefixes).
- Test the redaction mechanism by submitting synthetic inputs containing known PII and secrets through a non-production environment and verifying complete redaction in captured outbound traffic.
- Review access controls and audit logs for the redaction service or library to confirm only authorized personnel can modify redaction rules.
- Validate exception handling procedures by identifying any documented bypass mechanisms and confirming they require written approval and logging.
Where this control is tested