HITRUST CSF requirement statement [?] (07.07aAISecOrganizational.4)
The organization maintains a catalog of trusted data sources for use in
(1) training, testing, and validating AI models;
(2) fine-tuning AI models; and
(3) enhancing AI prompts via RAG, as applicable.
This catalog is
(4) periodically (at least semiannually) reviewed and updated.- Evaluative elements in this requirement statement [?]
-
1. The organization maintains a catalog of trusted data sources for use in training, testing, and validating AI models, as applicable.2. The organization maintains a catalog of trusted data sources for use in fine-tuning AI models, as applicable.3. The organization maintains a catalog of trusted data sources for use in enhancing AI prompts via RAG, as applicable.4. The organization's catalog of trusted data sources for use in AI is periodically (at least semiannually) reviewed and updated.in enhancing AI prompts via RAG, as applicable.
- Illustrative procedures for use during assessments [?]
- Policy: Examine policies related to each evaluative element within the requirement statement. Validate the existence of a written or undocumented policy as defined in the HITRUST scoring rubric.
- Procedure: Examine evidence that written or undocumented procedures exist as defined in the HITRUST scoring rubric. Determine if the procedures and address the operational aspects of how to perform each evaluative element within the requirement statement.
- Implemented: Examine evidence that all evaluative elements within the requirement statement have been implemented as defined in the HITRUST scoring rubric, using a sample based test where possible for each evaluative element. Example test(s):
- For example, evidence that the organization maintains a catalog of trusted data sources for use in training, testing, and validating AI models; fine-tuning AI models; and enhancing AI prompts via RAG, as applicable.
- For example, evidence that the organization maintains a catalog of trusted data sources for use in training, testing, and validating AI models; fine-tuning AI models; and enhancing AI prompts via RAG, as applicable.
- Measured: Examine measurements that formally evaluate and communicate the operation and/or performance of each evaluative element within the requirement statement. Determine the percentage of evaluative elements addressed by the organization’s operational and/or independent measure(s) or metric(s) as defined in the HITRUST scoring rubric. Determine if the measurements include independent and/or operational measure(s) or metric(s) as defined in the HITRUST scoring rubric. Example test(s):
- For example, measures indicate if the organization maintains a catalog of trusted data sources for use in training, testing, and validating AI models; tuning AI models; and enhancing AI prompts via RAG, as applicable.
- For example, measures indicate if the organization maintains a catalog of trusted data sources for use in training, testing, and validating AI models; tuning AI models; and enhancing AI prompts via RAG, as applicable.
- Managed: Examine evidence that a written or undocumented risk treatment process exists, as defined in the HITRUST scoring rubric. Determine the frequency that the risk treatment process was applied to issues identified for each evaluative element within the requirement statement.
- Policy: Examine policies related to each evaluative element within the requirement statement. Validate the existence of a written or undocumented policy as defined in the HITRUST scoring rubric.
- Placement of this requirement in the HITRUST CSF [?]
- Assessment domain: 07 Vulnerability Management
- Control category: 07.0 – Asset Management
- Control reference: 07.a – Inventory of Assets
- Specific to which parts of the overall AI system? [?]
-
AI application layer:
- Prompt enhancement via RAG, and associated RAG data sources
- Model tuning and associated datasets
- AI datasets and data pipelines
- Discussed in which authoritative AI security sources? [?]
-
- OWASP Machine Learning Security Top 10
2023, © The OWASP Foundation- Where:
- ML07:2023 Transfer learning attack > How to prevent > Bullet #2
- ML07:2023 Transfer learning attack > How to prevent > Bullet #2
- Where:
- Deploying AI Systems Securely: Best Practices for Deploying Secure and Resilient AI Systems
Apr 2024, National Security Agency (NSA)- Where:
- Secure the deployment environment > Ensure a robust deployment environment architecture > Bullet #3
- Secure the deployment environment > Ensure a robust deployment environment architecture > Bullet #3
- Where:
- Securing Machine Learning Algorithms
2021, © European Union Agency for Cybersecurity (ENISA)- Where:
- 4.1- Security Controls > Technical > Control all data used by the ML model
- 4.1- Security Controls > Technical > Control all data used by the ML model
- Where:
- OWASP Machine Learning Security Top 10
- Discussed in which commercial AI security sources? [?]
-
- Databricks AI Security Framework
Sept. 2024, © Databricks- Where:
- Control DASF 11: Capture and view data lineage
- Control DASF 17: Track and reproduce the training data used for ML model training
- Where:
- Snowflake AI Security Framework
2024, © Snowflake Inc.- Where:
- Indirect prompt injection > Mitigations > Sources validation and filtering
- Indirect prompt injection > Mitigations > Sources validation and filtering
- Where:
- Databricks AI Security Framework
- Control functions against which AI security threats? [?]
-
- Control function: Variance reduction
- Additional information
-
- Q: When will this requirement included in an assessment? [?]
- This requirement is included when the assessment’s in-scope AI system(s) leverage data-driven AI models (e.g., non-generative machine learning models, generative AI models).
- The
Security for AI systemsregulatory factor must also be present in the assessment.
- Q: When will this requirement included in an assessment? [?]