HITRUST CSF requirement statement [?] (New in v11.4.0, coming in Nov. 2024)
Documentation of the overall AI system discusses the creation, operation, and lifecycle
management of any
(1) models,
(2) datasets (including data used for training, tuning, and prompt enhancement via RAG),
(3) configurations (e.g., metaprompts),
(4) language model tools such as agents and plugins
maintained by the organization, as applicable.
Documentation of the overall AI system also describes the
(5) tooling resources (e.g., AI platforms, model engineering environments),
(6) system and computing resources, and
(7) human resources
needed for the development and operation of the AI system.
- Evaluative elements in this requirement statement [?]
-
1. Documentation of the overall AI system discusses the creation, operation, and lifecycle management of any models maintained by the organization, as applicable.
2. Documentation of the overall AI system discusses the creation, operation, and lifecycle management of any datasets maintained by the organization (including data used for training, tuning, and prompt enhancement via RAG), as applicable.
3. Documentation of the overall AI system discusses the creation, operation, and lifecycle management of any configurations (e.g., metaprompts) maintained by the organization, as applicable.
4. Documentation of the overall AI system discusses the creation, operation, and lifecycle management of any language model tools such as agents and plugins maintained by the organization, as applicable.
5. Documentation of the overall AI system describes the tooling resources (e.g., AI platforms, model engineering environments) needed for the development and operation of the AI system.
6. Documentation of the overall AI system describes the system and computing resources needed for the development and operation of the AI system.
7. Documentation of the overall AI system describes the human resources needed for the development and operation of the AI system.
- Illustrative procedures for use during assessments [?]
- Policy: Examine policies related to each evaluative element within the requirement statement. Validate the existence of a written or undocumented policy as defined in the HITRUST scoring rubric.
- Procedure: Examine evidence that written or undocumented procedures exist as defined in the HITRUST scoring rubric. Determine if the procedures and address the operational aspects of how to perform each evaluative element within the requirement statement.
- Implemented: Examine evidence that all evaluative elements within the requirement statement have been implemented as defined in the HITRUST scoring rubric, using a sample based test where possible for each evaluative element. Example test(s):
- For example, review the overall AI system documentation to confirm it discusses the creation, operation, and lifecycle management of any models, datasets (including data used for training, tuning, and prompt enhancement via RAG), configurations (e.g., metaprompts), language model tools such as agents and plugins maintained by the organization, as applicable. Further, confirm the documentation of the overall AI system also describes the tooling resources (e.g., AI platforms, model engineering environments), system and computing resources, and human resources needed for the development and operation of the AI system.
- For example, review the overall AI system documentation to confirm it discusses the creation, operation, and lifecycle management of any models, datasets (including data used for training, tuning, and prompt enhancement via RAG), configurations (e.g., metaprompts), language model tools such as agents and plugins maintained by the organization, as applicable. Further, confirm the documentation of the overall AI system also describes the tooling resources (e.g., AI platforms, model engineering environments), system and computing resources, and human resources needed for the development and operation of the AI system.
- Measured: Examine measurements that formally evaluate and communicate the operation and/or performance of each evaluative element within the requirement statement. Determine the percentage of evaluative elements addressed by the organization’s operational and/or independent measure(s) or metric(s) as defined in the HITRUST scoring rubric. Determine if the measurements include independent and/or operational measure(s) or metric(s) as defined in the HITRUST scoring rubric. Example test(s):
- For example, measures indicate completeness of the organization’s overall AI system documentation that discusses the required elements in the requirement statement. Reviews, tests, or audits are completed by the organization to measure the effectiveness of the implemented controls and to confirm that all the required elements are included in the overall AI system documentation.
- For example, measures indicate completeness of the organization’s overall AI system documentation that discusses the required elements in the requirement statement. Reviews, tests, or audits are completed by the organization to measure the effectiveness of the implemented controls and to confirm that all the required elements are included in the overall AI system documentation.
- Managed: Examine evidence that a written or undocumented risk treatment process exists, as defined in the HITRUST scoring rubric. Determine the frequency that the risk treatment process was applied to issues identified for each evaluative element within the requirement statement.
- Policy: Examine policies related to each evaluative element within the requirement statement. Validate the existence of a written or undocumented policy as defined in the HITRUST scoring rubric.
- Placement of this requirement in the HITRUST CSF [?]
- Assessment domain: 06 Configuration Management
- Control category: 10.0 – Information Systems Acquisition, Development, and Maintenance
- Control reference: 10h- Control of Operational Software
- Specific to which parts of the overall AI system? [?]
-
AI application layer:
- AI plugins and agents
- Prompt augmentations (e.g., via RAG) and associated data sources
- Application AI safety and security systems
- The deployed AI application (Considered in the associated HITRUST e1, i1, or r2 assessment)
- The AI platform and associated APIs (Considered in the associated HITRUST e1, i1, or r2 assessment)
- Model safety and security systems
- Model tuning and associated datasets
- The deployed AI model
- Model engineering environment and model pipeline
- AI datasets and data pipelines
- Discussed in which authoritative AI security sources? [?]
-
- ISO/IEC 42001:2023 Information technology — Artificial intelligence — Management system
2023, © International Standards Organization (ISO)/International Electrotechnical Commission (IEC)- Where: Annex A > A.4. Resources for AI systems > A.4. Resources for AI systems (all items)
- Where: Annex A > A.4. Resources for AI systems > A.4. Resources for AI systems (all items)
- OWASP AI Exchange
2024, © The OWASP Foundation- Where: #DEVPROGRAM
- Where: #DEVPROGRAM
- Guidelines for Secure AI System Development
Nov. 2023, Cybersecurity & Infrastructure Security Agency (CISA)- Where: 2. Secure development > Document your data, models and prompts
- Where: 2. Secure development > Document your data, models and prompts
- Generative AI framework for HM Government
2023, Central Digital and Data Office, UK Government- Where: Building generative AI solutions > Building the solution > Data Management > Bullet 2
- Where: Building generative AI solutions > Building the solution > Data Management > Bullet 2
- Securing Machine Learning Algorithms
2021, © European Union Agency for Cybersecurity (ENISA)- Where:
- 4.1- Security Controls > Technical > Ensure ML projects follow the global process for integrating security into projects
- 4.1- Security Controls > Organizational > Apply documentation requirements to AI projects
- Where:
- ISO/IEC 42001:2023 Information technology — Artificial intelligence — Management system
- Discussed in which commercial AI security sources? [?]
-
- Snowflake AI Security Framework
2024, © Snowflake Inc.- Where: Lack of explainability / transparency > Mitigations > Model transparency and documentation
- Where: Lack of explainability / transparency > Mitigations > Model transparency and documentation
- Snowflake AI Security Framework
- Helps to prevent, detect, and/or correct which AI security threats? [?]
- None directly (this is a variance reduction requirement)
- None directly (this is a variance reduction requirement)
- Additional information
- Q: When will this requirement included in an assessment? [?]
- This requirement will always be added to HITRUST assessments which include the
Cybersecurity for deployed AI systems
regulatory factor. - No other assessment tailoring factors affect this requirement.
- This requirement will always be added to HITRUST assessments which include the
- Q: When will this requirement included in an assessment? [?]
Post your comment on this topic.