HITRUST CSF requirement statement [?] (Updated version of 19.09zATLASOrganizational.2 in v11.4.0, coming in Nov. 2024)
When using a confidential and/or closed-source AI model, the organization restricts the
release of technical AI project details, including specifics on AI
(1) datasets used for model training, testing, validating, tuning, and prompt augmentation
via RAG;
(2) algorithm(s) used;
(3) model architecture;
(4) language model tools used such as agents and plugins;
(5) safety and security checkpoints; and
(6) information on teams developing and supporting the AI system.
- Evaluative elements in this requirement statement [?]
-
1. When using a confidential and/or closed-source AI model, the organization restricts the release of technical AI specifics on AI datasets used for model training, testing, validating, tuning, and prompt augmentation via RAG.
2. When using a confidential and/or closed-source AI model, the organization restricts the release of technical specifics of algorithm(s) used.
3. When using a confidential and/or closed-source AI model, the organization restricts the release of technical specifics of AI model architecture.
4. When using a confidential and/or closed-source AI model, the organization restricts the release of technical specifics of language model tools used such as agents and plugins.
5. When using a confidential and/or closed-source AI model, the organization restricts the release of technical AI safety and security checkpoints.
6. When using a confidential and/or closed-source AI model, the organization restricts the release of teams developing and supporting the AI system.
- Illustrative procedures for use during assessments [?]
- Policy: Examine policies related to each evaluative element within the requirement statement. Validate the existence of a written or undocumented policy as defined in the HITRUST scoring rubric.
- Procedure: Examine evidence that written or undocumented procedures exist as defined in the HITRUST scoring rubric. Determine if the procedures and address the operational aspects of how to perform each evaluative element within the requirement statement.
- Implemented: Examine evidence that all evaluative elements within the requirement statement have been implemented as defined in the HITRUST scoring rubric, using a sample based test where possible for each evaluative element. Example test(s):
- For example, evaluate the AI model to ensure the organization restricts the release of technical AI project details when utilizing a confidential and/or closed-source AI model. This includes specifics on AI datasets used for model training, testing, validating, tuning, and prompt augmentation via RAG; algorithms employed; model architecture; language model tools such as agents and plugins; safety and security checkpoints; and information about the teams developing and supporting the AI system.
- For example, evaluate the AI model to ensure the organization restricts the release of technical AI project details when utilizing a confidential and/or closed-source AI model. This includes specifics on AI datasets used for model training, testing, validating, tuning, and prompt augmentation via RAG; algorithms employed; model architecture; language model tools such as agents and plugins; safety and security checkpoints; and information about the teams developing and supporting the AI system.
- Measured: Examine measurements that formally evaluate and communicate the operation and/or performance of each evaluative element within the requirement statement. Determine the percentage of evaluative elements addressed by the organization’s operational and/or independent measure(s) or metric(s) as defined in the HITRUST scoring rubric. Determine if the measurements include independent and/or operational measure(s) or metric(s) as defined in the HITRUST scoring rubric. Example test(s):
- For example, measures indicate if the organization restricts the release of technical AI project details when utilizing a confidential and/or closed-source AI model. Reviews, tests, or audits are completed by the organization to measure the effectiveness of the implemented controls and to confirm that the restrictions include specifics on AI datasets used for model training, testing, validating, tuning, and prompt augmentation via RAG; algorithms employed; model architecture; language model tools such as agents and plugins; safety and security checkpoints; and information about the teams developing and supporting the AI system.
- For example, measures indicate if the organization restricts the release of technical AI project details when utilizing a confidential and/or closed-source AI model. Reviews, tests, or audits are completed by the organization to measure the effectiveness of the implemented controls and to confirm that the restrictions include specifics on AI datasets used for model training, testing, validating, tuning, and prompt augmentation via RAG; algorithms employed; model architecture; language model tools such as agents and plugins; safety and security checkpoints; and information about the teams developing and supporting the AI system.
- Managed: Examine evidence that a written or undocumented risk treatment process exists, as defined in the HITRUST scoring rubric. Determine the frequency that the risk treatment process was applied to issues identified for each evaluative element within the requirement statement.
- Policy: Examine policies related to each evaluative element within the requirement statement. Validate the existence of a written or undocumented policy as defined in the HITRUST scoring rubric.
- Placement of this requirement in the HITRUST CSF [?]
- Assessment domain: 19 Data Protection & Privacy
- Control category: 09.0 – Communications and Operations Management
- Control reference: 09.z – Publicly Available Information
- Specific to which parts of the overall AI system? [?]
-
AI application layer:
- AI plugins and agents
- Application AI safety and security systems
- Model tuning and associated datasets
- The deployed AI model
- Model engineering environment and model pipeline
- AI datasets and data pipelines
- Discussed in which authoritative AI security sources? [?]
-
- OWASP AI Exchange
2024, © The OWASP Foundation- Where: #DISCRETE
- Where: #DISCRETE
- MITRE ATLAS
2024, © The MITRE Corporation
- Securing Machine Learning Algorithms
2021, © European Union Agency for Cybersecurity (ENISA)- Where: 4.1- Security Controls > Specific ML > Reduce the available information about the model
- Where: 4.1- Security Controls > Specific ML > Reduce the available information about the model
- OWASP AI Exchange
- Helps to prevent, detect, and/or correct which AI security threats? [?]
-
- Compromised 3rd-party training datasets
- Compromised 3rd-party models or code
- Data poisoning
- Denial of AI service
- Evasion
- Model extraction and theft
- Model inversion
- Model poisoning
- Prompt injection
- Additional information
-
- Q: When will this requirement included in an assessment? [?]
- This requirement is only included when the assessment’s in-scope AI system(s) leverage models with technical architectures that are confidential to the organization.
- The
Cybersecurity for deployed AI systems
regulatory factor must also be present in the assessment.
- Q: When will this requirement included in an assessment? [?]
Post your comment on this topic.