HITRUST AI Security Assessment and Certification Specification

Model card publication (for model builders)

HITRUST CSF requirement statement [?] (06.10hAISecSystem.4)

The organization publishes model cards for the AI models it produces, which (minimally) include the following elements: 
(1) model details;
(2) intended use;
(3) training details (e.g., data, methodology); 
(4) evaluation details (e.g., methodology, metrics, outcomes); and 
(5) usage caveats (such as potential bias, risks, or limitations) and associated recommendations.

Evaluative elements in this requirement statement [?]
1. The organization publishes model cards for the AI models it produces, which includes model details.
2. The organization publishes model cards for the AI models it produces, which includes intended use.
3. The organization publishes model cards for the AI models it produces, which includes training details (e.g., data, methodology).
4. The organization publishes model cards for the AI models it produces, which includes evaluation details (e.g., methodology, metrics, outcomes).
5. The organization publishes model cards for the AI models it produces, which includes caveats (such 
as potential bias, risks, or limitations) and associated recommendations.


Illustrative procedures for use during assessments [?]

  • Policy: Examine policies related to each evaluative element within the requirement statement. Validate the existence of a written or undocumented policy as defined in the HITRUST scoring rubric.

  • Procedure: Examine evidence that written or undocumented procedures exist as defined in the HITRUST scoring rubric. Determine if the procedures and address the operational aspects of how to perform each evaluative element within the requirement statement.

  • Implemented: Examine evidence that all evaluative elements within the requirement statement have been implemented as defined in the HITRUST scoring rubric, using a sample based test where possible for each evaluative element. Example test(s):
    • For example, review the internally produced AI models deployed in in-scope AI systems and confirm the organization published a model card for each which included the elements outlined in this requirement statement.

  • Measured: Examine measurements that formally evaluate and communicate the operation and/or performance of each evaluative element within the requirement statement. Determine the percentage of evaluative elements addressed by the organization’s operational and/or independent measure(s) or metric(s) as defined in the HITRUST scoring rubric. Determine if the measurements include independent and/or operational measure(s) or metric(s) as defined in the HITRUST scoring rubric. Example test(s):
    • For example, measures indicate percentage of the organization’s internally produced AI models that feature a published model card relative to the total number of the organization’s internally produced AI models. Reviews, tests, or audits are completed by the organization to measure the effectiveness of the implemented controls and to confirm that all internally produced AI models feature published model cards inclusive of the elements outlined in this requirement statement.

  • Managed: Examine evidence that a written or undocumented risk treatment process exists, as defined in the HITRUST scoring rubric. Determine the frequency that the risk treatment process was applied to issues identified for each evaluative element within the requirement statement.

Placement of this requirement in the HITRUST CSF [?]

  • Assessment domain: 06 Configuration Management
  • Control category: 10.0 – Information Systems Acquisition, Development, and Maintenance
  • Control reference: 10h- Control of Operational Software

Specific to which parts of the overall AI system? [?]
AI platform layer:
  • The deployed AI model

Discussed in which authoritative AI security sources? [?]
  • Generative AI framework for HM Government
    2023, Central Digital and Data Office, UK Government
    • Where:
      • Using generative AI safely and responsibly > Ethics > Transparency and explainability > Practical recommendations > Bullet #4

Control functions against which AI security threats? [?]
Additional information
  • Q: When will this requirement included in an assessment? [?]
    • This requirement will always be added to HITRUST assessments which include the
      Security for AI systems regulatory factor.
    • Also, this requirement is only applicable to machine learning-based AI models (i.e., generative and predictive AI).

  • Q: Will this requirement be externally inheritable? [?] [?]
    • No. However, this requirement is applicable only to model builders.