HITRUST AI Security Assessment and Certification Specification

Humans can intervene if needed

HITRUST CSF requirement statement [?] (12.09abAISecSystem.3)

The design of the AI application allows its human operators the ability to 
(1) evaluate AI model outputs before relying on them and 
(2) intervene in AI model-initiated actions (e.g., sending emails, modifying records) if 
deemed necessary.

Evaluative elements in this requirement statement [?]
1. The design of the AI system allows its human operators the ability to evaluate AI 
model outputs before relying on.
2. The design of the AI system allows its human operators the ability to intervene in 
AI model-initiated actions (e.g., sending emails, modifying records) if deemed necessary.


Illustrative procedures for use during assessments [?]

  • Policy: Examine policies related to each evaluative element within the requirement statement. Validate the existence of a written or undocumented policy as defined in the HITRUST scoring rubric.

  • Procedure: Examine evidence that written or undocumented procedures exist as defined in the HITRUST scoring rubric. Determine if the procedures and address the operational aspects of how to perform each evaluative element within the requirement statement.

  • Implemented: Examine evidence that all evaluative elements within the requirement statement have been implemented as defined in the HITRUST scoring rubric, using a sample based test where possible for each evaluative element. Example test(s):
    • For example, review the AI application to confirm it allows a human operator to evaluate AI model outputs. Further, confirm the ability for human operators to intervene in AI model-initiated actions when necessary.

  • Measured: Examine measurements that formally evaluate and communicate the operation and/or performance of each evaluative element within the requirement statement. Determine the percentage of evaluative elements addressed by the organization’s operational and/or independent measure(s) or metric(s) as defined in the HITRUST scoring rubric. Determine if the measurements include independent and/or operational measure(s) or metric(s) as defined in the HITRUST scoring rubric. Example test(s):
    • For example, measures indicate if the AI application allows human operators to evaluate AI model outputs and intervene in AI model-initiated actions when necessary. Reviews, tests, or audits are completed by the organization to measure the effectiveness of the implemented controls.

  • Managed: Examine evidence that a written or undocumented risk treatment process exists, as defined in the HITRUST scoring rubric. Determine the frequency that the risk treatment process was applied to issues identified for each evaluative element within the requirement statement.

Placement of this requirement in the HITRUST CSF [?]

  • Assessment domain: 12 Audit Logging & Monitoring
  • Control category: 09.0 – Communications and Operations Management
  • Control reference: 09.ab – Monitoring System Use

Specific to which parts of the overall AI system? [?]
  • AI application layer:
    • Application AI safety and security systems

Discussed in which authoritative AI security sources? [?]
  • OWASP 2023 Top 10 for LLM Applications
    Oct. 2023, © The OWASP Foundation
    • Where:
      • LLM01: Prompt injection > Prevention and mitigation strategies > Bullet #2
      • LLM03: Training data poisoning > Prevention and mitigation strategies > Bullet #7
      • LLM07: Insecure plugin design > Prevention and mitigation strategies > Bullet #6
      • LLM08: Excessive agency > Prevention and mitigation strategies > Bullet #6
      • LLM09: Overreliance > Prevention and mitigation strategies > Bullet #1

  • Generative AI framework for HM Government
    2023, Central Digital and Data Office, UK Government
    • Where:
      • Building generative AI solutions > Building the solution > Getting reliable results > Bullet 9
      • Building generative AI solutions > Building the solution > Getting reliable results > Bullet 10
      • Using generative AI safely and responsibly > Ethics > Accountability and responsibility > Practical recommendations > Bullet 6
      • Using generative AI safely and responsibly > Ethics > Accountability and responsibility > Practical recommendations > Bullet 5
      • Using generative AI safely and responsibly > Ethics > Maintaining appropriate human involvement in automated processes > Bullet 3
      • Using generative AI safely and responsibly > Ethics > Maintaining appropriate human involvement in automated processes > Bullet 5
      • Building generative AI solutions > Building the solution > Data management > Practical recommendations > Bullet 2

Discussed in which commercial AI security sources? [?]

  • Databricks AI Security Framework
    Sept. 2024, © Databricks
    • Where:
      • DASF 29: Build MLOps workflows with human-in-the-loop (HILP) with permissions, versions and approvals to promote models to production

  • Snowflake AI Security Framework
    2024, © Snowflake Inc.
    • Where:
      • Prompt injection > Mitigations > Human-in-the-loop systems
      • Indirect prompt injection > Mitigations > Human-in-the-loop systems

Control functions against which AI security threats? [?]
Additional information
  • Q: When will this requirement included in an assessment? [?]
    • This requirement will always be added to HITRUST assessments which include the
      Security for AI systems regulatory factor.
    • No other assessment tailoring factors affect this requirement.

  • Q: Will this requirement be externally inheritable? [?] [?]
    • No. Implementing and/or configuring this requirement is the AI application provider’s sole responsibility.