HITRUST AI Security Assessment and Certification Specification

Monitoring for data, models, and configs for suspicious changes

HITRUST CSF requirement statement [?] (12.09abAISecSystem.5)

The organization performs monitoring, on at least a monthly basis, for suspicious manipulation of 
(1) AI-related datasets;
(2) AI models;
(3) AI-relevant code (e.g., code used to create, train, and/or deploy AI models, code of
language model tools such as agents and plugins); and
(4) AI-relevant configurations (e.g., metaprompts)
that might compromise the AI system's performance or security.

Evaluative elements in this requirement statement [?]
1. The organization performs monitoring, on at least a monthly basis, for suspicious
manipulation of AI datasets that might compromise the AI system's performance or security.
2. The organization performs monitoring, on at least a monthly basis, for suspicious 
manipulation of AI models that might compromise the AI system's performance or security.
3. The organization performs monitoring, on at least a monthly basis, for suspicious
manipulation of AI-relevant code (e.g., code used to create, train, and/or deploy AI models, 
code of language model tools such as agents and plugins) that might compromise the AI 
system's performance or security.
4. The organization performs monitoring, on at least a monthly basis, for suspicious
manipulation of AI-relevant configurations (e.g., metaprompts) that might compromise
the AI system's performance or security.


Illustrative procedures for use during assessments [?]

  • Policy: Examine policies related to each evaluative element within the requirement statement. Validate the existence of a written or undocumented policy as defined in the HITRUST scoring rubric.

  • Procedure: Examine evidence that written or undocumented procedures exist as defined in the HITRUST scoring rubric. Determine if the procedures and address the operational aspects of how to perform each evaluative element within the requirement statement.

  • Implemented: Examine evidence that all evaluative elements within the requirement statement have been implemented as defined in the HITRUST scoring rubric, using a sample based test where possible for each evaluative element. Example test(s):
    • For example, review the AI system to ensure the organization performs monitoring, on at least a monthly basis, for suspicious manipulation of AI-related datasets; AI models; AI-relevant code (e.g., code used to create, train, and/or deploy AI models, code of language model tools such as agents and plugins); and AI-relevant configurations (e.g., metaprompts) that might compromise the AI system’s performance or security.

  • Measured: Examine measurements that formally evaluate and communicate the operation and/or performance of each evaluative element within the requirement statement. Determine the percentage of evaluative elements addressed by the organization’s operational and/or independent measure(s) or metric(s) as defined in the HITRUST scoring rubric. Determine if the measurements include independent and/or operational measure(s) or metric(s) as defined in the HITRUST scoring rubric. Example test(s):
    • For example, measures indicate if the organization performs monitoring, on at least a monthly basis, for suspicious manipulation of AI-related datasets; AI models; AI-relevant code (e.g., code used to create, train, and/or deploy AI models, code of language model tools such as agents and plugins); and AI-relevant configurations (e.g., metaprompts) that might compromise the AI system’s performance or security.

  • Managed: Examine evidence that a written or undocumented risk treatment process exists, as defined in the HITRUST scoring rubric. Determine the frequency that the risk treatment process was applied to issues identified for each evaluative element within the requirement statement.

Placement of this requirement in the HITRUST CSF [?]

  • Assessment domain: 12 Audit Logging & Monitoring
  • Control category: 09.0 – Communications and Operations Management
  • Control reference: 09.ab – Monitoring System Use

Specific to which parts of the overall AI system? [?]
AI application layer:
  • AI plugins and agents
  • Application AI safety and security systems
  • The deployed AI application (Considered in the underlying HITRUST e1, i1, or r2 assessment)
AI platform layer
  • The AI platform and associated APIs (Considered in the underlying HITRUST e1, i1, or r2 assessment)
  • Model safety and security systems
  • The deployed AI model
  • Model engineering environment and model pipeline
  • AI datasets and data pipelines


Discussed in which authoritative AI security sources? [?]

Discussed in which commercial AI security sources? [?]
  • Databricks AI Security Framework
    Sept. 2024, © Databricks
    • Where:
      • DASF 14: Audit actions performed on datasets
      • DASF 23: Register, version, approve, promote, deploy and monitor models
      • DASF 55: Monitor audit logs

  • Snowflake AI Security Framework
    2024, © Snowflake Inc.
    • Where:
      • Backdooring models (insider attacks) > Mitigations > Data provenance and auditability

Control functions against which AI security threats? [?]
Additional information
  • Q: When will this requirement included in an assessment? [?]
    • This requirement will always be added to HITRUST assessments which include the
      Security for AI systems regulatory factor.
    • No other assessment tailoring factors affect this requirement.

  • Q: Will this requirement be externally inheritable? [?] [?]
    • Yes, partially. This may be a responsibility shared between an AI application provider and their AI platform provider (if used), performed independently on separate layers/components of the overall AI system.